Improving resilience of scientific software through a domain-specific approach
Not Available
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Not Available
We present our work on developing and training scalable, trustworthy, and energy-efficient predictive graph foundation models (GFMs) using HydraGNN, a multi-headed graph convolutional neural network architecture. HydraGNN expands the boundaries of graph neural network (GNN) computations in both training scale and data diversity. It abstracts over message passing algorithms, allowing both reproduction of and comparison across algorithmic innovations that define nearest-neighbor convolution in GNNs. This work discusses a series of optimizations that have allowed scaling up the GFMs training to tens of thousands of GPUs on datasets consisting of hundreds of millions of graphs. Our GFMs use multitask learning (MTL) to simultaneously learn graph-level and node-level properties of atomistic structures, such as energy and atomic forces. Using over 154 million atomistic structures for training, we illustrate the performance of our approach along with the lessons learned on two state-of-the-art US Department of Energy (US-DOE) supercomputers, namely the Perlmutter petascale system at the National Energy Research Scientific Computing Center and the Frontier exascale system at Oak Ridge Leadership Computing Facility. The HydraGNN architecture enables the GFM to achieve near-linear strong scaling performance using more than 2000 GPUs on Perlmutter and 16,000 GPUs on Frontier.
The landscape of High-Performance Computing (HPC) is changing as we enter the exascale era and power and energy management are key design points for any next generation of supercomputers. Efficiently utilizing procured power and optimizing performance of scientific applications under power and energy constraints is challenging due to several reasons including dynamic phase behavior, processor manufacturing variability and increasing heterogeneity of node-level components. Extending the scope from the node- and application-level up to the system-level introduces further challenges on top of power-unaware job scheduling, which is known to be NP-hard on its own. While there exists several individual efforts across the community to research automatic techniques for managing power and energy better, the majority of these techniques have been specialized to meet the needs of a specific HPC center or specific optimization goals and provide little support to connect them to each other. Some projects, most notably the PowerAPI efforts [1], discuss interfaces that form a good starting point for full stack integration, but these interfaces still need to be hooked up to the wide range of software components offered by academic partners, developers or vendors. Furthermore, a recent survey conducted by the EE HPC WG concluded that the majority of such techniques have lacked the application-awareness required to achieve the best system performance and throughput. Other observations were that each technique tended to target management of power and energy for a different subset of the site or system hardware and that the different techniques tended to perform management at different (and often conflicting) granularities. Unfortunately, the existing techniques have not been designed to coexist simultaneously on one site and cooperate on management in an integrated manner [2]. The lack of application-awareness, lack of coordinated management across different granularities, and the lack of widely accepted interfaces and consequent limited connectivity between modules, can result in substantially underutilized Watts and FLOPS. To address these gaps, the HPC community needs a holistic stack for power and energy management and none currently exists. In our view, a holistic stack is one that is extensible enough to support the present and anticipated future needs of various different HPC centers, one that achieves best system performance and throughput through application-awareness, one that is designed to coordinate management at different granularities, and one that enables the seamless integration of software components from different developers and vendors. In this seminar, our goal is bring together experts from academia, research laboratories, and industry in order to take stock of existing approaches, design points and power management concepts and requirements at the various participant’s institutions, and to design concepts for a holistic power and energy management stack, which we refer to as the HPC PowerStack. Further, we hope to take steps toward defining the interfaces necessary for providing a complete prototype shared among many groups. The intention is to align development and research efforts across the community so that we may share development resources, avoid duplicating effort, agree on common interfaces and reap the rewards together as a community. The intended final outcome of this collaboration will be a holistic, flexible and extensible concept of a software stack ecosystem that allows us to combine product-grade open-source software components to enable runtime optimization of system power, energy, and performance.
Here, this work extends our previous research entitled "MemXCT: Memory-centric X-ray CT Reconstruction with Massive Parallelization" that was originally published at SC19 conference (Hidayetoglu et al., 2019) with reproducibility of the computational imaging performance. X-ray computed tomography (XCT) is regularly used at synchrotron light sources to study the internal morphology of materials at high resolution. However, experimental constraints, such as radiation sensitivity, can result in noisy or undersampled measurements. Further, depending on the resolution, sample size and data acquisition rates, the resulting noisy dataset can be in the order of terabytes. Advanced iterative reconstruction techniques can produce high-quality images from noisy measurements, but their computational requirements have made their use an exception rather than the rule. We propose a novel memory-centric approach that avoids redundant computations at the expense of additional memory complexity. We develop a memory-centric iterative reconstruction system, MemXCT, that uses an optimized SpMV implementation with two-level pseudo-Hilbert ordering and multi-stage input buffering. We evaluate MemXCT on various supercomputer architectures involving KNL and GPU. MemXCT can reconstruct a large (11Kx11K) mouse brain tomogram in 10 seconds using 4096 KNL nodes (256K cores). The results presented in our original article at the SC19 were based on large-scale supercomputing resources. The MemXCT application was selected for the Student Cluster Competition (SCC) Reproducibility Challenge and evaluated on a variety of cloud computing resources by universities around the world in the SC20 conference. We summarize the results of the top-ranked SCC Reproducibility Challenge teams and identify the most pertinent measures for ensuring the reproducibility of our experiments in this article.
The central importance of large scale eigenvalue problems in scientific computation necessitates the development massively parallel algorithms for their solution. Recent advances in dense numerical linear algebra have enabled the routine treatment of eigenvalue problems with dimensions on the order of hundreds of thousands on the world's largest supercomputers. In cases where dense treatments are not feasible, Krylov subspace methods offer an attractive alternative due to the fact that they do not require storage of the problem matrices. However, demonstration of scalability of either of these classes of eigenvalue algorithms on computing architectures capable of expressing excessive parallelism is non-trivial due to communication requirements and serial bottlenecks, respectively. In this work, we introduce the SISLICE method: a parallel shift-invert algorithm for the solution of the symmetric self-consistent field (SCF) eigenvalue problem. The SISLICE method drastically reduces the communication requirement of current parallel shift-invert eigenvalue algorithms through various shift selection and migration techniques based on density of states estimation and k-means clustering, respectively. This work demonstrates the robustness and parallel performance of the SISLICE method on a representative set of SCF eigenvalue problems and outlines research directions which will be explored in future work.
The central importance of large-scale eigenvalue problems in scientific computation necessitates the development of massively parallel algorithms for their solution. Recent advances in dense numerical linear algebra have enabled the routine treatment of eigenvalue problems with dimensions on the order of hundreds of thousands on the world’s largest supercomputers. In cases where dense treatments are not feasible, Krylov subspace methods offer an attractive alternative due to the fact that they do not require storage of the problem matrices. However, demonstration of scalability of either of these classes of eigenvalue algorithms on computing architectures capable of expressing massive parallelism is non-trivial due to communication requirements and serial bottlenecks, respectively. In this work, we introduce the SISLICE method: a parallel shift-invert algorithm for the solution of the symmetric self-consistent field (SCF) eigenvalue problem. The SISLICE method drastically reduces the communication requirement of current parallel shift-invert eigenvalue algorithms through various shift selection and migration techniques based on density of states estimation and k-means clustering, respectively. This work demonstrates the robustness and parallel performance of the SISLICE method on a representative set of SCF eigenvalue problems and outlines research directions that will be explored in future work.
Energy consumption is one of the top challenges for achieving the next generation of supercomputing. Codesign of hardware and software is critical for improving energy efficiency (EE) for future large-scale systems. Many architectural power-saving techniques have been developed, and most hardware components are approaching physical limits. Accordingly, parallel computing software, including both applications and systems, should exploit power-saving hardware innovations and manage efficient energy use. In addition, new power-aware parallel computing methods are essential to decrease energy usage further. This article surveys software-based methods that aim to improve EE for parallel computing. It reviews the methods that exploit the characteristics of parallel scientific applications, including load imbalance and mixed precision of floating-point (FP) calculations, to improve EE. In addition, this article summarizes widely used methods to improve power usage at different granularities, such as the whole system and per application. In particular, it describes the most important techniques to measure and to achieve energy-efficient usage of various parallel computing facilities, including processors, memories, and networks. Overall, this article reviews the state-of-the-art of energy-efficient methods for parallel computing to motivate researchers to achieve optimal parallel computing under a power budget constraint.
At the request of the U.S. Department of Energy's (DOE) Office of Science (SC) Advanced Scientific Computing Research (ASCR) program office, a workshop was held June 2-3, 2015, in Gaithersburg, MD, to identify potential long term (10 to +20 year) cybersecurity fundamental basic research and development challenges, strategies and roadmap facing future high performance computing (HPC), networks, data centers, and extreme-scale scientific user facilities. This workshop was a follow-on to the workshop held January 7-9, 2015, in Rockville, MD, that examined higher level ideas about scientific computing integrity specific to the mission of the DOE Office of Science. Issues included research computation and simulation that takes place on ASCR computing facilities and networks, as well as network-connected scientific instruments, such as those run by various DOE Office of Science programs. Workshop participants included researchers and operational staff from DOE national laboratories, as well as academic researchers and industry experts. Participants were selected based on the submission of abstracts relating to the topics discussed in the previous workshop report [1] and also from other ASCR reports, including "Abstract Machine Models and Proxy Architectures for Exascale Computing" [27], the DOE "Preliminary Conceptual Design for an Exascale Computing Initiative" [28], and the January 2015 machine learning workshop [29]. The workshop was also attended by several observers from DOE and other government agencies. The workshop was divided into three topic areas: (1) Trustworthy Supercomputing, (2) Extreme-Scale Data, Knowledge, and Analytics for Understanding and Improving Cybersecurity, and (3) Trust within High-end Networking and Data Centers. Participants were divided into three corresponding teams based on the category of their abstracts. The workshop began with a series of talks from the program manager and workshop chair, followed by the leaders for each of the three topics and a representative of each of the four major DOE Office of Science Advanced Scientific Computing Research Facilities: the Argonne Leadership Computing Facility (ALCF), the Energy Sciences Network (ESnet), the National Energy Research Scientific Computing Center (NERSC), and the Oak Ridge Leadership Computing Facility (OLCF). The rest of the workshop consisted of topical breakout discussions and focused writing periods that produced much of this report.
Recent studies have revealed that among all the tropical oceans, the tropical Atlantic has experienced the most pronounced warming trend over the 20th century. Many extreme climate events affecting the U.S., such as hurricanes, severe precipitation and drought events, are influenced by conditions in the Gulf of Mexico and the Atlantic Ocean. It is therefore imperative to have accurate simulations of the climatic mean and variability in the Atlantic region to be able to make credible projections of future climate change affecting the U.S. and other countries adjoining the Atlantic Ocean. Unfortunately, almost all global climate models exhibit large biases in their simulations of tropical Atlantic climate. The atmospheric convection simulation errors in the Amazon region and the associated errors in the trade wind simulations are hypothesized to be a leading cause of the tropical Atlantic biases in climate models. As global climate models have resolutions that are too coarse to resolve some of the atmospheric and oceanic processes responsible for the model biases, we propose to use a high-resolution coupled regional climate model (CRCM) framework to address the tropical bias issue. We propose to combine the expertise in tropical coupled atmosphere-ocean modeling at Texas A&M University (TAMU) and the coupled land-atmosphere modeling expertise at Pacific Northwest National Laboratory (PNNL) to develop a comprehensive CRCM for the Atlantic sector within a general and flexible modeling framework. The atmospheric component of the CRCM will be the NCAR WRF model and the oceanic component will be the Rutgers/UCLA ROMS. For the land component, we will use CLM modified at PNNL to include more detailed representations of vegetation and soil hydrology processes. The combined TAMU-PNNL CRCM model will be used to simulate the Atlantic climate, and the associated land-atmosphere-ocean interactions at a horizontal resolution of 9 km or finer. A particular focus of the model development effort will be to optimize the performance of WRF and ROMS over several thousand of cores by focusing on both the parallel communication libraries and the I/O interfaces, in order to achieve the sustained throughput needed to perform simulations on such fine resolution grids. The CRCM model will be developed within the framework of the Coupler (CPL7) software that is part of the NCAR Community Earth System Model (CESM). Through efforts at PNNL and within the community, WRF and CLM have already been coupled via CPL7. Using the flux coupler approach for the whole CRCM model will allow us to flexibly couple WRF, ROMS, and CLM with each model running on its own grid at different resolutions. In addition, this framework will allow us to easily port parameterizations between CESM and the CRCM, and potentially allow partial coupling between the CESM and the CRCM. TAMU and PNNL will contribute cooperatively to this research endeavor. The TAMU team led by Chang and Saravanan has considerable experience in studying atmosphere-ocean interactions within tropical Atlantic sector and will focus on modeling issues that relate to coupling WRF and ROMS. The PNNL team led by Leung has extensive expertise in atmosphere-land interaction and will be responsible for improving the land surface parameterization. Both teams will jointly work on integrating WRF-ROMS and WRF-CLM to couple WRF, ROMS, and CLM through CPL7. Montuoro of the TAMU Supercomputing Center will be responsible for improving the MPI and Parallel IO interfaces of the CRCM. Both teams will contribute to the design and execution of the proposed numerical experiments and jointly perform analysis of the numerical experiments.
Given the tremendous volume of accessible Earth Observation (EO) data, there is a need to develop scalable Geospatial Artificial Intelligence (GeoAI) solutions for time-sensitive applications. Scalability in this context refers to rapidly processing large-scale EO data using high performance computing resources. Accurate mapping of the built environment from remote sensing (RS) imagery has been one of the crucial components in GeoAI workflows for a wide spectrum of humanitarian applications. Derived vector data of built environment is often leveraged for disaster preparedness and response activities. However, factors such as differences in ortho-rectification, atmospheric conditions and human error, results in spatial misalignment between vector data and the timely available RS imagery. Model training for downstream tasks such as object detection, change analysis, etc., is negatively impacted due to such spatial misalignment. Although there has been progress towards automatic alignment of vector data, the lack of scalability remains an open research challenge. This paper proposes to leverage parallel computing to optimize an automatic vector data alignment workflow. It further employs CPU-level multi-core parallelism for improving the performance of the workflow for scalable built environment mapping. We report observations and discuss findings from the preliminary experiments performed on the Summit Supercomputer.
The US Department of Energy’s (DOE’s) Oak Ridge National Laboratory (ORNL) is a leading institution in advanced materials, supercomputing, neutrons, and nuclear science. As a research laboratory managed by UT-Battelle, LLC for DOE, ORNL has national priorities in energy, security, and scientific discovery that necessitate facility improvements and expansions. DOE is also committed to environmental stewardship. The laboratory is located on the ~32,000-acre (~13,000-ha) Oak Ridge Reservation (ORR), much of which is categorized as a National Environmental Research Park (NERP) and a state Wildlife Management Area. DOE works with the Tennessee Wildlife Resources Agency (TWRA), Tennessee Department of Environment and Conservation (TDEC), US Fish and Wildlife Service (USFWS), US Department of Agriculture, and other agencies to serve as an effective steward of the ORR. Accordingly, project managers must conform to environmental regulations, agreements, and policies at the federal, state, and institutional levels. Per 40 CFR (Code of Federal Regulations) 1508.14, potential effects on research and science education also represent potential effects of federal actions on the NERP, and impacts on, e.g., deer harvest, must be considered on the Oak Ridge Wildlife Management Area when other aspects of the human environment are affected. The United States currently has no fast neutron testing capability to support advanced nuclear research and development. The proposed Versatile Test Reactor (VTR) will take advantage of current investments by the US government and private industry in nuclear reactors to expedite the design and construction process, using proven technology to create a world-class scientific infrastructure. The VTR will take advantage of fast neutrons provided by this proven technology, along with a capability to rapidly insert, conduct, and remove state-of-the-art experiments. An advantage of the VTR is that it can support future innovations in experimental capabilities without modifying the facility. The VTR will support progress in a variety of science and technology areas, including testing and qualification of advanced reactor fuels; testing and qualification of innovative structural materials; testing of innovative components and instruments; validation of advanced modeling and simulation tools; and versatility for future technical missions. Through proven technology, the VTR can take advantage of existing reactor designs and operating experience to reduce the risk, cost, and time for design and construction. The top available resources of DOE laboratories, industry, and universities will be used to expedite reactor design and construction toward developing the scientific infrastructure that affords a strong testing capability that can be sustained over many years. This report summarizes current knowledge of natural and cultural resources primarily within the VTR construction area. At the time of this report, the proposed VTR site design includes a construction area of ~150.4 acres (~69.9 ha), which contains an ~51.3-acre (~20.8-ha) operations area, located within forested natural areas of the ORR. The primary goal of the work presented here was to evaluate potential effects on sensitive resources that might result from development and construction activities associated with VTR. In addition to on-the-ground surveys during spring and summer 2020 by the ORNL Natural Resources Management Program and Aquatic Ecology Group staff, this report makes use of historical (pre-1995) and contemporary (1995 to present) data from additional confirmed sources (e.g., TDEC). Likewise, forest conditions were compiled from a 2011 forest inventory and supplemented with limited ground observations in 2020. The individuals who obtained and compiled the data presented here are familiar with and routinely assess sensitive resources on the ORR. Anyone who references this report must consider that the timing of surveys did not permit a complete delineation of the resources that will be affected. If the VTR project proceeds, additional surveys will be required to account for the seasonal patterns of various threatened and endangered species. Data deficiencies and potential resources that likely went undetected are indicated where possible. Accordingly, this report should facilitate more environmentally sound decisions during planning and 2 development of the VTR site, provide a foundation for further assessment of sensitive and cultural resources, and help project managers better address regulatory guidance and DOE policies on sustainable development in compliance with, for example, the US Endangered Species Act (ESA), Migratory Bird Treaty Act (MBTA), Tennessee Rare Plant Protection and Conservation Act of 1985, Tennessee Nongame and Endangered or Threatened Wildlife Species Conservation Act of 1974, several federal and state regulations regarding aquatic resource protection, and site-specific policies as outlined in various ORR management plans developed by ORNL and TWRA for DOE.
Dust aerosols have a wide size distribution from less than 0.1 to over 100 μm and dominate Earth's atmospheric aerosol mass. However, most Earth system models (ESMs) inadequately represent dust aerosols larger than 10 μm in diameter, limiting the accuracy of the simulated dust cycle and climate impacts. Here, we introduce a new modeling framework that captures the full observed size distribution of dust aerosols, incorporating recent advances into a mineral-resolved version of the Community ESM, while addressing known issues in previous versions. Comprehensive evaluation against diverse observations of bulk dust and component minerals demonstrates that the model reproduces the observed dust cycle across particle sizes. Incorporating the previously unrepresented large-dust fractions substantially alters dust budget estimates, highlighting potential changes in simulated climate impacts and underscoring the importance of comprehensive size-resolved dust modeling. Despite these advancements, uncertainties persist. Our results indicate that a size-dependent reduction in settling velocity is required to reproduce the observed dust size distribution downwind of source regions. Specifically, in the new model, the gravitational settling velocity of dust particles larger than 10 μm in diameter must be reduced by as much as 85% to achieve agreement with observations. This empirical reduction serves as a constraint on physics-based models of dust settling. Future developments should address misrepresented physical processes that hinder accurate modeling of the large dust aerosol transport. Expanding observational data sets covering the full-size distribution is also essential to better constrain the dust cycle and improve the representation of dust optical properties and climate effects.
The quantum many-body problem is a great unsolved, cross-cutting challenge in physics that is of fundamental importance. Our understanding of phenomena related to dense quark matter, in particular, is challenged by this practical intractability of classical simulations. Because of the sheer cost and challenge of performing experiments that probe the length and energy scales relevant to such physics, there are practical motivations for finding theoretical methods to address the many-body problem. One promising approach is based on the use of a programmable and controllable analog quantum systems to emulate the physics of many-body problems of interest. While there are currently broad efforts to develop mid- to large-scale quantum computers, we are still likely many years away from such devices outperforming classical supercomputers for useful calculations. Even though digital quantum computers are still at a stage too premature for such tasks, approaches based on analog quantum simulators have advanced rapidly over the past two decades and can now treat many-body problems of interest on small- to medium-scale systems. Tackling the challenging many-body problems of relevance to high energy physics represents one of the great new opportunities and challenges for the field of analog quantum simulation. To address this challenge, this award brought together an interdisciplinary team of physicists with expertise in high energy theory (El-Khadra and Draper), computational and condensed matter physics (Clark), and experimental quantum and atomic, molecular, optical (AMO) science (DeMarco and Gadway), with the goal of developing realistic and optimized strategies for the analog quantum simulation of lattice gauge theories using cold atoms and molecules. This track 1 project included both theoretical and experimental goals, stated as follows. The theory goals were to "develop and verify a novel approach to quantum emulation of (1+1)d quantum link models, based on arrays of trapped quantum spins with long-range interactions." This final project report details the progress made under this award, as well as new research directions developed under this award.
The many-body simulation of quantum systems is an active field of research that involves several different methods targeting various computing platforms. Many methods commonly employed, particularly coupled cluster methods, have been adapted to leverage the latest advances in modern high-performance computing. Selected configuration interaction (sCI) methods have seen extensive usage and development in recent years. However, the development of sCI methods targeting massively parallel resources has been explored only in a few research works. Here, we present a parallel, distributed memory implementation of the adaptive sampling configuration interaction approach (ASCI) for sCI. In particular, we will address the key concerns pertaining to the parallelization of the determinant search and selection, Hamiltonian formation, and the variational eigenvalue calculation for the ASCI method. Load balancing in the search step is achieved through the application of memory-efficient determinant constraints originally developed for the ASCI-PT2 method. The presented benchmarks demonstrate near optimal speedup for ASCI calculations of Cr 2 (24e, 30o) with 10 6 , 10 7 , and 3 × 10 8 variational determinants on up to 16 384 CPUs. Importantly, to the best of the authors’ knowledge, this is the largest variational ASCI calculation to date.
Finding actionable insights from data has always been difficult. As the scale and forms of data increase tremendously, the task of finding value becomes even more challenging. Data scientists at Oak Ridge National Laboratory are leveraging unique leadership infrastructure (e.g. Urika-XA and Urika-GD appliances) to develop scalable algorithms for semantic, logical and statistical reasoning with unstructured Big Data. We present the deployment of such a framework called ORiGAMI (Oak Ridge Graph Analytics for Medical Innovations) on the National Library of Medicine s SEMANTIC Medline (archive of medical knowledge since 1994). Medline contains over 70 million knowledge nuggets published in 23.5 million papers in medical literature with thousands more added daily. ORiGAMI is available as an open-science medical hypothesis generation tool - both as a web-service and an application programming interface (API) at http://hypothesis.ornl.gov . Since becoming an online service, ORIGAMI has enabled clinical subject-matter experts to: (i) discover the relationship between beta-blocker treatment and diabetic retinopathy; (ii) hypothesize that xylene is an environmental cancer-causing carcinogen and (iii) aid doctors with diagnosis of challenging cases when rare diseases manifest with common symptoms. In 2015, ORiGAMI was featured in the Historical Clinical Pathological Conference in Baltimore as a demonstration of artificial intelligence to medicine, IEEE/ACM Supercomputing and recognized as a Centennial Showcase Exhibit at the Radiological Society of North America (RSNA) Conference in Chicago. The final paper will describe the workflow built for the Cray Urika-XA and Urika-GD appliances that is able to reason with the knowledge of every published medical paper every time a clinical researcher uses the tool.
Quantum computing stands at the threshold of a transformative decade, where the field will evolve from small-scale demonstrations toward practical scientific computing at scale. This Blueprint identifies fault-tolerant quantum computers (FTQCs) as a viable, scalable, and broadly applicable path to achieving “quantum scientific utility,” defined as solving scientifically valuable problems beyond the reach of conventional, classical computers. This capability is expected to show scientific demonstrations in the late 2020s and to mature in the early-to-mid 2030s. This Blueprint outlines a strategy to prepare the U.S. Department of Energy (DOE) for FTQCs and their integration into the U.S. national scientific computing infrastructure. Its purpose is to identify the steps, milestones, and research directions necessary for DOE to enable initial deployment of FTQCs in 2028 as a scientific tool for the nation and mature this capability into the 2030s. DOE has a long history of supporting quantum information science and technology, contributing significantly to research advancements, training a quantum-ready workforce, and providing access to early small-scale quantum hardware. Given recent demonstrations of logical operations on error-corrected logical qubits and the advancement of commercial hardware roadmaps, DOE should begin preparations for large-scale, fault-tolerant quantum computing deployment for DOE science missions. This Blueprint proposes that DOE focus on (1) deploying first-generation scientifically relevant quantum computers with at least 100 logical qubits and performing at least 10,000 to 100,000 hard logical operations in scientifically relevant calculations; (2) developing essential FTQC programming competencies, system software, and facility readiness; and (3) investing in cutting edge focused R&D that fosters breakthroughs in scientific applications, algorithms, and logical architectures needed to accelerate the advent of scientific utility. This effort will position DOE to transition to larger systems: production-scale quantum computers that comprise 1,000 to 10,000 logical qubits, perform 1 to 10 billion hard logical operations, and execute scientifically useful computations at scale. Achieving these goals will require DOE facilities to evolve with urgency to support scientific campaigns that integrate quantum and classical computing resources into efficient workflows, novel software and firmware environments for compiling and routing quantum programs on FTQC machines, and suitable infrastructure for quantum hardware. It will also require further development and optimization of scientific applications from the fields of materials science, quantum chemistry, and high-energy and nuclear physics. The Blueprint calls for transformative R&D and collective action to accelerate the advent of scientific quantum utility and bring it within reach by 2028.
The United States (U.S.) is no longer the unambiguous leader in the vitally important field of high performance computing (HPC). Japan, the European Union (EU), and China have fielded systems that are on par with our fastest supercomputers. The supply chain for everything from semiconductors to scientific software is globally distributed. Yet our economic future and security depend critically on our ability to innovate faster than our competitors, and the speed of innovation depends increasingly on large-scale computational science and engineering and thus HPC. How should the United States respond to this challenge? This report seeks to initiate a new and potentially transformative national discussion on this vital question. The Department of Energy’s (DOE) Advanced Scientific Computing Research (ASCR) program is well-positioned to make informed, targeted decisions about where the United States should cooperate and where it should compete in the global market for scientific exploration and discovery. By setting its sights on problems critical to our nation and the world, by establishing productive new collaborations, and by making strategic investments, ASCR can restore and maintain U.S. scientific leadership in the critical areas described in this report while strengthening our research infrastructure and training a large, diverse cohort of scientists. In doing so, ASCR and its scientists will pave the way for a secure and prosperous future for America. For more than 30 years, the ASCR program has provided the HPC and networking capabilities and expertise needed to support DOE’s mission to advance the national, economic, and energy security of the United States. The program now faces the challenge of developing and deploying the next generation of HPC systems and technologies, as well as supporting the application of HPC and artificial intelligence (AI) technologies to a wide range of scientific and engineering research problems. Through its research and development efforts, the ASCR program must also advance the state of the art in HPC and accelerate the pace of scientific discovery and technological innovation. Fulfilling this promise will require significantly increased investments, as well as innovative policies and programs. This subcommittee is aware that we are making recommendations and calls for action at a time when federal resources are limited. We understand that a wide range of competing priorities must be balanced by the nation’s leaders and that there is a need to leverage resources in new ways and seek efficiencies in facilities and operations. However, we must not let these realities limit our imagination or silence our advocacy. The ASCR program is a key part of the U.S. research infrastructure and an important component of economic growth and U.S. competitiveness. ASCR has a responsibility to pursue its mission, including advanced scientific computing, applications of AI technologies, and the required advanced research facilities, with determination and enthusiasm. To fulfill the scientific enterprise’s responsibility to the nation, the ASCR program must not only develop and publish a clear vision with an associated list of goals, priorities, and recommendations but also demonstrate scientific leadership by consistently securing long-term funding. This will allow the program to build on its achievements to date, to realize its ambitious vision, and to make lasting contributions to the field.
In the push towards exascale computing, the limiting factors are power requirements and heat dissipation. Using current technology, the power required to drive an exascale supercomputer would near 400-MW. Clearly, future computing hardware will have to be more energy efficient in order to achieve exascale performance. Most of the current research in advanced computing architectures is in the use of standard CPUs with accelerators. However, these accelerators are not generally supported by the traditional scientific programming languages (C, C++, and Fortran) and the scientific software must be tailored specifically to take advantage of the accelerators. Large scale scientific software (hundreds-of-thousand to millions of lines of code) can not be easily ported to the computing platform du jour. The widespread use of low power ARM based processors in the mobile computing market offer a low power alternative to accelerators that are also supported by the traditional scientific programming languages and programming paradigms such as MPI and OpenMP. Computing architectures for scientific computing are often assessed using standard benchmarks such as LINPACK benchmark or using suites of mini-applications that are greatly simplified stand-ins for scientific software. Often the real applications that will deployed on the scientific supercomputers are not ported or tested until delivery of the machines. In order to evaluate the use of ARM processors for scientific computing the Monte Carlo particle transport code, MCNP version 6 was compiled on the ARM Cortex-A8 and a set of benchmark problems were used to assess performance in comparison to an Intel Xeon E5-2670 and the low power Intel Atom N270. MCNP is a continuous energy Monte Carlo particle transport code in development at Los Alamos National Laboratory since 1977. MCNP is used to transport neutrons, photon, electrons, and charged particles using continuous energy interaction data. It is commonly used in fields of nuclear power, nuclear medicine, radiation protection, and high-energy physics through-out the world. MCNP version 6 is approximately 450 thousand lines of code, written primary in Fortran90, with some C and C++. MCNP supports distributed multiprocessing using MPI and threading with OpenMP. The ARM Cortex-A8 executes MCNP6 using about twice as much power as the Intel Xeon E5-2670 and about the same amount of power as the Intel Atom N270. However, these results are encouraging for the use of low-power ARM chips in future supercomputers. The ARM Cortex-A8 is a 32-bit CPU and it's NEON general-purpose SIMD (single-instruction multiple data) engine does not support double-precision floating-point arithmetic. More recent ARM processors (ARM Cortex-A53 and Cortex-A57) support 64-bit addressing and have NEON co-processors that support double-precision floating-point arithmetic. ARM claims that the A50 series delivers a performance increase of 15 times the Cortex-A8 performance for similar power usage. (authors)