Search NASA⌕ Search

SEARCH · Search NASA

Results for “EXASCALE COMPUTING PROJECT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Scalable Analysis Methods and In Situ Infrastructure for Extreme Scale Knowledge Discovery

High performance computers have for many years been on a trajectory that gives them extraordinary compute power with the addition of more and more compute cores. At the same time, other system parameters such as the amount of memory per core and bandwidth to storage have remained constant or have barely increased. This creates an imbalance in the computer, giving it the ability to compute a lot of data that it cannot reasonably save out due to time and storage constraints. While technologies have been invented to mitigate this problem (burst buffers, etc.), software has been adapting to employ in situ libraries which perform data analysis and visualization on simulation data while it is still resident in memory. This avoids the need to ever have to pay the costs of writing many terabytes of data files. Instead, in situ enables the creation of more concentrated data products such as statistics, plots, and data extracts, which are all far smaller than the full-sized volume data. With the increasing popularity of in situ, multiple in situ infrastructures have been created, each with its own mechanism for integrating with a simulation. To make it easier to instrument a simulation with multiple in situ infrastructures and include custom analysis algorithms, this project created the SENSEI framework.

97 MATHEMATICS AND COMPUTING↗

Real Time Simulation of Power Grid Disruptions

DOE-OE and DOE-SC workshops (Reference 1-3) identified the key power grid problem that requires insight addressable by the next generation of exascale computing is coupling of real-time data streams (1-2 TB per hour) as the streams are ingested to dynamic models. These models would then identify predicted disruptions in time (2-4 seconds) to trigger the smart grid s self healing functions. This project attempted to establish the feasibility of this approach and defined the scientific issues, and demonstrated example solutions to important smart grid simulation problems. These objectives were accomplished by 1) using the existing frequency recorders on the national grid to establish a representative and scalable real-time data stream; 2) invoking ORNL signature identification algorithms; 3) modeling dynamically a representative region of the Eastern interconnect using an institutional cluster, measuring the scalability and computational benchmarks for a national capability; and 4) constructing a prototype simulation for the system s concept of smart grid deployment. The delivered ORNL enduring capability included: 1) data processing and simulation metrics to design a national capability justifying exascale applications; 2) Software and intellectual property built around the example solutions; 3) demonstrated dynamic models to design few second self-healing.

Chinthavali, Supriya↗

Efficiently resolving the terrestrial-aquatic interface in E3SM with sub-grid methods to improve coastal simulations (Final Technical Report)

The broad goal of this project was to extend the capabilities of MPAS‐Ocean, the ocean component of DOE’s Energy Exascale Earth System Model (E3SM), to better resolve the hydrodynamics of the terrestrial‐aquatic interface without significantly increasing computing resource requirements. Specifically, our goal was to better resolve the small-scale features across the terrestrial‐aquatic interface which includes the low lying coastal floodplain and a dendritic network of hydraulically efficient channels connected to estuarine systems and back bays, barrier islands and high energy inlets. The processes driving flow across the terrestrial‐aquatic interface include upland and floodplain hydrology, tides, wind and atmospheric pressure (including coastally generated storm surge across continental shelves and within estuaries and bays), wind waves and water temperature and salinity gradients.

58 GEOSCIENCES↗

Co-design in ACES and exascale.

The Alliance for Computing at the Extreme Scale (ACES) is a Los Alamos and Sandia collaboration encompassing not only HPC procurements and operations, but also computer science and architecture research and development. One area of focus within ACES relates to the critical technology developments for future high performance computing systems and the applications that would run on them, and ACES is heavily involved in the proposed DOE Exascale Initiative. The proposed Exascale Initiative emphasizes the need for co-design, which is the three-way collaborative and concurrent design of HPC hardware, software, and the applications themselves. Transformational changes will occur not only in HPC hardware, but also in the applications space, and taken together these will require transformation changes in the overall software layers supporting the programming models, tools, runtimes, file systems, and operating systems. Co-design involving all three areas of hardware, software, and applications will be the key to success. This talk will outline key aspects of the Exascale Initiative and its emphasis on co-design. It will provide some examples from LANL & Sandia experiences in co-design including aspects of the innovative Roadrunner architecture and software, a ACES-Cray project studying advanced interconnects within Cray, and Sandia's work in computer system simulators and mini-applications.

99 GENERAL AND MISCELLANEOUS↗

Holistic Measurement Driven Resilience: Combining Operational Fault and Failure Measurements and Fault Injection for Quantifying Fault Detection, Propagation and Impact. Final report

For HPC systems to date, application resilience to faults and failures has been accomplished by the brute- force method of checkpoint/restart, which allows an application to make forward progress in the face of system and application faults, errors, and failures independent of root cause or end result. It has remained the primary resilience mechanism because we lack a way to identify faults and anticipate consequences early enough to take meaningful mitigating action. However, checkpoint/restart implementations put a tremendous burden on system resources and on the applications themselves and is becoming less feasible at scale. Because we have not yet operated at scales at which checkpoint/restart fails to provide forward progress, despite increasing costs, vendors have had little motivation to provide the instrumentation necessary for early identification of faults and failures. However, as we move from petascale to exascale, component mean time to failure (MTTF) will render the existing techniques ineffectual and/or too expensive. Furthermore, fault recovery mechanisms such as failover and/or error correction introduce performance inconsistency. Instrumentation allowing early indication of problems and tools to enable use of such information by systems, operating systems, and applications offer an alternative, more scalable and less costly solution. In the HMDR project, we built on our experience and expertise developed and accumulated over years of research on design, monitoring, measurement, and assessment of resilient computing systems. Analysis of field data on the current and past generations of extreme-scale systems revealed several challenges that, if not addressed in increasingly larger and more complex systems, may hinder the effectiveness of future exascale computing systems. Specifically, i) file systems and interconnects in current-generation large-scale systems already operate at the margins of resiliency, including consistent performance, and may not scale to larger deployments; ii) automated, software-based failover mechanisms are frequently inadequate and can introduce wider failures, such that failures during recovery may lead to system/application failures, including system-wide outages; and iii) silent data corruption represents a critical fault mode and will require efficient detection mechanisms if next-generation applications are to take full advantage of exascale hardware. To address the above challenges, we assembled a team of world-renowned experts in resilient extreme- scale computing from the University of Illinois (Electrical and Computer Engineering, Computer Science, and NCSA), SNL, LANL, NERSC, and Cray. Our team includes representatives from centers that house many of the largest HPC resources in the world, both today and over the coming years. The team has a unique track record of research in i) system and application failure characterization based on the analysis of field data, ii) data-driven design of fault/error detection mechanisms, and iii) experimental characterization of system/application resiliency. The team includes system owners/operators who provide continuous data collection and access and ensure installation of appropriate analysis tools.

97 MATHEMATICS AND COMPUTING↗

HDF5 in the exascale era: Delivering efficient and scalable parallel I/O for exascale applications

Accurately modeling real-world systems requires scientific applications at exascale to generate massive amounts of data and manage data storage efficiently. However, parallel input and output (I/O) faces challenges due to new application workflows and the state-of-the-art memory, interconnect, and storage architectures considered in exascale designs. The storage hierarchy has expanded with node-local persistent memory, solid-state storage, and traditional disk and tape-based storage, thus requiring efficiency at each layer and much more efficient data movement among these layers. This paper discusses how the ExaHDF5 project improved the I/O performance and data management for exascale architectures by enhancing HDF5, a widely used parallel I/O library. The team developed an Asynchronous I/O Virtual Object Layer (VOL) connector that allowed overlapping I/O with computation. They also created a Cache VOL to complement asynchronous I/O by incorporating fast storage layers, such as burst buffer and node-local storage, into the parallel I/O workflow through caching and staging data. Additionally, the team enabled data aggregation and I/O at the node level by using a Subfiling Virtual File Driver (VFD). To demonstrate superior I/O performance with HDF5 at exascale, the ExaHDF5 team collaborated with several exascale applications. In this paper, we show I/O performance improvements for three applications: Cabana (a particle-based simulation library), EQSIM (a regional earthquake simulation software), and E3SM (a climate system modeling library).

Asynchronous I/Ol↗

A Software Defined Storage Approach to Exascale Storage Services (Final Technical Report)

The Mochi project began in 2015 in recognition of a set of changes that were set to alter the landscape of storage and I/O for DOE science. On the one hand, an influx of new classes of applications including an initial set of data intensive and machine learning applications were operating with new types of data and in new ways, as compared to traditional computational science simulation codes. Simultaneously, new memory, storage, and networking technologies were driving a need to adapt software designs to a rapidly changing design space. The goal of the three-year Mochi project was to develop and demonstrate an approach for composing storage services which provides new levels of functionality, performance, and reliability for science applications at extreme scale. The team, including a mix of Laboratory, university, and industry collaborators, focused on systems and applications of interest to DOE Office of Science, tracking developments at NERSC and the LCFs as well as application trends within ECP and SciDAC. Technical work focused on three themes: composition (understand the building blocks needed to provide storage services that meet DOE mission needs and how these components are flexibly composed), application interfacing (what service compositions are most effective for common DOE application use cases), and consistency and synchronization (taking advantage of knowledge about, and from, the system and scientific applications to provide better service than would otherwise be possible). The project was highly successful, producing 13 publications and tens of thousands of lines of open source software that is now being used by multiple ECP teams, HPC software vendors, and international collaborators. The uptake of Mochi technologies speaks to both the value of the service composition model and the quality of architecture and software developed by the team. Through continued funding under ASCR and ECP, the Mochi team and its technologies are on track to further contribute to DOE science successes into the exascale timeframe.

97 MATHEMATICS AND COMPUTING↗

ExaSGD: 2022 Kernel Thrust Activities

The Kernel Thrust milestone ADSE22-407 covers the development of device-capable optimization algorithms and solvers technologies required by the ExaSGD project’s software stack in order to solve security-constrained alternating current optimal power flow (SC-ACOPF) problems on emerging exascale architectures. To this extent, in FY22 the main objective of the Kernel Thrust was (i) provide sparse optimization solver that runs efficiently on hardware accelerator devices (i.e., NVIDIA and AMD GPUs) to perform intra-node computations, (ii) strengthen the reliability and increase the performance of the mixed-dense sparse (MDS) solver of HiOp for deployment on the FY22 target architectures, Summit and Crusher, and (iii) increase performance by improving the mathematical algorithm and refining the parallel MPI-based implementation of the coarse-grain parallel solver HiOp-PriDec for capabilities deployment on the FY22 target architectures, Summit and Crusher. This document presents the developments and contributions done by the Kernels Thrust Team in FY22 toward completion of the above-mentioned objectives. These contributions progressed along four main development (sub)thrusts: (1) Design and implementation of a sparse optimization solver for use on hardware accelerators; (2) Improvement of the mathematical algorithm and of the parallel implementation of HiOp-PriDec to ensure readiness and efficient coarse-grain parallelism for FY23 target exascale machine; and (3) Support Software and Application Development Thrusts of the exaSGD project in their deployment of the project’s software stack on AMD- and NVIDIA-based architectures. The development of the sparse optimization solver (thrust 1 above) was new in FY22 and resulted in a new sparse solver in HiOp (available as of version 0.6). The second development thrust was a continuation of the efforts from FY21 and improved the mathematical algorithm and the communication strategy of the HiOp-PriDec solver. The last developement thrust is a large collaborative effort. Namely, the project’s teams from multiple labs (LLNL, PNNL, ORNL, and NREL) performed large-scale demonstration of the ExaSGD software stack, namely the optimization solvers of HiOp interfaced with the modeling front-end ExaGO and the stochastic sampler PowerScenarios. These demonstration efforts solved large-scale instances of the SC-ACOPF challenge problem of medium network sizes (10, 000-bus system) and large number of contingencies on Summit (NVIDIA accelerators) and Crusher (AMD accelerators) systems at ORNL.

97 MATHEMATICS AND COMPUTING↗

Collective input/output under memory constraints

Compared with current high-performance computing (HPC) systems, exascale systems are expected to have much less memory per node, which can significantly reduce necessary collective input/output (I/O) performance. In this study, we introduce a memory-conscious collective I/O strategy that takes into account memory capacity and bandwidth constraints. The new strategy restricts aggregation data traffic within disjointed subgroups, coordinates I/O accesses in intranode and internode layers, and determines I/O aggregators at run time considering memory consumption among processes. We have prototyped the design and evaluated it with commonly used benchmarks to verify its potential. The evaluation results demonstrate that this strategy holds promise in mitigating the memory pressure, alleviating the contention for memory bandwidth, and improving the I/O performance for projected extreme-scale systems. Given the importance of supporting increasingly data-intensive workloads and projected memory constraints on increasingly larger scale HPC systems, this new memory-conscious collective I/O can have a significant positive impact on scientific discovery productivity.

collective input/output↗

PanDA: Production and Distributed Analysis System

The Production and Distributed Analysis (PanDA) system is a data-driven workload management system engineered to operate at the LHC data processing scale. The PanDA system provides a solution for scientific experiments to fully leverage their distributed heterogeneous resources, showcasing scalability, usability, flexibility, and robustness. The system has successfully proven itself through nearly two decades of steady operation in the ATLAS experiment, addressing the intricate requirements such as diverse resources distributed worldwide at about 200 sites, thousands of scientists analyzing the data remotely, the volume of processed data beyond the exabyte scale, dozens of scientific applications to support, and data processing over several billion hours of computing usage per year. PanDA’s flexibility and scalability make it suitable for the High Energy Physics community and wider science domains at the Exascale. Beyond High Energy Physics, PanDA’s relevance extends to other big data sciences, as evidenced by its adoption in the Vera C. Rubin Observatory and the sPHENIX experiment. As the significance of advanced workflows continues to grow, PanDA has transformed into a comprehensive ecosystem, effectively tackling challenges associated with emerging workflows and evolving computing technologies. The paper discusses PanDA’s prominent role in the scientific landscape, detailing its architecture, functionality, deployment strategies, project management approaches, results, and evolution into an ecosystem.

97 MATHEMATICS AND COMPUTING↗

Exploring versioned distributed arrays for resilience in scientific applications: Global view resilience

Exascale studies project reliability challenges for future HPC systems. We present the Global View Resilience (GVR) system, a library for portable resilience. GVR begins with a subset of the Global Arrays interface, and adds new capabilities to create versions, name versions, and compute on version data. Applications can focus versioning where and when it is most productive, and customize for each application structure independently. This control is portable, and its embedding in application source makes it natural to express and easy to maintain. The ability to name multiple versions and “partially materialize” them efficiently makes ambitious forward-recovery based on “data slices” across versions or data structures both easy to express and efficient. Using several large applications (OpenMC, preconditioned conjugate gradient (PCG) solver, ddcMD, and Chombo), we evaluate the programming effort to add resilience. The required changes are small (< 2% lines of code (LOC)), localized and machine-independent, and perhaps most important, require no software architecture changes. We also measure the overhead of adding GVR versioning and show that overheads < 2% are generally achieved. This overhead suggests that GVR can be implemented in large-scale codes and support portable error recovery with modest investment and runtime impact. Our results are drawn from both IBM BG/Q and Cray XC30 experiments, demonstrating portability. We also present two case studies of flexible error recovery, illustrating how GVR can be used for multi-version rollback recovery, and several different forward-recovery schemes. GVR’s multi-version enables applications to survive latent errors (silent data corruption) with significant detection latency, and forward recovery can make that recovery extremely efficient. Lastly, our results suggest that GVR is scalable, portable, and efficient. GVR interfaces are flexible, supporting a variety of recovery schemes, and altogether GVR embodies a gentle-slope path to tolerate growing error rates in future extreme-scale systems.

97 MATHEMATICS AND COMPUTING↗

Steps toward fault-tolerant quantum chemistry.

Developing quantum chemistry programs on the coming generation of exascale computers will be a difficult task. The programs will need to be fault-tolerant and minimize the use of global operations. This work explores the use a task-based model that uses a data-centric approach to allocate work to different processes as it applies to quantum chemistry. After introducing the key problems that appear when trying to parallelize a complicated quantum chemistry method such as coupled-cluster theory, we discuss the implications of that model as it pertains to the computational kernel of a coupled-cluster program - matrix multiplication. Also, we discuss the extensions that would required to build a full coupled-cluster program using the task-based model. Current programming models for high-performance computing are fault-intolerant and use global operations. Those properties are unsustainable as computers scale to millions of CPUs; instead one must recognize that these systems will be hierarchical in structure, prone to constant faults, and global operations will be infeasible. The FAST-OS HARE project is introducing a scale-free computing model to address these issues. This model is hierarchical and fault-tolerant by design, allows for the clean overlap of computation and communication, reducing the network load, does not require checkpointing, and avoids the complexity of many HPC runtimes. Development of an algorithm within this model requires a change in focus from imperative programming to a data-centric approach. Quantum chemistry (QC) algorithms, in particular electronic structure methods, are an ideal test bed for this computing model. These methods describe the distribution of electrons in a molecule, which determine the properties of the molecule. The computational cost of these methods is high, scaling quartically or higher in the size of the molecule, which is why QC applications are major users of HPC resources. The complexity of these algorithms means that MPI alone is insufficient to achieve parallel scaling; QC developers have been forced to use alternative approaches to achieve scalability and would be receptive to radical shifts in the programming paradigm. Initial work in adapting the simplest QC method, Hartree-Fock, to this the new programming model indicates that the approach is beneficial for QC applications. However, the advantages to being able to scale to exascale computers are greatest for the computationally most expensive algorithms; within QC these are the high-accuracy coupled-cluster (CC) methods. Parallel coupledcluster programs are available, however they are based on the conventional MPI paradigm. Much of the effort is spent handling the complicated data dependencies between the various processors, especially as the size of the problem becomes large. The current paradigm will not survive the move to exascale computers. Here we discuss the initial steps toward designing and implementing a CC method within this model. First, we introduce the general concepts behind a CC method, focusing on the aspects that make these methods difficult to parallelize with conventional techniques. Then we outline what is the computational core of the CC method - a matrix multiply - within the task-based approach that the FAST-OS project is designed to take advantage of. Finally we outline the general setup to implement the simplest CC method in this model, linearized CC doubles (LinCC).

99 GENERAL AND MISCELLANEOUS↗

Iterative Workflow for Quantification and Minimization of Reduced Chemistry-Induced Uncertainties in Reacting Flow Simulation

Reduced chemistry models mitigate computational cost but introduce two sources of uncertainties in reacting flow simulation, including chemical information loss due to model reduction, and approximation errors due to non-optimal projection. We present an iterative workflow for quantification and minimization of reduced chemistry-induced uncertainties in reacting flow simulations.

chemistry↗

ExaWind: Exascale Predictive Wind Plant Flow Physics Modeling

The scientific goal of the ExaWind project is to advance our fundamental understanding of the flow physics governing whole wind plant performance, including wake formation, complex terrain impacts, and turbine-turbine-interaction effects. The primary application codes in the ExaWind environment are Nalu-Wind, an unstructured-grid computational fluid dynamics (CFD) code, AMR-Wind, a structured-grid CFD code, and OpenFAST, a whole-turbine simulation code. In this poster we present our progress towards simulating the ExaWind challenge problem, which is a predictive simulation of a wind farm with tens of megawatt-scale wind turbines dispersed over an area of 50 square kilometers.

49 EE - Wind and Water Power Program - Wind (EE-4W↗

Improving the Parameterization of Cloud and Rain Microphysics in E3SM using Novel Observationally-Constrained Bayesian Approach (Final Technical Report)

In this project, we sought to develop new cloud and rain microphysics frameworks within the Energy Exascale Earth System Model (E3SM). This work encompassed two primary avenues of research: 1) Further development of a Bayesian-based scheme called BOSS (Bayesian Observationally-constrained Statistical-physical Scheme) to represent cloud and rain microphysics, testing it in realistic high-resolution cloud models, and implementing it in E3SM; 2) Development of a methodology utilizing machine learning to enable computationally tractable use of tractable use of Markov chain Monte Carlo sampling for Bayesian parameter estimation in Earth system and cloud models. In this project, we adapted the BOSS microphysics scheme, originally formulated for rain-only, to include all liquid-phase microphysical processes for cloud and rain, in particular the processes that mediate between these two categories, for example the conversion from cloud to rain through collision and coalescence of drops. We constrained the scheme via comparison and testing against a detailed model that explicitly represents the evolution of cloud and rain particles, called a bin microphysics scheme.

54 ENVIRONMENTAL SCIENCES↗

E3SM‐Arctic: Regionally Refined Coupled Model for Advanced Understanding of Arctic Systems Interactions

Earth system models are essential tools for climate projections, but coarse resolutions limit regional accuracy, especially in the Arctic. Regionally refined meshes (RRMs) enhance resolution in key areas while maintaining computational efficiency. This paper provides an overview of the United States (U.S.) Department of Energy's (DOE's) Energy Exascale Earth System Model version 2.1 with an Arctic RRM, hereafter referred to as E3SMv2.1-Arctic, for the atmosphere (25 km), land (25 km), and ocean/ice (10 km) components. We evaluate the atmospheric component and its interactions with land, ocean, and cryosphere by comparing the RRM (E3SM2.1-Arctic) historical simulations (1950–2014) with the uniform low-resolution (LR) counterpart, reanalysis products, and observational data sets. The RRM generally reduces biases in the LR model, improving simulations of Arctic large-scale mean fields, such as precipitation, atmospheric circulation, clouds, atmospheric river frequency, and sea ice thickness. However, it introduces a seasonally dependent surface air temperature bias, reducing the LR cold bias in summer but enhancing the LR warm bias in winter, which contributes to the underestimated winter sea ice area and volume. Radiative feedback analysis shows similar climate feedback strengths in both model configurations, with the RRM exhibiting a more positive surface albedo feedback and contributing to a stronger surface warming than LR. These findings underscore the importance of high-resolution modeling for advancing our understanding of Arctic climate changes and their broader global impacts, although some persistent biases appear to be independent of model resolution at 10–100 km scales.

Energy Exascale Earth System Model (E3SM)↗

Keeping LAMMPS cutting edge

Since its inception 30 years ago, LAMMPS has grown to be a world-class molecular dynamics code and a cornerstone of computational materials science research. This project aimed to keep LAMMPS at the forefront of molecular dynamics simulations by adapting LAMMPS to the latest developments in machine learning technology and hardware. Initially, the project set out to provide a unified implementation of active learning for efficient training data generation in LAMMPS, but the research trajectory pivoted to address more immediate and impactful opportunities. On the hardware side, recent record-breaking molecular dynamics simulations were developed on the Cerebras wafer-scale AI chip, and this project has developed an interface between LAMMPS and the hardware-specific molecular dynamics code to accelerate and simplify development and user adoption. On the software side, PyTorch’s Ahead-of-Time (AOT) compilation features promised increased performance for state-of-the-art equivariant neural network potentials, and this project laid the groundwork for their adoption in LAMMPS, resulting in a nearly 20x acceleration in extreme cases. Combined with a comprehensive benchmark study of LAMMPS across all current exascale systems, this project has reinforced LAMMPS’s role as a versatile, high-performance tool for current and future materials science applications.

36 MATERIALS SCIENCE↗

Collaborative Research: Improved Efficiency and Coupling of the Radiation Code in the ACME Earth System Model. Final Report

This final report details all work performed on the project by both project partners. This project provided support to properly couple RTE+RRTMGP, a high-performance broadband radiation code, within DOE’s Energy Exascale Earth System Model (E3SM). RTE+RRTMGP is a successor to the RRTMG radiation code, which has been widely accepted for its speed and accuracy by the global modeling community, and has been in use in the NCAR CESM for many years and was implemented in the initial version of E3SM. However, the computational cost of RRTMG remains high relative to other components in part due to its complexity and to its inefficient use of modern optimization strategies, issues that were rectified by the development of RTE+RRTMGP. Many of the accomplishment in this project necessitated significant collaboration with the E3SM development team. One focus of the project was to enhance the code’s optimization on the limited number of emerging computing systems on which the model is expected be used, including Many Integrated Core (MIC) architectures and Graphics Processing Unit (GPU) hardware. We also developed additional capabilities for RTE+RRTMGP that E3SM scientists identified as important for the planned applications of the model. The result of our project was optimization of a key physical component (radiative transfer calculations) of E3SM, directly supporting E3SM’s overarching global modeling objectives. More broadly, this project provided overall advancements in the use of radiative transfer calculations in atmospheric modeling and simulation, particularly for climate.

54 ENVIRONMENTAL SCIENCES↗