Search NASA⌕ Search

SEARCH · Search NASA

Results for “exascale simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

In situ multi-tier auto-ignition detection applied to dual-fuel combustion simulations

Here we use an anomaly detection methodology that is centered on analyzing fourth-order joint moments (co-kurtosis), particularly focusing on its application in auto-ignition of combustion problems with large numbers of species. Unsupervised anomaly detection is challenging to generalize across problem types and domains. A recent technique, centered on analyzing information in the fourth-order joint moment co-kurtosis, has shown promise, especially for high-dimensional scientific data. In this work we present developments to the co-kurtosis based anomaly detection method needed to make it effective and scalable for large-scale distributed scientific data, such as those generated by massively parallel simulations. An in situ co-kurtosis algorithm is employed as the anomaly detection method for identifying ignition kernels in simulations of turbulent combustion. Here, we extend an existing methodology which identifies regions of the domain where anomalies are present, and add another tier of anomaly detection where the individual samples contributing to the anomaly are identified. We apply this algorithm on-the-fly to a variety of turbulent reacting flow problems and compare it to the widely used (but significantly more expensive) chemical explosive mode analysis (CEMA). We demonstrate the ability of the method to detect and identify the onset of low and high temperature ignition which can be used for computational steering, as chemical and combustion anomalies occur intermittently at spatio-temporal locations unknown a priori. Finally, we apply our lightweight in situ algorithm to an exascale high-fidelity simulation with a total of 2.4 Trillion degrees of freedom, performed using an adaptive mesh refinement solver. Furthermore, through a scalability analysis, we show that the relative computational cost of this in-situ anomaly detection algorithm compared to an iteration of the reacting flow solver is negligible.

97 MATHEMATICS AND COMPUTING↗

Attribution of Snowpack Errors to Simulated Temperature and Precipitation in E3SMv1 Over the Contiguous United States

Abstract Snow water equivalent (SWE), temperature, and precipitation biases and trends are evaluated in the atmosphere‐land simulations of the Energy Exascale Earth System Model version 1 (E3SMv1) in comparison to the Community Earth System Model version 2 (CESM2) and two other models using the ground measurement‐based University of Arizona (UA) snow product. SWE, temperature, and precipitation biases are highest in magnitude in the Western contiguous United States (CONUS). SWE errors are attributed to temperature and precipitation through multiple linear regressions of normalized errors, the coefficients of which represent the sensitivities to temperature and precipitation errors. SWE errors are more sensitive to temperature errors throughout the CONUS. Model SWE and temperature trends are generally opposite from UA product trends in the Western CONUS. SWE trend errors are also attributed to temperature and precipitation trend errors using multiple linear regressions of normalized trend errors. SWE trend errors are more sensitive to those of precipitation at higher elevations (>1,500 m) in the Western CONUS in these simulations. Thus, the sensitivity to temperature and precipitation differ for SWE errors and its trend errors. Furthermore, the SWE trend errors are more sensitive to temperature and precipitation in the atmosphere‐ocean coupled simulations in which the atmosphere‐land is coupled to an active ocean model. These results suggest that both errors in simulated temperature and precipitation contribute to SWE errors.

Meteorology & Atmospheric Sciences↗

Estimating the carbon footprint of computational fluid dynamics

In this study, computational resources have grown exponentially in the past few decades. These machines make possible research and design in fields as diverse as medicine, astronomy, and engineering. Despite ever-increasing computational capabilities, direct simulation of complex systems has remained challenging owing to the degrees of freedom involved. At the cusp of exascale computing, high-resolution simulation of practical problems with minimal model assumptions may soon experience a renaissance. However, growing reliance on modern computers comes at the cost of a growing carbon footprint. To illustrate this, we examine historic computations in fluid dynamics where larger computers have afforded the opportunity to simulate flows at increasingly relevant Reynolds numbers. Under a variety of flow configurations, the carbon footprint of such simulations is found to scale roughly with the fourth power of Reynolds number. This is primarily explained by the computation cost in core-hours, which is also described by similar scaling, though regional differences in renewable energy use also play a role. Using the established correlation, we examine a large database of simulations to develop estimates for the carbon footprint of computational fluid dynamics in a given year. Collectively, the analysis provides an additional benchmark for new computations where, in addition to balancing considerations of model fidelity, carbon footprint should also be considered.

42 ENGINEERING↗

Ginkgo - A math library designed to accelerate Exascale Computing Project science applications

Large-scale simulations require efficient computation across the entire computing hierarchy. A challenge of the Exascale Computing Project (ECP) was to reconcile highly heterogeneous hardware with the myriad of applications that were required to run on these supercomputers. Mathematical software forms the backbone of almost all scientific applications, providing efficient abstractions and operations that are crucial to harness the performance of computing systems. Ginkgo is one such mathematical software library, nurtured by ECP, providing high-performance, user-friendly, and performance portable interfaces for applications in ECP and beyond. In this paper, we elaborate on Ginkgo’s philosophy of high-performance software that is sustainable, reproducible, and easy to use. We showcase the wide feature set of solvers and preconditioners available in Ginkgo and the central concepts involved in their design. We elaborate on four different ECP software integrations: MFEM, PeleLM + SUNDIALS, XGC, and ExaSGD that use Ginkgo to accelerate their science runs. Performance studies of different problems from these applications highlight the effectiveness of Ginkgo and the benefits incurred by these ECP applications.

Cojean, Terry↗

Regional-scale fault-to-structure earthquake simulations with the EQSIM framework: Workflow maturation and computational performance on GPU-accelerated exascale platforms

Continuous advancements in scientific and engineering understanding of earthquake phenomena, combined with the associated development of representative physics-based models, is providing a foundation for high-performance, fault-to-structure earthquake simulations. However, regional-scale applications of high-performance models have been challenged by the computational requirements at the resolutions required for engineering risk assessments. The EarthQuake SIMulation (EQSIM) framework, a software application development under the US Department of Energy (DOE) Exascale Computing Project, is focused on overcoming the existing computational barriers and enabling routine regional-scale simulations at resolutions relevant to a breadth of engineered systems. This multidisciplinary software development—drawing upon expertise in geophysics, engineering, applied math and computer science—is preparing the advanced computational workflow necessary to fully exploit the DOE’s exaflop computer platforms coming online in the 2023 to 2024 timeframe. Achievement of the computational performance required for high-resolution regional models containing upward of hundreds of billions to trillions of model grid points requires numerical efficiency in every phase of a regional simulation. This includes run time start-up and regional model generation, effective distribution of the computational workload across thousands of computer nodes, efficient coupling of regional geophysics and local engineering models, and application-tailored highly efficient transfer, storage, and interrogation of very large volumes of simulation data. This article summarizes the most recent advancements and refinements incorporated in the workflow design for the EQSIM integrated fault-to-structure framework, which are based on extensive numerical testing across multiple graphics processing unit (GPU)-accelerated platforms, and demonstrates the computational performance achieved on the world’s first exaflop computer platform through representative regional-scale earthquake simulations for the San Francisco Bay Area in California, USA.

58 GEOSCIENCES↗

Increasing Functional Diversity in a Global Land Surface Model Illustrates Uncertainties Related to Parameter Simplification

Abstract Simulations of the land surface carbon cycle typically compress functional diversity into a small set of plant functional types (PFT), with parameters defined by the average value of measurements of functional traits. In most earth system models, all wild plant life is represented by between five and 14 PFTs and a typical grid cell (≈100 × 100 km) may contain a single PFT. Model logic applied to this coarse representation of ecological functional diversity provides a reasonable proxy for the carbon cycle, but does not capture the non‐linear influence of functional traits on productivity. Here we show through simulations using the Energy Exascale Land Surface Model in 15 diverse terrestrial landscapes, that better accounting for functional diversity markedly alters predicted total carbon uptake. The shift in carbon uptake is as great as 30% and 10% in boreal and tropical regions, respectively, when compared to a single PFT parameterized with the trait means. The traits that best predict gross primary production vary based on vegetation phenology, which broadly determines where traits fall within the global distribution. Carbon uptake is more closely associated with specific leaf area for evergreen PFTs and the leaf carbon to nitrogen ratio in deciduous PFTs.

58 GEOSCIENCES↗

Climate Base State Influences on South Asian Monsoon Processes Derived From Analyses of E3SMv2 and CESM2

Abstract The effects of differences in climate base state are related to processes associated with the present‐day South Asian monsoon simulations in the Energy Exascale Earth System Model version 2 (E3SMv2) and the Community Earth System Model version 2 (CESM2). Though tropical Pacific and Indian Ocean base state sea surface temperatures (SSTs) are over 1°C cooler in E3SMv2 compared to CESM2, and there is an overall reduction of Indian sector precipitation, the pattern of South Asian monsoon precipitation is similar in the two models. Monsoon‐ENSO teleconnections, dynamically linked by the large‐scale east‐west atmospheric circulation, are reduced in E3SMv2 compared to CESM2. In E3SMv2, this is related to cooler tropical SSTs and ENSO amplitude that is less than half that in CESM2. Comparison to a tropical Pacific pacemaker experiment shows, to a first order, that the base state SSTs and ENSO amplitude contribute roughly equally to lower amplitude monsoon‐ENSO teleconnections in E3SMv2.

54 ENVIRONMENTAL SCIENCES↗

The Influence of Climate Variability and Future Climate Change on Atlantic Hurricane Season Length

Abstract Atlantic hurricane season length is important for emergency management preparation, motivating the need to understand its variability and change. We investigated the influence of ocean variability on Atlantic hurricane season length in observations and a future climate simulated by the Energy Exascale Earth System Model (E3SM). We found that multiple factors influence hurricane season length, through their influence on season start and end. Warm western subtropical Atlantic sea‐surface temperature anomalies (SSTAs) during boreal spring (before the official hurricane season start) drive early starts to the hurricane season, and vice versa for cool SSTAs. Meanwhile, La Niña in autumn (before the official hurricane season end) drives late ends to the hurricane season, and vice versa for El Niño. E3SM projects a 27‐day increase in future Atlantic hurricane season length given La Niña and warm northern tropical Atlantic SSTAs. This research documents sources of predictability for Atlantic hurricane season length.

54 ENVIRONMENTAL SCIENCES↗

On the Dependence of Simulated Convection on Domain Size in CRMs

Abstract We present a heuristic model to explain the suppression of deep convection in convection‐resolving models (CRMs) with a small number of grid columns, such as those used in super‐parameterized or multi‐scale modeling framework (MMF) general circulation models (GCM) of the atmosphere. Domains with few grid columns require greater instability to sustain convection because they force a large convective fraction, driving strong compensating subsidence warming. Updraft dilution, which is stronger for reduced horizontal grid spacing, enhances this effect. Thus, suppression of deep convection in CRMs with few grid columns can be reduced by increasing grid spacing. Radiative‐convective equilibrium simulations using standalone CRM simulations with the System for Atmospheric Modeling (SAM) and using GCM‐coupled CRM simulations with the Energy Exascale Earth System Model (E3SM)‐MMF confirm the heuristic model results.

CRM↗

Assessing Tropical Pacific-induced Predictability of Southern California Precipitation Using a Novel Multi-input Multi-output Autoencoder

We construct a novel Multi-Input Multi-Output Autoencoder-decoder (MIMO-AE) to capture the non-linear relationship of Southern California precipitation and tropical Pacific Ocean sea surface temperature. The MIMO-AE is trained on both monthly TP-SST and SC-PRECIP anomalies simultaneously. The co-variability of the two fields in the MIMO-AE shared nonlinear latent space can be condensed into an index, termed the MIMO-AE index. We use a transfer learning approach to train a MIMO-AE on the combined dataset of 100 years of output from a historical simulation with the Energy Exascale Earth Systems Model version 1 and a segment of observational data. We further use Long Short-Term Memory networks to assess sub-seasonal predictability of SC-PRECIP using the MIMO-AE index. We find that the MIMO-AE index provides enhanced predictability of SC-PRECIP for a lead-time of up-to four months as compared to Niño 3.4 index and the El Niño Southern Oscillation Longitudinal Index.

54 ENVIRONMENTAL SCIENCES↗

Demonstrate moving-grid multi-turbine simulations primarily run on GPUs and propose improvements for successful KPP-2

The goal of the ExaWind project is to enable predictive simulations of wind farms comprised of many megawatt-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines, capturing the thin boundary layers, and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources.

17 WIND ENERGY↗

Demonstrate multi-turbine simulation with hybrid-structured / unstructured-moving-grid software stack running primarily on GPUs and propose improvements for successful KPP-2

The goal of the ExaWind project is to enable predictive simulations of wind farms comprised of many megawatt-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines, capturing the thin boundary layers, and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources.

17 WIND ENERGY↗

Simulating Extreme Precipitation in the United States in the Energy Exascale Earth System Model: Investigating the Importance of Representing Convective Intensity versus Dynamic Structure (Final Report)

Weather events that produce extreme precipitation are associated with severe flooding and winds that result in thousands of deaths and billions of dollars in damages annually in the United States (U.S.). This project aims to improve understanding of the small- and large-scale processes that govern these events and improve our ability to project changes in these extreme events under the influences of natural variability and human activities. We focus on two promising directions in the development of the U.S. Department of Energy’s Energy Exascale Earth System Model (E3SM) to investigate the tradeoffs between resolving the convective-scale processes that control the intensity versus the intermediate-scale processes that control the dynamic structure of the most prominent extreme precipitation events that impact the U.S. throughout the year (i.e., mesoscale convective systems, tropical cyclones, and extratropical cyclone).

58 GEOSCIENCES↗

IMPACT: Design of Integrated Multiphysics Producible Additive Components for Turbomachinery

The overall objective of the IMPACT program was to enable a dramatic reduction in design maturation time for an additive hot-section turbomachinery component through the following: • A fast crack-risk producibility surrogate model generated from machine learning applied to additive process simulation data generated via exascale computing, • Linking this surrogate model to multi-physics topology optimization (TO) to enable the creation of producible, near-optimal structural/thermal designs for additive hot-section components, • Maturing this toolset to reduce hot-section component design-for-manufacturing iterations by a large fraction, and eventually, • Using these tools to develop more efficient gas turbines in much shorter design cycle times.

42 ENGINEERING↗

From PeleC to PeleACC, to PeleC++

PeleC is an Exascale Computing Project application for simulating compressible combustion in complex geometries. It has been built on top of the popular AMReX library. In the beginning of the Exascale Computing Project, PeleC was focused on KNL. It uses a mixture of C++, C, and kernels written in Fortran to obtain performance by focusing on vectorization. Recently we have taken two approaches in deciding PeleC's future for obtaining performance on exascale GPU machines. In the first programming model, we decorated the Fortran kernels with OpenACC directives. This expedited our ability to run at large scales on Summit's GPUs, where we achieved a significant speedup over the CPUs on Summit. The second programming model involved rewriting the Fortran kernels in C++ and using AMReX's Kokkos-like lambda abstractions for running on the GPU. This resulted in similar speedups on Summit's GPUs over merely utilizing the CPUs. Both approaches involved AMReX's management of memory transfers between the device and host. In this work, we compare and contrast the benefits and pitfalls to both programming approaches regarding performance, performance portability, and productivity. We also discuss advantages we have found in taking the time to modernize our code and why have chosen a specific pathway to prepare our code for the future DOE exascale machines.

exascale computing↗

Advanced Turbulence Models for Large-Scale Atmospheric Boundary Layer Flows

We present high-fidelity large-eddy-simulation (LES) modeling approaches for the turbulent atmospheric boundary layer (ABL) flows. Wind energy is a prime example of an application driven by ABL. Generation of electrical energy from farms of wind turbines at night in the stable ABL is a particularly interesting situation. In this report, we consider the well-known GEWEX (Global Energy and Water Cycle Experiment) Atmospheric Boundary Layer Study (GABLS) stably stratified benchmark LES case. We use a high-order spectral element code Nek5000/RS, which is supported under the DOE's Exascale Computing Project (ECP) Center for Efficient Exascale Discretizations (CEED) project, targeting application simulations on various acceleration-device based exascale computing platforms. In our earlier ANL report, we demonstrated our newly developed subgrid-scale (SGS) models based on high-pass filter (HPF), mean-field eddy viscosity (MFEV), and Smagorinsky (SMG) with no-slip and traction boundary conditions, provided with low-order statistics, convergence and turbulent structure analysis. In this report, we extend the range of our SGS modeling approaches in the context of the mean-field eddy viscosity (MFEV), to include the solution of an SGS turbulent kinetic energy equation (TKE). We demonstrate the model fidelity of Nek5000/RS in comparison to that of AMR-Wind, a block-structured second-order finite-volume code with adaptive-mesh-refinement capabilities, with which we studied scaling performance for both codes in comparison on DOE's leadership computing platforms.

17 WIND ENERGY↗

Preparing sparse solvers for exascale computing

Sparse solvers provide essential functionality for a wide variety of scientific applications. Highly parallel sparse solvers are essential for continuing advances in high-fidelity, multi-physics and multi-scale simulations, especially as we target exascale platforms. This paper describes the challenges, strategies and progress of the US Department of Energy Exascale Computing project towards providing sparse solvers for exascale computing platforms. We address the demands of systems with thousands of high-performance node devices where exposing concurrency, hiding latency and creating alternative algorithms become essential. The efforts described here are works in progress, highlighting current success and upcoming challenges.

97 MATHEMATICS AND COMPUTING↗

An Implicit Approach to Phase Field Modeling of Solidification for Additively Manufactured Alloys [Slides]

We are leveraging modern algorithms and computational science to provide a route to predictive simulation of microstructure evolution on emerging exascale architectures. We are utilizing the fastest supercomputers in the world for modeling and simulation of microstructure evolution for generation of data under AM conditions. Solidification conditions in AM can be tailored for the reliable design of materials to specific performance requirements. Developing computational tools to further characterize alloys and correlate the processing-structure-properties-performance (PSPP) relationship.

36 MATERIALS SCIENCE↗