Search NASA⌕ Search

SEARCH · Search NASA

Results for “exascale simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Early Evaluation of Fugaku A64FX Architecture Using Climate Workloads

The Energy Exascale Earth System Model (E3SM) Project is an ongoing, state-of-the-science Earth system modeling, simulation, and prediction project that targets efficient utilization of U.S. Department of Energy’s (DOE) supercomputers to meet the science needs of the nation and the mission needs of DOE. This work focuses on our early evaluation of the A64FX architecture on Fugaku supercomputer using E3SM benchmarks. We will present results that track hardware trends, facilitate architecture comparison and the specific impact on our workload using an atmospheric model benchmark. We have two variants of the code written in Fortran and C++/Kokkos respectively which were used to collect data on a variety of CPU and GPU platforms. Furthermore, we have conducted a comparative evaluation of the compilers on the A64FX architecture and found GNU to be the best performer for our workload. Our experience so far indicates that Fugaku/A64FX shows promising energy efficiency (performance/Watt) with further performance gains possible through architecture-aware optimization efforts.

Sreepathi, Sarat↗
DOE OSTI2020

Evaluating Trade-offs in Potential Exascale Interconnect Technologies

This report details work to study trade-offs in topology and network bandwidth for potential interconnects in the exascale (2021-2022) timeframe. The work was done using multiple interconnect models across two parallel discrete event simulators. Results from each independent simulator are shown and discussed and the areas of agreement and disagreement are explored.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Nek5000/RS Performance on Advanced GPU Architectures

We demonstrate NekRS performance results on various advanced GPU architectures. NekRS is a GPU-accelerated version of Nek5000 that is targeting high performance on forthcoming exascale platforms. It is being developed in DOE’s Center of Efficient Exascale Discretizations (CEED), which is one of the co-design centers under the Exascale Computing Project (ECP). In this report, we consider Frontier, Crusher, Spock, Polaris, Perlmutter, ThetaGPU and Summit. Simulations are performed using ExaSMR’s 17x17 rod-bundle geometries with different problem sizes. The report focuses on strong scaling performance and analysis. Many of the results shown in this report are the outcome from participation in the ALCF GPU Hackathon on Polaris, which was held on 7/19/22, and 7/26–7/28/22. Members of the Nek5000/RS team included Misun Min, Yu-Hsiang Lan, Paul Fischer and Thilina Rathnayake. Mentors were Kris Rowe (ALCF) and Peng Wang (NVIDIA). Also presented in this report are results on Frontier, which were obtained in collaboration with John Holmen at OLCF.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Atmospheric river representation in the Energy Exascale Earth System Model (E3SM) version 1.0

Abstract. The Energy Exascale Earth System Model (E3SM) project is an ongoing, state-of-the-science Earth system modeling, simulation, and prediction project developed by the US Department of Energy (DOE). With an emphasis on supporting the DOE's energy mission, understanding and quantifying how well the model simulates water cycle processes is of particular importance. Here, we evaluate E3SM version 1.0 (v1.0) for its ability to represent atmospheric rivers (ARs), which play significant roles in water vapor transport and precipitation. The characteristics and precipitation associated with global ARs in E3SM at standard resolution (1∘ × 1∘) are compared to the Modern-Era Retrospective analysis for Research and Applications, version 2 (MERRA2). Global patterns of AR frequencies in E3SM show high degrees of correlation (≥0.97) with MERRA2 and low mean absolute errors (MAEs; <1 %) annually, seasonally, and across different ensemble members. However, some large-scale condition biases exist, leading to AR biases – most significant of which are the double intertropical convergence zone (ITCZ), a stronger and/or equatorward-shifted subtropical jet during boreal and austral winters, and enhanced Northern Hemisphere westerlies during summer. By comparing atmosphere-only and fully coupled simulations, we attribute the sources of the biases to the atmospheric component or to a coupling response. Using relationships revealed in Dong et al. (2021), we provide evidence showing the stronger North Pacific jet in winter and the enhanced Northern Hemisphere westerlies during summer, associated with E3SM's double ITCZ and related weaker Atlantic meridional overturning circulation (AMOC), respectively, which are significant sources of the AR biases found in the coupled simulations.

58 GEOSCIENCES↗
DOE OSTI2024

Integrating Tide‐Driven Wetland Soil Redox and Biogeochemical Interactions Into a Land Surface Model

Abstract Redox processes, aqueous and solid‐phase chemistry, and pH dynamics are key drivers of subsurface biogeochemical cycling and methanogenesis in terrestrial and wetland ecosystems but are typically not included in terrestrial carbon cycle models. These omissions may introduce errors when simulating systems where redox interactions and pH fluctuations are important, such as wetlands where saturation of soils can produce anoxic conditions and coastal systems where sulfate inputs from seawater can influence biogeochemistry. Integrating cycling of redox‐sensitive elements could therefore allow models to better represent key elements of carbon cycling and greenhouse gas production. We describe a model framework that couples the Energy Exascale Earth System Model (E3SM) Land Model (ELM) with PFLOTRAN biogeochemistry, allowing geochemical processes and redox interactions to be integrated with land surface model simulations. We implemented a reaction network including aerobic decomposition, fermentation, sulfate reduction, sulfide oxidation, methanogenesis, and methanotrophy as well as pH dynamics along with iron oxide and iron sulfide mineral precipitation and dissolution. We simulated biogeochemical cycling in tidal wetlands subject to either saltwater or freshwater inputs driven by tidal hydrological dynamics. In simulations with saltwater tidal inputs, sulfate reduction led to accumulation of sulfide, higher dissolved inorganic carbon concentrations, lower dissolved organic carbon concentrations, and lower methane emissions than simulations with freshwater tidal inputs. Model simulations compared well with measured porewater concentrations and surface gas emissions from coastal wetlands in the Northeastern United States. These results demonstrate how simulating geochemical reaction networks can improve land surface model simulations of subsurface biogeochemistry and carbon cycling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

AMR-Wind [SWR-20-85]

AMR-Wind is a massively parallel, block-structured adaptive-mesh, incompressible flow solver for wind turbine and wind farm simulations. The solver is built on top of the AMReX library. AMReX is developed at LBNL , NREL , and ANL as part of the Block-Structured AMR Co-Design Center in DOE's Exascale Computing Project. AMReX library provides the mesh data structures, mesh adaptivity, as well as the linear solvers used for solving the governing equations. The primary applications for AMR-Wind are: performing large-eddy simulations (LES) of atmospheric boundary layer (ABL) flows, simulating wind farm turbine-wake interactions using actuator disk or actuator line models for turbines, and as a background solver when coupled with a near-body solver with overset methodology to perform blade-resolved simulations of multiple wind turbines within a wind farm.

Ananthan, Shreyas↗
DOE OSTI2021

Toward exascale whole-device modeling of fusion devices: Porting the GENE gyrokinetic microturbulence code to GPU

GENE solves the five-dimensional gyrokinetic equations to simulate the development and evolution of plasma microturbulence in magnetic fusion devices. The plasma model used is close to first principles and computationally very expensive to solve in the relevant physical regimes. In order to use the emerging computational capabilities to gain new physics insights, several new numerical and computational developments are required. Here, we focus on the fact that it is crucial to efficiently utilize GPUs (graphics processing units) that provide the vast majority of the computational power on such systems. In this paper, we describe the various porting approaches considered and given the constraints of the GENE code and its development model, justify the decisions made, and describe the path taken in porting GENE to GPUs. We introduce a novel library called gtensor that was developed along the way to support the process. Performance results are presented for the ported code, which in a single node of the Summit supercomputer achieves a speed-up of almost 15× compared to running on central processing unit (CPU) only. Typical GPU kernels are memory-bound, achieving about 90% of peak. Our analysis shows that there is still room for improvement if we can refactor/fuse kernels to achieve higher arithmetic intensity. We also performed a weak parallel scalability study, which shows that the code runs well on a massively parallel system, but communication costs start becoming a significant bottleneck.

Germaschewski, K. (ORCID:0000000284956354)↗
DOE OSTI2021

In Situ Machine Learning for Intelligent Data Capture on Exascale Platforms. Final Report

In many dynamic systems, interesting events occur locally in time and space. Examples of such systems include ignition events in combustion simulations, material fractures in mechanics simulations, and extreme weather events in climate simulations. Due to memory constraints and data I/O costs, current simulation workflows save data at regularly spaced time-steps, at a fixed rate determined before the start of the simulation. Often this mode of operation results in missed events of interest, necessitating a simulation restart from before an event occurred with more frequent data saves. This data saving workflow is grossly inefficient and is already a bottleneck in the computing process. We propose to develop machine learning algorithms that can detect when interesting dynamical events are occurring, triggering data saves. These machine learning algorithms will perform in situ anomaly detection to flag regions with different dynamical properties than those previously recorded. The adaptive data saves would be local in time and space to match the event of interest, thereby enabling a much more efficient workflow that will reduce data I/O costs and data storage memory requirements. The algorithms will be tested on two applications: auto-ignition simulations and climate simulations. A critical component of this project will be developing machine learning algorithms that can be deployed efficiently in situ on HPC platforms with out-of-the-box functionality. The development of in situ machine learning methods to detect anomalous events would enable a more efficient and effective workflow, in which all the relevant data are saved in a single simulation run, without re-starts or scientist intervention.

42 ENGINEERING↗
DOE OSTI2025

Efficiently resolving the terrestrial-aquatic interface in E3SM with sub-grid methods to improve coastal simulations (Final Technical Report)

The broad goal of this project was to extend the capabilities of MPAS‐Ocean, the ocean component of DOE’s Energy Exascale Earth System Model (E3SM), to better resolve the hydrodynamics of the terrestrial‐aquatic interface without significantly increasing computing resource requirements. Specifically, our goal was to better resolve the small-scale features across the terrestrial‐aquatic interface which includes the low lying coastal floodplain and a dendritic network of hydraulically efficient channels connected to estuarine systems and back bays, barrier islands and high energy inlets. The processes driving flow across the terrestrial‐aquatic interface include upland and floodplain hydrology, tides, wind and atmospheric pressure (including coastally generated storm surge across continental shelves and within estuaries and bays), wind waves and water temperature and salinity gradients.

58 GEOSCIENCES↗
DOE OSTI2022

Investigating coastal backwater effects and flooding in the coastal zone using a global river transport model on an unstructured mesh

Abstract. Coastal backwater effects are caused by the downstream water level increase as a result of elevated sea level, high river discharge and their compounding influence. Such effects have crucial impacts on floods in densely populated regions but have not been well represented in large-scale river models used in Earth system models (ESMs), partly due to model mesh deficiency and oversimplifications of river hydrodynamics. Using two mid-Atlantic river basins as a testbed, we perform the first attempt to simulate the backwater effects comprehensively over a coastal region using the MOSART river transport model under an ESM framework, i.e., Energy Exascale Earth System Model (E3SM) configured on a regionally refined unstructured mesh, with a focus on understanding the backwater drivers and their long-term variations. By including sea level variations at the river downstream boundary, the model performance in capturing backwaters is greatly improved. We also propose a new flood event selection scheme to facilitate the decomposition of backwater drivers into different components. Our results show that while storm surge is a key driver, the influence of extreme discharge cannot be neglected, particularly when the river drains to a narrow river-like estuary. Compound flooding, while not necessarily increasing the flood peaks, exacerbates the flood risk by extending the duration of multiple coastal and fluvial processes. Furthermore, our simulations and analysis highlight the increasing strength of backwater effects due to sea level rise and more frequent storm surge during 1990–2019. Thus, backwaters need to be properly represented in ESMs to improve the predictive understanding of coastal flooding.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Impacts of spatial heterogeneity of anthropogenic aerosol emissions in a regionally refined global aerosol–climate model

Abstract. Emissions of anthropogenic aerosol and their precursors are often prescribed in global aerosol models. Most of these emissions are spatially heterogeneous at model grid scales. When remapped from low-resolution data, the spatial heterogeneity in emissions can be lost, leading to large errors in the simulation. It can also cause the conservation problem if non-conservative remapping is used. The default anthropogenic emission treatment in the Energy Exascale Earth System Model (E3SM) is subject to both problems. In this study, we introduce a revised emission treatment for the E3SM Atmosphere Model (EAM) that ensures conservation of mass fluxes and preserves the original emission heterogeneity at the model-resolved grid scale. We assess the error estimates associated with the default emission treatment and the impact of improved heterogeneity and mass conservation in both globally uniform standard-resolution (∼ 165 km) and regionally refined high-resolution (∼ 42 km) simulations. The default treatment incurs significant errors near the surface, particularly over sharp emission gradient zones. Much larger errors are observed in high-resolution simulations. It substantially underestimates the aerosol burden, surface concentration, and aerosol sources over highly polluted regions, while it overestimates these quantities over less-polluted adjacent areas. Large errors can persist at higher elevation for daily mean estimates, which can affect aerosol extinction profiles and aerosol optical depth (AOD). We find that the revised treatment significantly improves the accuracy of the aerosol emissions from surface and elevated sources near sharp spatial gradient regions, with significant improvement in the spatial heterogeneity and variability of simulated surface concentration in high-resolution simulations. In the next-generation E3SM running at convection-permitting scales where the resolved spatial heterogeneity is significantly increased, the revised emission treatment is expected to better represent the aerosol emissions as well as their lifecycle and impacts on climate.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Impacts of Topography-Based Subgrid Scheme and Downscaling of Atmospheric Forcing on Modeling Land Surface Processes in the Conterminous US

The effects of small-scale topography-induced land surface heterogeneity are not well represented in current Earth System Models (ESMs). In this study, a new topography-based subgrid structure referred to as topographic units (TGU) designed to better capture subgrid topographic effects, and methods to downscale atmospheric forcing to the land TGUs have been implemented in the Energy Exascale Earth System Model (E3SM) Land Model (ELM). Effects of the subgrid scheme and downscaling methods on ELM simulated land surface processes are evaluated over the conterminous United States (CONUS). For this purpose, ELM simulations are performed using two configurations without (NoD ELM) and with (D ELM) downscaling, both using TGUs derived for the 0.5-degree grids and the same land surface parameters. Simulations using the two ELM configurations are compared over the CONUS domain, regional levels, and at observational sites (e.g., SNOTEL). The CONUS-level results suggest that D ELM simulates more snowfall and snow water equivalent (SWE), higher runoff, and less ET during spring and summer. Regional-level results suggest more pronounced impacts of downscaling over regions dominated by higher elevation TGUs and regions with maximum precipitation occurring during cool seasons. Results at the SNOTEL sites suggest that D ELM has superior capability of reproducing the observed SWE at 83% of the sites, with more pronounced performance over topographically heterogeneous TGUs with their maximum precipitation occurring during cool seasons. The results highlight the importance of improving representation of small-scale surface heterogeneity in ESMs and motivate future research to understand their effects on land-atmosphere interactions, streamflow, and water resources management over mountainous regions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Evaluating E3SM Global Storm‐Resolving Model Simulations of Deep Convection: Insights From DP‐SCREAM During TRACER

Global Storm-Resolving Models (GSRMs) are becoming increasingly vital for advancing climate modeling and improving the prediction of extreme weather events. Houston, a coastal region frequently affected by deep convective storms, offers an ideal setting to evaluate the ability of GSRMs to simulate deep convection. This study assesses the performance of the Doubly Periodic Simple Cloud-Resolving E3SM (Energy Exascale Earth System Model) Atmosphere Model (DP-SCREAM) using observations from the TRacking Aerosol Convection interactions ExpeRiment (TRACER) campaign. DP-SCREAM effectively reproduces the diurnal cycles of clouds and precipitation, demonstrating much greater skill than the E3SM single column model. The DP-SCREAM is demonstrated to be applicable to coastal regions, partially due to the forcing data sets already capturing the influence of breezes. DP-SCREAM also replicates biases persistent in the global version of SCREAM: the underrepresentation of boundary layer shallow clouds, a lack of mid-level congestus clouds, and the popcorn convection, characterized by small and disorganized convective cells generating the strongest precipitation. To investigate these issues, two sensitivity experiments were conducted: increasing the mixing length and scaling up the buoyancy flux within the Simplified Higher Order Closure scheme. Increasing the mixing length improved mid-level congestus representation and reduced unrealistic early morning fog occurrence. Enhancing buoyancy flux only marginally improved the bias of underproduced big convective cells. In conclusion, an additional resolution sensitivity test at 0.5 km grid spacing demonstrated that a refined horizontal resolution alone is insufficient to resolve these biases.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Then and Now: Improving Software Portability, Productivity, and 100× Performance

The US Exascale Computing Project (ECP) has succeeded in preparing applications to run efficiently on the first reported Exascale supercomputers in the world. To achieve this, it modernized the whole leadership software stack, from libraries to simulation codes. In this article, we contrast selected leadership software before and after ECP. We discuss how sustainable research software development for leadership computing can embrace the conversation with the hardware vendors, the leadership computing facilities, the software community, and the domain scientists who are the application developers and integrators of software products. We elaborate on how software needs to take portability as a central design principle and to benefit from interdependent teams; we also demonstrate how moving to programming languages with high momentum, like modern C++, can help improve the sustainability, interoperability, and performance of research software. Finally, we showcase how cross-institutional efforts can enable algorithm advances that are beyond incremental performance optimization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

The Scientific Impact of the Exascale Computing Project

The recent arrival of the Frontier Supercomputer at Oak Ridge National Laboratory officially marked the dawn of the exascale computing era. Its successful deployment coincided with the culmination of the U.S. Department of Energy Exascale Computing Project (ECP), an ambitious, complex, and risky research and development effort that integrated contributions from a broad and diverse subset of the high-performance computing community. The success of ECP will ultimately be judged by the scientific and engineering advances that it enabled. In conclusion, this Special Issue is focused on showcasing early successes in the use of exascale resources to enable breakthroughs in key areas of science in engineering.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

HDF5 in the exascale era: Delivering efficient and scalable parallel I/O for exascale applications

Accurately modeling real-world systems requires scientific applications at exascale to generate massive amounts of data and manage data storage efficiently. However, parallel input and output (I/O) faces challenges due to new application workflows and the state-of-the-art memory, interconnect, and storage architectures considered in exascale designs. The storage hierarchy has expanded with node-local persistent memory, solid-state storage, and traditional disk and tape-based storage, thus requiring efficiency at each layer and much more efficient data movement among these layers. This paper discusses how the ExaHDF5 project improved the I/O performance and data management for exascale architectures by enhancing HDF5, a widely used parallel I/O library. The team developed an Asynchronous I/O Virtual Object Layer (VOL) connector that allowed overlapping I/O with computation. They also created a Cache VOL to complement asynchronous I/O by incorporating fast storage layers, such as burst buffer and node-local storage, into the parallel I/O workflow through caching and staging data. Additionally, the team enabled data aggregation and I/O at the node level by using a Subfiling Virtual File Driver (VFD). To demonstrate superior I/O performance with HDF5 at exascale, the ExaHDF5 team collaborated with several exascale applications. In this paper, we show I/O performance improvements for three applications: Cabana (a particle-based simulation library), EQSIM (a regional earthquake simulation software), and E3SM (a climate system modeling library).

Asynchronous I/Ol↗
DOE OSTI2022

Benthic Kinetic Energy in an Earth System Model and Observations

The benthic layer of the ocean is closely correlated with every upper layer above it. Close analysis of benthic layer stress can yield an accurate depiction of the state of the ocean, as well as the atmosphere above it. Through the use of the Department of Energy’s Energy Exascale Earth System Model (E3SM), we took a look at how these features changed over time in a simulation spanning the historical period from 1958 to 2018 in the Arctic Ocean north of 60° N. Making use of E3SM’s regional refinement capability to place higher resolution in regions of interest, the E3SM-Arctic configuration has ~10 km resolution in the ocean and sea ice components in this region, coarsening up to ~60 km outside of the Arctic (Veneziani et al. 2022). In particular, we narrowly focused on the benthic kinetic energy levels, and how trends changed over time. To validate the fidelity of the E3SM model, we compared model current speed outputs over a 369-day period in the Bering Strait to actual recorded observations during this time period. Our analysis of the benthic kinetic energy mean, standard deviation, and trend throughout this time period can be used to characterize the effects of environmental impacts in the Arctic Ocean.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Position Papers for Inverse Methods for Complex Systems under Uncertainty Workshop

The ability to solve inverse problems – inferring unknown parameters, structures, or states of a system from observed data – is essential for advancing scientific discovery and innovation capabilities for the DOE mission. Basic research needs and challenges are particularly acute in emerging areas such as the interactive, data-driven, modeling and simulation of digital twins; decision support for experiments at DOE scientific user facilities; and for other complex systems and workflows. Inverse problems are at the heart of understanding and controlling complex systems due to factors such as observational data with varying modalities and fidelities, inherent uncertainties in physical measurements and numerical models, and the computational demands of rapid and high-fidelity simulations. The convergence of recent scientific computing trends – scientific machine learning, artificial intelligence, and computing advances such as exascale computing – is creating unprecedented opportunities. These advancements offer the potential to revolutionize how we approach inverse problems to extract actionable insights with the required level of accuracy and computational efficiency. This workshop and the Call for Position Papers are vital steps in bringing together experts to collectively explore and identify the new computational and mathematical directions needed in inverse methods for complex systems under uncertainty.

97 MATHEMATICS AND COMPUTING↗