Search NASASearch

SEARCH · Search NASA

Results for “Exascale simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Large-Scale Computational Fluid Dynamics Simulations of Aerospace Configurations on the Frontier Exascale System

Over the past fifteen years, the high performance computing landscape has undergone a seismic shift in both hardware and software paradigms, which has been necessary to realize a 1000× leap in computational performance while meeting stringent constraints on power consumption. A historical overview of a long-term research effort aimed at addressing these challenges within the context of a commonly-used aerospace computational fluid dynamics (CFD) application is presented. Details of the current implementation as they relate to the new era of exascale-relevant hardware architectures and programming models are described. Two large-scale simulations of aerospace configurations are performed using the entire Frontier exascale system, currently ranked as the most powerful supercomputing system in the world. The effort serves to address a 2024 milestone posed a decade ago by the seminal CFD Vision 2030 Study.

Eric J Nielsen

Implicit Thermochemical Nonequilibrium Compressible Flow Simulations on Unstructured Grids Using GPUs

As next-generation exascale-class systems arrive, existing software must be updated accordingly to effectively utilize these systems. For high concurrency and energy efficiency, many of these systems utilize GPU architectures. In this work, we present a CUDA C++ implementation of FUN3D's thermochemical nonequilibrium capability for turbulent flows. Efficiency is demonstrated at scale using the Summit system at the Oak Ridge Leadership Computing Facility which is representative of future exascale systems. This work enables faster, higher fidelity, and scale-resolving simulations of thermochemical nonequilibrium flows including reentry, hypersonics, and combustion.

CFD, GPU, HPC, Hypersonics, Chemistry

Building a Transdisciplinary, Exascale-Capable Workforce for Geospace Science

Key Points: ●First-principles, self-consistent geospace modeling will require at least exascale-level computing capabilities; however, the technical skills necessary to develop such simulation codes are not taught as part of Heliophysics training/PhD programs. ●Developing exascale Heliophysics codes will require transdisciplinary collaborations between physicists, computer scientists, software engineers, data scientists, and applied mathematics. Such teams must be persistent and formed around specific skills, not specific problems. ●We must have stable, positive long-term career outcomes for Heliophysics scientific developers in order to retain knowledge of and promote future innovation in scientific algorithm development and advanced computing techniques. One such way to do this is by funding long-term scientific programming groups, akin to Heliophysics DRIVE Centers. ●We must have a balance between promoting innovation via funding short-term closed-source development and allowing the open-source community to benefit from and build on the newest scientific modeling techniques.

C Bard

High Latitude Atmospheric Rivers: Teleconnections and Impacts

Atmospheric rivers (ARs) are long, narrow synoptic scale features that act as moisture transport vehicles by moving water vapor poleward. They are typically found in storm tracks and often have profound implications for regional hydroclimate. For high latitude locales such as Antarctica and the Arctic, ARs are fewer in number but often have an outsized impact by either producing significant snow accumulation events, or, depending on their thermal characteristics, initiating melt events on ice sheets, ice shelves, and sea ice. In this study, we apply both reanalysis and climate model data to evaluate high latitude ARs for both the Arctic and Antarctic in the context of teleconnections and modes of variability. Specifically, we fold in both MERRA-2 and ERA5 reanalysis as well as historical and climate change simulations from the CESM2 (Community Earth System Model, Version 2) and E3SMv2 (Energy Exascale Earth System Model, v2) with multiple ensemble members to better understand natural variability versus climate change signals. An overview of AR climatology and impacts will be presented, including a baseline uncertainty quantification analysis using data from ARTMIP (the Atmospheric River Tracking Method Intercomparison Project). AR impacts during specific modes of variability will be characterized by diagnosing precipitation and boundary layer temperature associated with Ars influencing the cryosphere. Modes of variability investigated include the SAM (Southern Annular Mode), PDO (Pacific Decadal Oscillation), PSA2 (Pacific South American Mode 2), the IOD (Indian Ocean Dipole), and the AO (Arctic Oscillation).

Atmospheric Rivers

GEOS Atmospheric Model: Challenges at Exascale

The Goddard Earth Observing System (GEOS) model at NASA's Global Modeling and Assimilation Office (GMAO) is used to simulate the multi-scale variability of the Earth's weather and climate, and is used primarily to assimilate conventional and satellite-based observations for weather forecasting and reanalysis. In addition, assimilations coupled to an ocean model are used for longer-term forecasting (e.g., El Nino) on seasonal to interannual times-scales. The GMAO's research activities, including system development, focus on numerous time and space scales, as detailed on the GMAO website, where they are tabbed under five major themes: Weather Analysis and Prediction; Seasonal-Decadal Analysis and Prediction; Reanalysis; Global Mesoscale Modeling, and Observing System Science. A brief description of the GEOS systems can also be found at the GMAO website. GEOS executes as a collection of earth system components connected through the Earth System Modeling Framework (ESMF). The ESMF layer is supplemented with the MAPL (Modeling, Analysis, and Prediction Layer) software toolkit developed at the GMAO, which facilitates the organization of the computational components into a hierarchical architecture. GEOS systems run in parallel using a horizontal decomposition of the Earth's sphere into processing elements (PEs). Communication between PEs is primarily through a message passing framework, using the message passing interface (MPI), and through explicit use of node-level shared memory access via the SHMEM (Symmetric Hierarchical Memory access) protocol. Production GEOS weather prediction systems currently run at 12.5-kilometer horizontal resolution with 72 vertical levels decomposed into PEs associated with 5,400 MPI processes. Research GEOS systems run at resolutions as fine as 1.5 kilometers globally using as many as 30,000 MPI processes. Looking forward, these systems can be expected to see a 2 times increase in horizontal resolution every two to three years, as well as less frequent increases in vertical resolution. Coupling these resolution changes with increases in complexity, the computational demands on the GEOS production and research systems should easily increase 100-fold over the next five years. Currently, our 12.5 kilometer weather prediction system narrowly meets the time-to-solution demands of a near-real-time production system. Work is now in progress to take advantage of a hybrid MPI-OpenMP parallelism strategy, in an attempt to achieve a modest two-fold speed-up to accommodate an immediate demand due to increased scientific complexity and an increase in vertical resolution. Pursuing demands that require a 10- to 100-fold increases or more, however, would require a detailed exploration of the computational profile of GEOS, as well as targeted solutions using more advanced high-performance computing technologies. Increased computing demands of 100-fold will be required within five years based on anticipated changes in the GEOS production systems, increases of 1000-fold can be anticipated over the next ten years.

ESMF

Changing Characteristics of Tropical Extreme Precipitation-Cloud Regimes in Warmer Climates

In this paper, we have investigated the changing characteristics of climatic scale (monthly) tropical extreme precipitation in warming climates using the Energy Exascale Earth System Model (E3SM). Results are from Atmospheric Model Intercomparison Project (AMIP)-type simulations driven by a) control experiment with present-day sea surface temperature (SST) and CO 2 concentration, b) P4K, same as in a) but with uniform increase by 4K in SST globally, and c) same as in a), but with imposed SST and CO 2 concentration from outputs of coupled E3SM forced by 4xCO 2 concentration. We find that as the surface warms under P4K and 4xCO 2 , both convective and stratiform rain increase. Importantly, there is increasing fractional contribution of stratiform rain as a function of precipitation intensity, with the most extreme but rare events occurring preferentially over land more than ocean, more so under 4xCO 2 than P4K. Extreme precipitation is facilitated by increased precipitation efficiency, reflecting accelerated rates of recycling of precipitation-cloud water (both liquid and ice phases) in regions with colder anvil cloud tops. Changes in vertical profiles of clouds, condensation heating and vertical motions indicate increasing precipitation-cloud-circulation organization from control, P4K to 4xCO 2 . Results suggest that large-scale ocean warming, i.e., P4K, is the primary cause contributing to organization structure resembling the well-known Mesoscale Convective System (MCS) for increased extreme precipitation on shorter (hourly to daily) time scales. Additional 4xCO 2 atmospheric radiative heating, and dynamically consistent anomalous SST further amplifies the MCS organization under P4K. Analyses of the surface moist static energy distribution show that increase in surface moisture (temperature) under P4K and 4xCO 2 is the key driver leading to enhanced convective instability over tropical ocean (land). However, the fast and large increase in land surface temperature and lack of available local moisture result in strong reduction in land surface relative humidity, reflecting severe drying and enhanced convective inhibition (CIN). It is argued that very extreme and rare “record-breaking” precipitation events found over land under P4K, more so under 4xCO 2 , are likely due to the delayed onset of deep convection, i.e., the longer the suppression of deep convection by CIN, the more severe the extreme precipitation when eventually occur, due to the release of large amount of stored surplus convective available potential energy in the lower troposphere during prolonged CIN.

climate-scale extreme tropical precipitation

Implicit Thermochemical Nonequilibrium Flow Simulations on Unstructured Grids using GPUs

Thermochemical nonequilibrium flow simulation capabilities have been previously implemented, verified, and validated for central processing unit (CPU) systems in NASA’s unstructured-grid computational fluid dynamics solver FUN3D. Many exascale-class high-performance computing systems will rely on graphics processing unit (GPU) architectures for high throughput and energy efficiency; thus, CPU-based scientific computing software unable to effectively utilize these systems must be updated. In this work, we present a CUDA C++ implementation of FUN3D’s thermochemical nonequilibrium flow simulation capabilities targeting NVIDIA Tesla GPUs. An overview of the porting and optimization strategy is described and performance comparisons with other recent architectures are presented. Scaling to thousands of GPUs is demonstrated, yielding computational performance equivalent to that of several million CPU cores. The implementation enables efficient, high-fidelity, scale-resolving simulations of thermochemical nonequilibrium flows for many applications including atmospheric entry, hypersonics, and combustion.

GPU

A model-independent data assimilation (MIDA) module and its applications in ecology

Models are an important tool to predict Earth system dynamics. An accurate prediction of future states of ecosystems depends on not only model structures but also parameterizations. Model parameters can be constrained by data assimilation. However, applications of data assimilation to ecology are restricted by highly technical requirements such as model-dependent coding. To alleviate this technical burden, we developed a model-independent data assimilation (MIDA) module. MIDA works in three steps including data preparation, execution of data assimilation, and visualization. The first step prepares prior ranges of parameter values, a defined number of iterations, and directory paths to access files of observations and models. The execution step calibrates parameter values to best fit the observations and estimates the parameter posterior distributions. The final step automatically visualizes the calibration performance and posterior distributions. MIDA is model independent, and modelers can use MIDA for an accurate and efficient data assimilation in a simple and interactive way without modification of their original models. We applied MIDA to four types of ecological models: the data assimilation linked ecosystem carbon (DALEC) model, a surrogate-based energy exascale earth system model: the land component (ELM), nine phenological models and a stand-alone biome ecological strategy simulator (BiomeE). The applications indicate that MIDA can effectively solve data assimilation problems for different ecological models. Additionally, the easy implementation and model-independent feature of MIDA breaks the technical barrier of applications of data–model fusion in ecology. MIDA facilitates the assimilation of various observations into models for uncertainty reduction in ecological modeling and forecasting.

Earth system dynamics

Closed-Loop Simulations of Human-Scale Mars Lander Descent Trajectories on Frontier

A computational campaign was performed to run high-fidelity, free-flight simulations of a human-scale Mars lander concept vehicle decelerating under retropropulsion through the Martian atmosphere with closed-loop flight control. A novel approach is used to couple computational fluid dynamics (CFD) software with a mature flight mechanics package, where the two applications communicate in real-time across two geographically-dispersed computational facilities. The CFD is performed on the Frontier exascale system located at Oak Ridge National Laboratory, and the flight mechanics are executed on a system located at NASA Langley Research Center. In the current campaign, CFD is performed using finite-rate chemistry to account for the interactions between the LOXCH 4 engines and the CO 2 Martian atmosphere. A simulation of a closed-loop main engine throttling and RCS actuation is presented, demonstrating that the vehicle and model are able to maintain stability in a long-duration CFD-in-the-loop flight simulation. Comparisons are made to a reduced order model ignoring aero-propulsive interactions.

CFD

Land-Atmosphere Coupling at the U.S. Southern Great Plains: A Comparison on Local Convective Regimes Between ARM Observations, Reanalysis, and Climate Model Simulations

Using the 9-yr warm-season observations at the Atmospheric Radiation Measurement Southern Great Plains site, we assess the land-atmosphere (L-A) coupling in North American Regional Reanalysis (NARR) and two climate models: hindcasts with the Community Atmosphere Model version 5.1 by Cloud-Associated Parameterizations Testbed (CAM5-CAPT) and nudged runs with the Energy Exascale Earth System Model Atmosphere Model version 1 Regionally Refined Model (EAMv1-RRM). We focus on three local convective regimes and diagnose model behaviors using the Local Coupling metrics (Santanello et al. 2018). NARR agrees well with observations except a slightly warmer and drier surface with higher downwelling shortwave radiation and lower evaporative fraction. On clear-sky days, it shows warmer and drier early-morning conditions in both models with significant underestimates in surface evaporation by EAMv1-RRM. On the majority of the ARM-observed shallow cumulus days, there is no or little low-level clouds in either model. When captured in models, the simulated shallow cumulus shows much less cloud fraction and lower cloud bases than observed. On the days with late-afternoon deep convection, models tend to present a stable early-morning lower atmosphere more frequently than the observations, suggesting that the deep convection is triggered more often by elevated instabilities. Generally, CAM5-CAPT can reproduce the local L-A coupling processes to some extent due to the constrained early-morning conditions and large-scale winds. EAMv1-RRM exhibits large precipitation deficits and warm and dry biases towards mid-to-late summers, which may be an amplification through a positive L-A feedback among initial atmosphere and land states, convection triggering and large-scale circulations.

land-atmosphere

Global Dust Cycle and Direct Radiative Effect in E3SM Version 1: Impact of Increasing Model Resolution

Quantification of dust aerosols in Earth System Models (ESMs) has important implications for water cycle and biogeochemistry studies. This study examines the global life cycle and direct radiative effects (DREs) of dust in the U.S. Department of Energy's Energy Exascale Earth System Model version 1 (E3SMv1), and the impact of increasing model resolution both horizontally and vertically. The default 1° E3SMv1 captures the spatial and temporal variability in the observed dust aerosol optical depth (DAOD) reasonably well, but overpredicts dust absorption in the shortwave (SW). Simulations underestimate the dust vertical and long-range transport, compared with the satellite dust extinction profiles. After updating dust refractive indices and correcting for a bias in partitioning size-segregated emissions, both SW cooling and longwave (LW) warming of dust simulated by E3SMv1 are increased and agree better with other recent studies. The estimated net dust DRE of −0.42 Wm−2 represents a stronger cooling effect than the observationally based estimate −0.2 Wm−2 (−0.48 to +0.2), due to a smaller LW warming. Constrained by a global mean DAOD, model sensitivity studies of increasing horizontal and vertical resolution show strong influences on the simulated global dust burden and lifetime primarily through the change of dust dry deposition rate; there are also remarkable differences in simulated spatial distributions of DAOD, DRE, and deposition fluxes. Thus, constraining the global DAOD is insufficient for accurate representation of dust climate effects, especially in transitioning to higher- or variable-resolution ESMs. Better observational constraints of dust vertical profiles, dry deposition, size, and LW properties are needed.

dust aerosol

Evolving HPC and Application Design Toward a Coupled Data Assimilation System at NASA Suitable for Emerging Exascale Platforms

The prediction capabilities of global models have continuously evolved from the traditional medium-range global weather prediction application to span scales in support of hourly prediction of convective scale storms to seasonal Earth system prediction. This evolution has increased the demands on the system infrastructure design and workflow to achieve the required performance on modern high-performance computing (HPC) platforms. The planned evolution of the Goddard Earth Observing System (GEOS) modeling and assimilation system will stress the capabilities of conventional HPC overwhelming the available compute cycles at the NASA Center for Climate Simulation (NCCS) at the NASA Goddard Space Flight Center in the coming 5-10 years. This has led to the re-design of key elements of the assimilation and modeling systems to achieve significant gains in performance on anticipated Exacale platforms. The transition of the assimilation system to the Joint Effort for Data assimilation Integration (JEDI) framework has positioned GEOS to exploit new efficient algorithms for data assimilation (DA) in a fully-coupled Earth system context. The suitability of the GEOS model to leverage a domain specific language (DSL) approach and artificial intelligence (AI) is being explored to accelerate computational performance and data exchange efficiency of the coupled Earth system model. The storage and processing of large data volumes produced by these advance systems is being redesigned with a data-centric cloud-based approach. We will highlight the recent efforts in these areas and emphasize the demand for further development and re-design to achieve the science objectives in support of NASA's Earth system modeling and assimilation missions.

Putman, Bill