SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Code Analysis and Refactoring with Clang Tools is a small set of example code that demonstrates techniques for applying tools distributed with the open source Clang compiler. Examples include analyzing where variables are used and replacing old data structures with standard structures.
Code Analysis and Refactoring with Clang Tools is a small set of example code that demonstrates techniques for applying tools distributed with the open source Clang compiler. Examples include analyzing where variables are used and replacing old data structures with standard structures.
The Goddard Chemistry Aerosol Radiation and Transport (GOCART) model, which controls the sources sinks and chemistry within the Goddard Earth Observing System, recently underwent a major refactoring and update to the representation of physical processes. The code refactoring increases flexibility such multiple instances of an aerosol species can be run and interact with radiation and cloud microphysics, in addition to the output of multiple wavelength aerosol optical properties in support of data assimilation. From a science perspective, a new radiatively active tracer, brown carbon, was added to distinguish smoke from other sources of organic aerosol thereby improving optical properties entering the radiative calculations. A four-year benchmark simulation was evaluated using in situ and space borne measurements to develop a baseline and prioritize future development. A comparison of simulated aerosol optical depth between GOCART-2G and MODIS retrievals indicates the model captures the overall spatial pattern and seasonal cycle of aerosol optical depth but overestimates aerosol extinction over dusty regions and underestimates aerosol extinction over northern hemisphere boreal forests, requiring further tuning of emissions. This MODIS-based analysis is corroborated by comparisons to MISR and selected AERONET stations. Despite the underestimate of aerosol optical depth in biomass burning regions in GEOS, there is an overestimate in the surface mass of organic carbon in the United States, especially during the summer months.
The intent of this document is to assist the programmer in understanding details of contemporary and Exascale hardware system design and how these designs provide opportunities and place constraints on next-generation simulation software design. We attempt to clarify hardware organization and component details for our most current and Exascale systems to help program developers understand how software needs to change in order to take best advantage of the performance available. Exascale success is specifically defined for ECP as a 50x improvement over baseline in the aggregate "capability volume" on several KPP axes, of which raw floating point performance is only one, but also includes characteristics such as problem size, system memory size, node memory size, power, and efficiency. This multi-axis approach is particularly important to understand in the context of delivered improvements in real applications, since, for instance, the floating point computation may comprise less than 10% of the actual computational work required. Given the Exascale requirements and the constraints these requirements put on the performance expectations of fundamental system components, the programmer will be forced to re-think several application implementation details in order to achieve exaflop performance on these systems. The remainder of this document aims to present more detail on Exascale era system hardware and the specific areas that the programmer should address to extract performance from these systems. We attempt to give the programmer guidance at both a high- and low-level, providing some abstract suggestions on how to refactor codes given the expected system architectures and some low-level recommendations on how to implement these modifications. We also include a section on training resources that are helpful for both programmers that are just beginning to understand code modifications for contemporary and Exascale systems and for those that have done some refactoring and are now trying to extract maximal application performance from these systems.
The Goddard Chemistry Aerosol Radiation and Transport (GOCART) model, which controls the sources sinks and chemistry within the Goddard Earth Observing System, recently underwent a major refactoring and update to the representation of physical processes. This paper serves to document code changes that were included in GOCART 2nd Generation (GOCART-2G) and establishes a benchmark simulation that is to be used for future development of the system. The code refactoring increases flexibility such multiple instances of an aerosol species can be run and interact with radiation and cloud microphysics, in addition to the output of multiple wavelength aerosol optical properties in support of data assimilation. From a science perspective, a new radiatively active tracer, brown carbon, was added to distinguish smoke from other sources of organic aerosol thereby improving optical properties entering the radiative calculations. A four-year benchmark simulation was evaluated using in situ and space borne measurements to develop a baseline and prioritize future development. A comparison of simulated aerosol optical depth between GOCART-2G and MODIS retrievals indicates the model captures the overall spatial pattern and seasonal cycle of aerosol optical depth but overestimates aerosol extinction over dusty regions and underestimates aerosol extinction over northern hemisphere boreal forests, requiring further tuning of emissions. This MODIS-based analysis is corroborated by comparisons to MISR and selected AERONET stations. Despite the underestimate of aerosol optical depth in biomass burning regions in GEOS, there is an overestimate in the surface mass of organic carbon in the United States, especially during the summer months.
The PI was supported by the DOE grant entitled “Theoretical Investigations of Nano and Bio Structures,” 3/1/13-2/28/17, $440,000, and a one-year no cost extension, 3/1/17-2/28/18. Its main emphasis was on nonequilibrium electron transport and novel nanoscale materials and devices, such as graphene nanoribbons, nanotube-based sensors, nanocircuits able to detect DNA replication, and DNA nanowires. We have also made major enhancements to our RMG suite of electronic structure codes, leading to much increased scalability, excellent parallelization of across both CPU and GPU nodes, and cleaner, refactored code base. For example, RMG performs excellently on Summit, DOE’s flagship supercomputer and the fastest supercomputer in the world. It scales to over 100,000 CPU cores and 20,000 GPUs, and can use multiple GPUs per node. It performs 80x faster per node than on DOE’s previous flagship supercomputer, Titan. This is the final report for grant DE-FG02-98ER45685.
We present a new capability of the ice sheet model SICOPOLIS that enables flexible adjoint code generation via source transformation using the open-source algorithmic differentiation (AD) tool OpenAD.The adjoint code enables efficient calculation of the sensitivities of a scalar-valued objective function or quantity of interest (QoI) to a range of important, often spatially varying and uncertain model input variables, including initial and boundary conditions, as well as model parameters.Compared to earlier work on the adjoint code generation of SICOPOLIS, our work makes several important advances:(i) it is embedded within the up-to-date trunk of the SICOPOLIS repository – accounting for 1.5 decades of code development and improvements – and is readily available to the wider community;(ii) the AD tool used, OpenAD, is an open-source tool;(iii) the adjoint code developed is applicable to both Greenland and Antarctica, including grounded ice as well as floating ice shelves, with an extended choice of thermodynamical representations.A number of code refactorization steps were required. They are discussed in detail in an Appendix as they hold lessons for the application of AD to legacy codes at large.As an example application, we examine the sensitivity of the total Antarctic Ice Sheet volume to changes in initial ice thickness, austral summer precipitation, and basal and surface temperatures across the ice sheet.Simulations of Antarctica with floating ice shelves show that over 100 years of simulation the sensitivity of total ice sheet volume to the initial ice thickness and precipitation is almost uniformly positive, while the sensitivities to surface and basal temperature are almost uniformly negative. Sensitivity to austral summer precipitation is largest on floating ice shelves from Queen Maud to Queen Mary Land. The largest sensitivity to initial ice thickness is at outlet glaciers around Antarctica. Comparison between total ice sheet volume sensitivities to surface and basal temperature shows that surface temperature sensitivities are higher broadly across the floating ice shelves, while basal temperature sensitivities are highest at the grounding lines of floating ice shelves and outlet glaciers. A uniformly perturbed region of East Antarctica reveals that, among the four control variables tested here, total ice sheet volume is the most sensitive to variations in austral summer precipitation as formulated in SICOPOLIS.Comparison between adjoint- and finite-difference-derived sensitivities shows good agreement, lending confidence that the AD tool is producing correct adjoint code.The new modeling infrastructure is freely available at http://www.sicopolis.net (last access: 2 April 2020) under the development trunk.
The Density Matrix Renormalization Group (DMRG++) is a condensed matter physics application used to study superconductivity properties of materials. It’s main computations consist of calculating hamiltonian matrix which requires sparse matrix-vector multiplications. This paper presents task-based parallelization and optimization strategies of the Hamiltonian algorithm. The algorithm is implemented as a mini-application in C++ and parallelized with OpenMP. The optimization leverages tasking features, such as dependencies or priorities included in the OpenMP standard 4.5. The code refactoring targets performance as much as programmability. The optimized version achieves a speedup of 8.0× with 8 threads and 20.5× with 40 threads on a Power9 computing node while reducing the memory consumption to 90 MB with respect to the original code, by adding less than ten OpenMP directives.
The high-performance computing (HPC) landscape is quickly changing to systems where most of the performance comes from specialized chips, specifically graphics processing units (GPUs). Such GPU systems are throughput machines, where efficient use of the GPU often requires code refactoring to expose a few orders of magnitude more fine grain parallelism than was previously used on the CPU. Recent modifications to OVERFLOW, an overset, structured grid, computational fluid dynamics flow solver, written in Fortran will be presented. These modifications include both code modernization efforts and algorithmic changes to enable OVERFLOW to efficiently utilize GPUs. Many of these algorithmic changes would likely also be applicable for other structured grid, stencil-based codes wanting to utilize GPUs. The capabilities that have been ported to run on the GPUs are presented, along with the performance gains of the GPU version relative the CPU version of OVERFLOW.
The high-performance computing (HPC) landscape is quickly changing to systems where most of the performance comes from specialized chips, specifically graphics processing units (GPUs). Such GPU systems are throughput machines, where efficient use of the GPU often requires code refactoring to expose a few orders of magnitude more fine grain parallelism than was previously used on the CPU. Recent modifications to OVERFLOW, an overset, structured grid, computational fluid dynamics flow solver, written in Fortran will be presented. These modifications include both code modernization efforts and algorithmic changes to enable OVERFLOW to efficiently utilize GPUs. Many of these algorithmic changes would likely also be applicable for other structured grid, stencil-based codes wanting to utilize GPUs. The capabilities that have been ported to run on the GPUs are presented, along with the performance gains of the GPU version relative the CPU version of OVERFLOW.
Purpose: Monte Carlo particle transport simulation (MC) codes have become important tools in proton therapy and biology, both for research and practice. TOPAS is an MC toolkit serving users worldwide (213 licensed users at 95 institutions in 21 countries). It provides unprecedented ease in 4D placement of geometry components, beam sources and scoring through its user-friendly and reproducible parameter file interface. Quality control (QC) of stochastic simulation software is inherently difficult, and the versatility of TOPAS introduces additional challenges. But QC is vital as the TOPAS development team implements new features, addresses user feedback and reacts to upgrades of underlying software (i.e. Geant4). Methods: Whenever code is committed to our repository, over 50 separate module tests are automatically triggered via a continuous integration service. They check that the various module options execute successfully and that their results are statistically consistent with previous reference values. Prior to each software release, longer end-to-end tests automatically validate TOPAS against experimental data and a TOPAS benchmark. These include simulating multiple properties of spread-out Bragg peaks, validating nuclear models, and searching for differences in patient simulations. Results: Continuous integration has proven effective in catching regressions at the time they are introduced, particularly when implementing new features that involve refactoring code (e.g. multithreading and ntuple output). Code coverage statistics highlight untested portions of code and guide development of new tests. The various end-to-end tests demonstrate that TOPAS accurately describes the physics of proton therapy within clinical tolerances. Conclusion: The TOPAS QC strategy of frequent short tests and pre-release long tests has led to a more reliable tool. However, the versatility of TOPAS makes it difficult to predict how users might combine different modules, and so QC ultimately remains a partnership between the developer and the user. This work was funded by National Cancer Institute grant R01 CA 140735.
We present a methodology for solution reproducibility for the Energy Exascale Earth System Model during its ongoing software infrastructure development to prepare for exascale computers. The nonlinear chaotic nature of climate system simulations precludes traditional model verification approaches since machine precision differences—resulting from code refactoring, changes in software environment, and so on—grow exponentially to a different weather state. Here, we leverage the nature of climate as a statistical description of the atmosphere in order to establish model reproducibility. We evaluate the degree to which two-sample equality of distribution tests can confidently detect the change in climate from minor tuning parameter changes on model output variables in order to establish the level of difference that indicates a new climate. To apply this (baselined test), we target a section of the model’s development cycle wherein no intentional science changes have been applied to its source code. We compare an ensemble of short simulations that were conducted using a verified model configuration against a new ensemble with the same configuration but with the latest software infrastructure (Common Infrastructure for Modeling the Earth, CIME5.0), compiler versions, and software libraries. We also compare these against ensemble simulations conducted using the original version of the software infrastructure (CIME4.0) of the earlier model configuration, but with the latest compilers and software libraries, to test the impact of new compilers and libraries in isolation from additional software infrastructure. The two-sample equality of distribution tests indicates that these ensembles indeed represent the same climate.
The Liquid Argon Time Projection Chamber (LArTPC) technology is widely used in high energy physics experiments, including the upcoming Deep Underground Neutrino Experiment (DUNE). Accurately simulating LArTPC detector responses is essential for analysis algorithm development and physics model interpretations. Accurate LArTPC detector response simulations are computationally demanding, and can become a bottleneck in the analysis workflow. Compute devices such as General-Purpose Graphics Processing Units (GPGPUs) have the potential to substantially accelerate simulations compared to traditional CPU-only processing. The software development for these compute accelerators often carries the cost of specialized code refactorization and porting to match the target hardware architecture. With the rapid evolution and increased diversity of the computer architecture landscape, it is highly desirable to have a portable solution that also maintains reasonable performance. We report our ongoing effort in evaluating Kokkos as a basis for this portable programming model using LArTPC simulations in the context of the Wire-Cell Toolkit, a C++ library for LArTPC simulations, data analysis, reconstruction and visualization.
Novel high performance computing systems that feature hybrid architectures require large scale code refactoring to unravel underlying exploitable parallelism. Such redesign can often be accompanied with machine-precision changes as the order of computation cannot always be maintained. For chaotic systems like climate models, these round-off level differences can grow rapidly. Systematic errors may also manifest initially as machine-precision differences. Isolating genuine round off level differences from such errors remains a challenge. Here, we apply two-sample equality of distribution tests to evaluate statistical reproducibility of the ocean model component of US Department of Energy's Energy Exascale Earth System Model (E3SM). A 2-year control simulation ensemble is compared to a modified ensemble as a test case - after a known non-bit-for-bit change in a model component is introduced - to evaluate the null hypothesis that the two ensembles are statistically indistinguishable. To quantify the false negative rates of these tests, we conduct a formal power analysis using a targeted suite of short simulation ensembles. The ensemble suite contains several perturbed ensembles, each with a progressively different climate than the baseline ensemble - obtained by perturbing the magnitude of a single model tuning parameter, the Gent and McWilliams κ, in a controlled manner. The null hypothesis is evaluated for each of perturbed ensembles using these tests. The power analysis informs on the detection limits of the tests for given ensemble size allowing model developers to evaluate the impact of an introduced non-bit-for-bit change to the model.
In HPC applications, memory access behavior is one of the main factors affecting performance. Improving an application’s memory access behavior involves optimizing data layout and/or restructuring code, and requires studying spatial-temporal data locality. Existing data locality analyses focus on single-location metrics and are restricted to evaluating temporal locality. We introduce spatial-temporal affinity metrics that quantify temporal access proximity, forward access correlation, and nearby access correlation between pairs of memory locations. We describe methods for distinguishing between potential vs. realized affinity and for reasoning about affinity at multiple resolutions (3D, 2D, 1D). Finally, we construct spatial-temporal affinity signatures that classify memory behavior and that be used to reason about changes in software (data relayout, code refactoring) or hardware (caching, prefetching). We describe methods for signature visualization, interpretation, and quantitative comparison of signatures. We evaluate our methodology using applications with variants that contrast data structures, data layouts and algorithms. We show that spatial-temporal affinity analysis provides novel insights and enables predictive reasoning about application performance when contrasted with reuse distance analysis.
Abstract. Surface gravity waves play a critical role in several processes, including mixing, coastal inundation, and surface fluxes. Despite the growing literature on the importance of ocean surface waves, wind–wave processes have traditionally been excluded from Earth system models (ESMs) due to the high computational costs of running spectral wave models. The development of the Next Generation Ocean Model for the DOE’s (Department of Energy) E3SM (Energy Exascale Earth System Model) Project partly focuses on the inclusion of a wave model, WAVEWATCH III (WW3), into E3SM. WW3, which was originally developed for operational wave forecasting, needs to be computationally less expensive before it can be integrated into ESMs. To accomplish this, we take advantage of heterogeneous architectures at DOE leadership computing facilities and the increasing computing power of general-purpose graphics processing units (GPUs). This paper identifies the wave action source terms, W3SRCEMD, as the most computationally intensive module in WW3 and then accelerates them via GPU. Our experiments on two computing platforms, Kodiak (P100 GPU and Intel(R) Xeon(R) central processing unit, CPU, E5-2695 v4) and Summit (V100 GPU and IBM POWER9 CPU) show respective average speedups of 2× and 4× when mapping one Message Passing Interface (MPI) per GPU. An average speedup of 1.4× was achieved using all 42 CPU cores and 6 GPUs on a Summit node (with 7 MPI ranks per GPU). However, the GPU speedup over the 42 CPU cores remains relatively unchanged (∼ 1.3×) even when using 4 MPI ranks per GPU (24 ranks in total) and 3 MPI ranks per GPU (18 ranks in total). This corresponds to a 35 %–40 % decrease in both simulation time and usage of resources. Due to too many local scalars and arrays in the W3SRCEMD subroutine and the huge WW3 memory requirement, GPU performance is currently limited by the data transfer bandwidth between the CPU and the GPU. Ideally, OpenACC routine directives could be used to further improve performance. However, W3SRCEMD would require significant code refactoring to make this possible. We also discuss how the trade-off between the occupancy, register, and latency affects the GPU performance of WW3.
PVDeg version 0.7.0, released in December 2025, introduced major enhancements to improve usability and performance. This update reorganized tutorials and tool notebooks to create a more intuitive experience, enabling users to easily follow and adapt workflows for their specific analyses. In addition to structural improvements, both the notebooks and core logic underwent significant optimization for efficiency, robustness, and style. These refinements were supported by new testing frameworks built on nbval and pytest, adherence to PEP8 standards, and extensive code refactoring, which collectively simplify onboarding for new developers. Looking ahead, version 0.8.0 will deliver advanced AI-driven capabilities. The primary focus is to further develop and automate the degradation workflow, designed to analyze PV module degradation across diverse locations and system configurations. By integrating large language models (LLMs) to scan literature and compile a comprehensive database of materials and degradation rates, this feature will enable modeling of multiple materials and mechanisms within a single, streamlined workflow. Users will be able to evaluate degradation impacts on different system architectures under varying environmental conditions, facilitating informed decisions on bill-of-materials optimization for specific deployment scenarios. These advancements position PVDeg as a powerful, user-friendly tool for accelerating PV reliability research and system design.
Umbra is a flexible simulation framework for complex systems that can be used by itself for modeling, simulation, and analysis, or to create specific applications. It has been applied to many operations, primarily dealing with robotics and system of system simulations. This version, from 4.8 to 4.8.3b, incorporates bug fixes, refactored code, and new managed C++ wrapper code that can be used to bridge new applications written in C# to the C++ libraries. The new managed C++ wrapper code includes (project/directories) BasicSimulation, CSharpUmbraInterpreter, LogFileView, UmbraAboutBox, UmbraControls, UmbraMonitor and UmbraWrapper.