SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This is a time series data set of resource utilizations and runtime for jobs run on both HPC systems (IC2 at Brookhaven Naional Lab, Polaris at Argonne National Lab and Amazon Web Services. The data set can be used for machine learning models to predict runtime and resource utilization of jobs on a variety of systems.
Presented and discussed here is the implementation of a software solution that provides prompt X-ray diffraction data analysis during fast dynamic compression experiments conducted within the dynamic diamond anvil cell technique. It includes efficient data collection, streaming of data and metadata to a high-performance cluster (HPC), fast azimuthal data integration on the cluster, and tools for controlling the data processing steps and visualizing the data using the DIOPTAS software package. This data processing pipeline is invaluable for a great number of studies. The potential of the pipeline is illustrated with two examples of data collected on ammonia–water mixtures and multiphase mineral assemblies under high pressure. The pipeline is designed to be generic in nature and could be readily adapted to provide rapid feedback for many other X-ray diffraction techniques, e.g. large-volume press studies, in situ stress/strain studies, phase transformation studies, chemical reactions studied with high-resolution diffraction etc.
The rapid increase in data center energy demand, driven by AI and large-scale data processing, poses significant challenges to global energy infrastructure. Data centers require substantial and reliable energy for continuous operations and high-performance computing. Current electrical grids face issues such as transmission bottlenecks and aging infrastructure, making it difficult to meet these demands. Integrating inverter-based-resources (IBRs) like solar and wind presents both opportunities and challenges due to their intermittent nature. Small Modular Reactors (SMRs) offer a promising solution with their enhanced safety, modularity, reliability, and scalability, providing consistent base load power ideal for data center operations. This study presents a comprehensive techno-economic assessment of powering data center load demand using a combination of SMRs and IBRs with grid-connected and islanded mode. This study utilized Idaho National Laboratory’s (INL) HPC data center hourly load profiles and Xendee microgrid optimization platform to conduct the analysis. In this configuration, SMRs serves as the primary base load power source, consistently providing a steady supply of electricity necessary to meet the minimum load demand of the data center with support from the IBRs. Key performance indicators such as Levelized Cost of Electricity (LCOE), Net Present Value (NPV) has been calculated to assess the economic feasibility. The findings from this research will underscore the strategic benefits of integrating SMR plant with DERs – particularly for critical infrastructure load such as data centers.
This project was a collaborative effort between Lawrence Livermore National Security, LLC (LLNS) as manager and operator of Lawrence Livermore National Laboratory (LLNL) and Materials Sciences, LLC, to develop a technology for design and optimization of heat exchangers using powerful desktop and laptop computers. The project was originally designated as a 12-month project, and consisted of 3# major tasks and the following 8# major deliverables: 1) CFD models of 3D heat exchangers based on existing and new geometry. 2) Validation against experimental data provided by MSC and published in the literature. 3) CFD models of 3D unit cells based on TPMS. 4) Surrogate models capable of delivering the gradients of the homogenized properties with respect to the parametrization. 5) 3D design methodology using TO algorithms. 6) Conventional reference and topology optimized designs. 7) 3D optimized designs stored in a 3D printer build format. 8) Verification of the improved performance. All of the deliverables for this project were successfully completed with two no-cost time extensions.
As part of the Game Changing Development (GCD) Program, funded by NASA’s Space Technology Mission Directorate (STMD), the development of simulation capability for the prediction of extra-terrestrial Plume Surface Interaction (PSI) environments has been undertaken by the Fluid Dynamics Branch at NASA/MSFC. The Predictive Simulation Capability (PSC) Element is focused on creating simulation capability for the reliable and accurate prediction of PSI in Martian (~650 Pa) and Lunar (vacuum) ambient environments. In addition to the predictive simulation capability, the GCD Program also contains a companion Ground Testing Element for development of focused datasets for validation of predictive capability as well as a Flight-focused Instrumentation Element. This paper will present the status of implementing and maturing particle-particle interaction constituent physics models essential in simulating the landing surface granular material flow under PSI effects. This gas-particle multi-phase interaction modeling of plume impingement flow on the extra-terrestrial soil material is performed with the Gas-Granular Flow Solver (GGFS) addressed in a companion paper. The response of regolith particle flow induced by lander PSI requires accurate representation of the regolith granular material fluidic behavior and gas-granular interactions. The lunar regolith, as the extreme example, is poorly sorted with broad particle size distributions and large fines content. It has significant cohesion, due to interlocking particle shapes for the very jagged particles. The combination of particle shape and size distribution has been identified as major drivers in the complex particle flow response and resulting crater shape characteristics of extraterrestrial granular material. Constituent models for spherical particles can be formulated directly from particle kinetics theory. Complex particle shapes can be modeled by gluing together elemental spherical shapes into composite particles, requiring a Discrete Element Model (DEM) particle kinetics modeling approach to extract data and formulate constituent models. Mixture constituent models for poly-disperse mixtures (i.e, containing distribution of particle sizes) have recently been developed. The required non-spherical particle mixture granular material response closure models are then obtained through small-scale unit physics DEM simulations for the range of particle shapes, mixtures and packing densities. The granular material response closure models are then implemented in the Eulerian granular flow formulation. This DEM-based constituent model extraction process and formulation of poly-disperse particle mixtures has been successfully developed by small business and academic partners in the development of the Gas-Granular Flow Solver (GGFS) simulation program simulation framework. The currently implemented capabilities have reached the capability level of modeling bi-disperse, non-spherical particle mixtures is being continuously extended towards computational modeling of full range irregular particle mixtures. Under the GCD project, this technology is being further developed, transferred to NASA analysts, and matured towards application readiness. The predictive simulation capability team under the GCD project has acquired the modeling tools and processes of the DEM based constituent model formulation from the GGFS development team and is developing the capability to replicate the existing process. This is the first important step towards the ability of the NASA team to independently perform such model development in a production setting. Further efforts are underway to migrate the DEM based model simulation process performed with the academic based tools to more capable Open Source, highly parallelized simulation tools for efficient operation on NASA HPC assets. Evaluation of the currently implemented (such as mono-disperse and bi-disperse spherical and irregular shape particle constituent model applications) and continuously evolving full-range particle physics models in the GGFS tool is performed by the NASA team to advance application readiness of the simulations. Application testing for complex PSI erosions and cratering scenarios such as the Apollo LM is performed for axi-symmetric and full 3D simulations to aid the tool developers in achieving practical application readiness for NASA projects. Important validation and application testing will further be performed against experimental data generated under the GCD PSI project experimental component.
Training in Deep learning (DL) remains highly compute- and data-intensive, with I/O becoming a critical bottleneck as models and datasets scale. Recent studies report that data loading can dominate training time, especially on large-scale HPC systems with shared parallel file systems (PFS). Existing caching approaches either rely on single-tier designs or require intrusive modifications to training pipelines, limiting their portability and effectiveness. In this work, we present FitCache, a transparent drop-in framework for multi-tier caching to accelerate distributed DL training by coordinating fast local memory (e.g., DRAM, Persistent Memory (PMem)) and NVMe as hierarchical caches atop PFS. Our design adapts to hardware diversity, i.e., if NVMe is missing, memory transparently acts as a caching tier, ensuring stable performance. FitCache transparently intercepts I/O requests and issues concurrent fetches across all tiers, returning data from the fastest responder without centralized metadata or static redirection paths. FitCache adapts to dynamic workloads and heterogeneous clusters while maintaining POSIX compatibility. Experiments on Frontier (2048 GPUs) and smaller research clusters show that FitCache reduces training time by up to 40% and per-batch I/O latency by up to 71.6% compared to Lustre Orion PFS, offering a drop-in solution for scalable DL training.
HPC is designed for large-scale simulations using monolithic codes of tightly coupled processes highly optimized to deliver decreased time to solution. Medical image processing is not a traditional field of HPC. Similar to AI applications, medical image processing parses large datasets, typically multiple times, to support a variety of studies for classification, diagnosis or monitoring purposes. The convergence of AI, HPC and Big Data encouraged more fields using image processing to transition to HPC. However, not all applications benefit from the same optimizations. In this paper we focus on high throughput medical image processing applications that analyze a huge dataset of small MRI images and that require HPC systems to decrease the time of parsing the entire dataset and not individual MRIs. We show in this research the performance of running SLANT, an image processing application for a whole brain segmentation, on large-scale systems and highlight performance limitations. We present optimizations prioritizing throughput that exhibit a 3.5x speed-up on the Summit Supercomputer that can be used as a baseline for building a high-throughput execution framework for other HPC systems.
The emergence and development of advanced technologies and vehicle types have created a growing demand for new forms of flight operations. These new and increasingly complex operational paradigms, such as Advanced and Urban Air Mobility (AAM/UAM), present regulatory authorities and the aviation community with several design-and-implementation challenges – particularly for highly autonomous vehicles. An overarching and daunting task is to develop protocols that can integrate these operations without compromising safety or disrupting traditional airspace operations. A shift toward a more predictive, autonomous, risk mitigation capability becomes critical to meet this challenge. This paper proposes and evaluates a computationally-efficient path planning approach to perform pre-flight planning and autonomous in-flight re-routing to minimize exposures to selected hazards. In our evaluation, hazards associated with degraded and missing critical GPS navigation data are considered. In this paper, we first present a high-performance computing path planning approach based on an adapted Bellman-Ford algorithm, developed in the CUDA programming language. Using the adapted path planning algorithm, we test this algorithm when encountering issues with GPS quality, and deliver an implementation that can produce flight paths that minimize exposure to risks, while maintaining a low computational burden. In our evaluation, the computation of periodic and aperiodic path updates are evaluated, prioritizing specific events as triggers for updates, based on changes to satellite availability. These critical events can lead to significant exposure to navigational hazards if not dealt with correctly.
The emergence and development of advanced technologies and vehicle types have created a growing demand for new forms of flight operations. These new and increasingly complex operational paradigms, such as Advanced and Urban Air Mobility (AAM/UAM), present regulatory authorities and the aviation community with several design-and-implementation challenges – particularly for highly autonomous vehicles. An overarching and daunting task is to develop protocols that can integrate these operations without compromising safety or disrupting traditional airspace operations. A shift toward a more predictive, autonomous, risk mitigation capability becomes critical to meet this challenge. This paper proposes and evaluates a computationally-efficient path planning approach to perform pre-flight planning and autonomous in-flight re-routing to minimize exposures to selected hazards. In our evaluation, hazards associated with degraded and missing critical GPS navigation data are considered. In this paper, we first present a high-performance computing path planning approach based on an adapted Bellman-Ford algorithm, developed in the CUDA programming language. Using the adapted path planning algorithm, we test this algorithm when encountering issues with GPS quality, and deliver an implementation that can produce flight paths that minimize exposure to risks, while maintaining a low computational burden. In our evaluation, the computation of periodic and aperiodic path updates are evaluated, prioritizing specific events as triggers for updates, based on changes to satellite availability. These critical events can lead to significant exposure to navigational hazards if not dealt with correctly.
Scientific workflows increasingly involve both HPC and machine-learning tasks, combining MPI-based simulations, training, and inference in a single execution. Launchers such as Slurm’s srun constrain concurrency and throughput, making them unsuitable for dynamic and heterogeneous workloads. We present a performance study of RADICAL-Pilot (RP) integrated with Flux and Dragon, two complementary runtime systems that enable hierarchical resource management and high-throughput function execution. Using synthetic and production-scale workloads on Frontier, we characterize the task execution properties of RP across runtime configurations. RP+Flux sustains up to 930 tasks/s, and RP+Flux+Dragon exceeds 1,500 tasks/s with over 99.6% utilization. In contrast, srun peaks at 152 tasks/s and degrades with scale, with utilization below 50%. For IMPECCABLE.v2 drug discovery campaign, RP+Flux reduces makespan by 30–60% relative to srun/Slurm and increases throughput more than four times on up to 1,024. These results demonstrate hybrid runtime integration in RP as a scalable approach for hybrid AI-HPC workloads.
The Joint Laboratory for Extreme-Scale Computing (JLESC) focuses on software challenges in high-performance computing systems to meet the needs of today’s science campaigns, which often require large resources, consist of multiple tasks, and generate vast amounts of data. In this context, extreme-scale workflows have been the key factor in enabling scientific discoveries by helping scientists automate the dependencies and data exchanges between workflow tasks, instead of managing those manually. Here, in this paper, we present representative extreme-scale workflows and feature workflow systems developed by JLESC participating institutions. We present lessons learned while developing these tools, alongside with the open challenges and future research directions in the field of extreme-scale workflows.
The HFIR Activity WorkbooK generator (HAWK) is a Python code that automates and streamlines the activity calculation of samples after irradiation in the High Flux Isotope Reactor (HFIR). HAWK’s results provide estimates of the activity and nuclide inventory of irradiated specimens before they are moved to hot cell facilities, where they undergo post-irradiation examination. The samples’ activity results guide the packing of shipping containers and inform the accountable inventories for the hot cell facilities. The toolkit was originally developed by Charles Daily, a former R&D staff member at Oak Ridge National Laboratory (ORNL). As of May 2025, HAWK is developed by the Radiation Transport & HPC Methods Group (Nuclear Energy and Fuel Cycle Division) at ORNL. Figure 1 presents HAWK’s workflow. To use HAWK, users need to: 1. Develop an Excel input workbook (i.e., XLSX extension) containing data from the experiment’s materials, irradiation history (cycles), and irradiation positions. 2. Make minor edits to an existing template JSON file (i.e., auxiliary_data.JSON) and to the Python driver. The driver sets the necessary environment variables, defines the material compositions, and ultimately calls HAWK. Once configured, HAWK runs the Oak Ridge Isotope Generation code (ORIGEN) to calculate the masses, activities, and heat load at the end of irradiation for each isotope in the specimen. ORIGEN is part of SCALE, ORNL’s in-house computational tool for performing nuclear safety and design calculations. Following this step, HAWK postprocesses the results and generates three output workbooks summarizing the activity calculations.
The power & energy demands of HPC machines have grown significantly. Modern exascale HPC systems require tens of megawatts of combined power for computing resources and cooling facilities at full capacity. The current energy trend is not sustainable for future HPC systems, and there is a need to work toward the energy efficiency aspect of HPC performance. Energy awareness of the HPC applications at the job level is essential for running an efficient HPC system. This work aims to develop a pipeline to provide a production-level system-wide overview of the HPC workloads' power profile while handling evolving workloads exhibiting new power trends. We developed an open-set classification model for HPC jobs based on the properties of power profiles to continuously provide a system-wide holistic view of recently completed jobs. The pipeline helps continuously monitor the job-level power usage pattern of HPC and enables us to capture the new trends in applications' power behavior. We employed a comprehensive set of techniques to generate job-level data, custom-designed feature extraction methods to extract critical features from jobs' power profiles, clustering techniques powered by generative modeling, and open-set classification for identifying job profiles into known classes or an unknown set. With extensive evaluations, we demonstrate the effectiveness of each component in our pipeline. We provide an analysis of the resulting clusters that characterize the power profile landscape of the Summit supercomputer from more than 60K jobs executed in a year. The open-set classification classifies the known data sets into known classes with high accuracy and identifies unknown data noints with over 85% accuracy.
Batched sparse linear algebra operations in general, and solvers in particular, have become the major algorithmic development activity and foremost performance engineering effort in the numerical software libraries work on modern hardware with accelerators such as GPUs. Many applications, ECP and non-ECP alike, require simultaneous solutions of many small linear systems of equations that are structurally sparse in one form or another. In order to move towards high hardware utilization levels, it is important to provide these applications with appropriate interface designs to be both functionally efficient and performance portable and give full access to the appropriate batched sparse solvers running on modern hardware accelerators prevalent across DOE supercomputing sites since the inception of ECP. To this end, we present here a summary of recent advances on the interface designs in use by HPC software libraries supporting batched sparse linear algebra and the development of sparse batched kernel codes for solvers and preconditioners. We also address the potential interoperability opportunities to keep the corresponding software portable between the major hardware accelerators from AMD, Intel, and NVIDIA, while maintaining the appropriate disclosure levels conforming to the active NDA agreements. The presented interface specifications include a mix of batched band, sparse iterative, and sparse direct solvers with their accompanying functionality that is already required by the application codes or we anticipated to be needed in the near future. This report summarizes progress in Kokkos Kernels and the xSDK libraries MAGMA, Ginkgo, hypre, PETSc, and SuperLU.
Background In 2022, the Office of Science and Technology Policy (OSTP) issued a memo that significantly reshaped the landscape of access to federally funded research. The memo mandated that all taxpayer-funded research be made available to the public without delay upon publication, without an embargo period, superseding the 2013 OSTP public access policy. This public access policy promotes transparency and the democratization of knowledge, ensuring that the fruits of scientific endeavors funded by federal agencies could be immediately accessed and built upon by scientists, educators, students, and the public at large. To implement the requirements of the OSTP guidance and DOE Public Access Plan, the Office of Nuclear Energy (NE) has implemented public access plan guidance and has identified several areas where better data management practices would further expand public access to important nuclear energy related scientific data, reports, and other technical products. Significant NE supported efforts are already underway for data management and public access to important nuclear energy related data.1 2 To address gaps in data management practices, and improve retention and accessibility of data, NE is actively exploring enhanced data management options utilizing its high-performance computing resources administered by its Nuclear Scientific User Facility Program. A newly piloted system, the Nuclear Research Data System (NRDS) acts as a portal for data collection and dissemination. Nuclear Energy University Program Research and Development Portfolio According to Web of Science, NEUP has produced 2,345 journal publication that have been cited more than 61,000 times3 and countless conference proceedings. These publications are publicly available through OSTI.gov and in the open literature. Additional scientific and technical products including project milestones that are not publications and NEUP project final reports are vetted through OSTI.gov and released once reviewed and approved by DOE. Since 2009, NEUP has awarded close to 1,000 different R&D projects in technical areas across the NE research programs. As of June 2023, 512 NEUP reports are publicly available on OSTI. The underlying data for projects is still held at universities, and data transfer, co-location, and dissemination has not occurred in a systematic way. NEUP data is currently accessible through myriad university-based data repositories, or through direct requests to PIs. The program identified this patchwork of repositories, or often lack of publicly available data, as a significant barrier to an organized, accessible, and comprehensive solution to sharing data with the larger nuclear energy community. Approach The goal of this pilot project is to establish a pathway to a consolidated long-term repository for NEUP project data. To accomplish this goal, the pilot strives to accomplish the following objectives: Establish data collection standards, including a standard set of required supplementary information to contextualize and support raw data files. Work with the HPC group collect and upload information and to modify the NRDS system, as needed, to support a standardized approach. Resolve potential barriers to successful roll out of an expanded data collection strategy, including modifying data management plan guidelines and establishing a document and data release process that accounts for potential intellectual property and/or export control concerns. Results Overall, the pilot was successful in collecting 8,982 raw and processes data files, 220 reports, 56 calibration files, and 5,931 other supplementary documents. Supplementary documents included experimental plans, methods, journal publications and conference proceedings, milestone reports, and final reports. Figure 2 shows the number of data sets and supplementary project information provided by each project. Projects has significantly different input, depending on experimental data produced and completeness of the datasets provided.
In this paper, we explore using the Harrow–Hassidim–Lloyd (HHL) algorithm to address scientific and engineering problems through quantum computing, utilizing the NWQSim simulation package on a high-performance computing platform. Focusing on domains such as power-grid management and climate projection, we demonstrate the correlations of the accuracy of quantum phase estimation, along with various properties of coefficient matrices, on the final solution and quantum resource cost in iterative and non-iterative numerical methods such as the Newton–Raphson method and finite difference method, as well as their impacts on quantum error correction costs using the Microsoft Azure Quantum resource estimator. We summarize the exponential resource cost from quantum phase estimation before and after quantum error correction and illustrate a potential way to reduce the demands on physical qubits. This work lays down a preliminary step for future investigations, urging a closer examination of quantum algorithms’ scalability and efficiency in domain applications.
ExaFEL is an HPC-capable X-ray Free Electron Laser (XFEL) data analysis software suite for both Serial Femtosecond Crystallography (SFX) and Single Particle Imaging (SPI) developed in collaboration with the Linac Coherent Lightsource (LCLS), Lawrence Berkeley National Laboratory (LBNL) and Los Alamos National Laboratory. ExaFEL supports real-time data analysis via a cross-facility workflow spanning LCLS and HPC centers such as NERSC and OLCF. Our work therefore constitutes initial path-finding for the US Department of Energy's (DOE) Integrated Research Infrastructure (IRI) program. We present the ExaFEL team's 7 years of experience in developing real-time XFEL data analysis software for the DOE's exascale supercomputers. We present our experiences and lessons learned with the Perlmutter and Frontier supercomputers. Furthermore we outline essential data center services (and the implications for institutional policy) required for real-time data analysis. Finally we summarize our software and performance engineering approaches and our experiences with NERSC's Perlmutter and OLCF's Frontier systems. This work is intended to be a practical blueprint for similar efforts in integrating exascale compute resources into other cross-facility workflows.
As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. Here, in this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI’s progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.