SEARCH · Search NASA

Results for “High Performance Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

All sources NASA NTRS DOE OSTI

DOE OSTI2025

I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey

Growing interest in Artificial Intelligence (AI) has resulted in a surge in demand for faster methods of Machine Learning (ML) model training and inference. This demand for speed has prompted the use of high performance computing (HPC) systems that excel in managing distributed workloads. Because data is the main fuel for AI applications, the performance of the storage and I/O subsystem of HPC systems is critical. In the past, HPC applications accessed large portions of data written by simulations or experiments or ingested data for visualizations or analysis tasks. ML workloads perform small reads spread across a large number of random files. This shift of I/O access patterns poses several challenges to modern parallel storage systems. In this paper, we survey I/O in ML applications on HPC systems, and target literature within a 6-year time window from 2019 to 2024. We define the scope of the survey, provide an overview of the common phases of ML, review available profilers and benchmarks, examine the I/O patterns encountered during offline data preparation, training, and inference, and explore I/O optimizations utilized in modern ML frameworks and proposed in recent literature. Lastly, we seek to expose research gaps that could spawn further R&D.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

OmniFed: A Modular Framework for Configurable Federated Learning from Edge to HPC

Federated Learning (FL) is critical for edge and High Performance Computing (HPC) where data is not centralized and privacy is crucial. We present OmniFed, a modular framework designed around decoupling and clear separation of concerns for configuration, orchestration, communication, and training logic. Its architecture supports configuration-driven prototyping and code-level override-what-you-need customization. We also support different topologies, mixed communication protocols within a single deployment, and popular training algorithms. It also offers optional privacy mechanisms including Differential Privacy (DP), Homomorphic Encryption (HE), and Secure Aggregation (SA), as well as compression strategies. These capabilities are exposed through well-defined extension points, allowing users to customize topology and orchestration, learning logic, and privacy/compression plugins, all while preserving the integrity of the core system. We evaluate multiple models and algorithms to measure various performance metrics. By unifying topology configuration, mixed-protocol communication, and pluggable modules in one stack, OmniFed streamlines FL deployment across heterogeneous environments. Github repository is available at https://github.com/at-aaims/OmniFed.

Tyagi, Sahil [ORNL] (ORCID:0009000783144745)↗

DOE OSTI2025

HPC Digital Twins for Evaluating Scheduling Policies, Incentive Structures and their Impact on Power and Cooling

Schedulers are critical for optimal resource utilization in high-performance computing. Traditional methods to evaluate sched- ulers are limited to post-deployment analysis, or simulators, which do not model associated infrastructure. In this work, we present the first-of-its-kind integration of scheduling and digital twins in HPC. This enables what-if studies to understand the impact of parameter configurations and scheduling decisions on the physical assets, even before deployment, or regarching changes not easily realizable in production. We (1) provide the first digital twin framework extended with scheduling capabilities, (2) integrate various top-tier HPC systems given their publicly available datasets, (3) implement extensions to integrate external scheduling simulators. Finally, we show how to (4) implement and evaluate incentive structures, as- well-as (5) evaluate machine learning based scheduling, in such novel digital-twin based meta-framework to prototype scheduling. Our work enables what-if scenarios of HPC systems to evaluate sustainability, and the impact on the simulated system.

Maiterth, Matthias [ORNL] (ORCID:000000018698460X)↗

DOE OSTI2025

Characterizing the Impact of GPU Power Management on an Exascale System

As GPU-accelerated high-performance computing (HPC) systems approach exascale performance, controlling energy consumption without compromising throughput is essential. Architectures such as the AMD MI250X-based Frontier supercomputer provide runtime mechanisms like frequency and power capping, enabling energy tuning without modifying application code. Although both target energy reduction, they operate via distinct hardware control paths and influence workloads differently. We present a comprehensive evaluation of these strategies on a leadership-class system using diverse HPC proxy applications representative of production workloads. Our study analyzes performance–energy trade-offs across multiple capping levels, node counts (1 and 32), and application profiles. Results show that frequency capping generally achieves higher energy efficiency and scalability, with gains of up to 13.2% without performance loss, while power capping is more effective for single-node runs or bursty GPU utilization. We also provide practical guidelines to help system administrators and users balance energy efficiency and performance in large-scale scientific workloads.

Costa, Mariana [Universidade Federal do Rio Grande↗

DOE OSTI2025

A Survey on Error-Bounded Lossy Compression for Scientific Datasets

Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this article, we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into six classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 47 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data.

Error-Bounded Lossy Compression↗

DOE OSTI2025

Data Readiness for Scientific AI at Scale

This paper examines how Data Readiness for AI (DRAI) principles apply to leadership-scale scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, bio/health, and materials—to identify common preprocessing patterns and domain-specific constraints. We introduce a two-dimensional readiness framework that combines canonical preprocessing patterns with a five-level operational readiness scale, both tailored to high-performance computing (HPC) environments. This framework helps outline key challenges in transforming large-scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross-domain support for scalable and reproducible AI for science.

Brewer, Wes [ORNL] (ORCID:0000000236393956)↗

DOE OSTI2025

Enabling Command-and-Control in Advanced In Situ Workflows

Scientific discovery is progressing towards autonomous science with the combination of scientific instruments, high-performance computing, and artificial intelligence in complex workflows. This evolution introduces new requirements for managing scientific workflows, including feedback loops, near real-time constraints, and the ability to dynamically control workflow execution. In situ workflows that analyze and visualize data as it is generated are well-suited to satisfy stringent time constraints and their iterative nature offers greater opportunities for command-and-control. However, only a few of the many workflow management systems available have been specifically designed to manage in situ workflows and often lack support for automated feedback loops that allow analysis and visualization components to interact with the main scientific data producer. To address this need, we present in this paper how to add command-and-control capabilities to a workflow management system. We identify the functional design requirements of such a command-and-control system, detail its architecture, interface, and core mechanisms, and illustrate how advanced in situ workflows can leverage command-and-control in three use cases: graceful termination with checkpoint, dynamic and adaptive data reduction, and event-triggered analysis.

Mehta, Kshitij [ORNL] (ORCID:0000000297149981)↗

DOE OSTI2026

New Time Integrators and Capabilities in SUNDIALS Versions 6.2.0-7.4.0

SUNDIALS is a well-established numerical library that provides robust and efficient time integrators and nonlinear solvers. This article overviews several significant improvements and new features added over the last 3 years to support scientific simulations run on high-performance computing systems. Notably, three new classes of one-step methods have been implemented: low storage Runge–Kutta, symplectic partitioned Runge–Kutta, and operator splitting. In addition, we describe new timestep adaptivity support for multirate methods, adjoint sensitivity analysis capabilities for explicit Runge–Kutta methods, additional options for Anderson acceleration in nonlinear solvers, and improved error handling and logging.

Computer science↗

DOE OSTI2024

EchemAMR (electro-chemical microsctructure scale models with adaptive meshing) [SWR-23-111]

A 3D microstructure resolving electrochemical transport and interfacial chemistry solver. Electrode microstructure plays an important role in determining the performance of an electrochemical system, e.g. lithium ion battery. EchemAMR is a microstructure scale model that solves the governing equations for ion transport, electrical current continuity, interfacial chemistry and structural mechanics. Complex microstructure geometries from imaging can be directly imported into EchemAMR. A volume fraction based description of the geometry on Cartesian grid with an immersed interface formulation enables simplified meshing and large-scale simulations with millions of degrees of freedom. EchemAMR has been tested against systems with analytic solutions for numerical convergence and highly resolved lithium ion battery microstructures. EchemAMR demonstrates excellent mass conversation and efficient scaling on heterogenous High-Performance Computing (HPC) with central and graphics processing units.

Sitaraman, Hariswaran↗

DOE OSTI2024

via-wind (A Visual Impact Assessment Tool for Wind Turbines) [SWR-24-87]

Via-wind is an open-source tool for conducting visual impact assessments for wind turbines. It combines geographic information system (GIS) and 3D simulation methods to account for the key factors driving the visual impact of installed wind turbines, including distance, viewing angle, turbine orientation, visual exposure, and the cumulative effects of multiple turbines. This software is optimized for use in high-performance computing environments to enable large scale (e.g., country-wide) analysis, but can also be run on a single server or personal computer. For more information, please see the related journal article: https://www.sciencedirect.com/science/article/pii/S0306261924021846

Lopez, Anthony↗

DOE OSTI2025

VerifyIO: Verifying Adherence to Parallel I/O Consistency Semantics

VerifyIO is a tool designed for verifying I/O consistency semantics in High-Performance Computing (HPC) applications. It addresses the challenges of ensuring correctness and portability across different I/O consistency models, such as POSIX, Commit, Session, and MPI-IO. By analyzing execution traces, detecting conflicts, and verifying synchronization adherence, VerifyIO provides actionable insights for both application developers and I/O library designers.

Wang, Chen [Lawrence Livermore National Laboratory↗

DOE OSTI2025

GeoGridFusion (Open-Source Geospatial Toolkit for Solar Data Integration) [SWR-25-19]

GeoGridFusion facilitates the usage and storage of gridded geospatial satellite data by users outside of the National Renewable Lab (NREL), particularly those without access to high-performance computing (HPC) resources. This tool builds on work done by the PVDegradationTools project for DuraMAT, with the goal of making our advancements from this project widely accessible. This repo contains utilities to allow for the storage of user downloaded geospatial weather data by providing a local datastore for storage and spatial queries, supporting large-scale analyses without the need for HPC resources.

Ford, Tobin [National Renewable Energy Laboratory ↗

DOE OSTI2025

OLCF Test Harness

Acceptance and regression testing of a High Performance Computing (HPC) system requires an automated and reproducible framework and tool for running and logging results. Manually running tests across a system is labor intensive and prone to reproducibility errors. The OLCF Test Harness (OTH) provides a framework in which to document required tests for a HPC system. The OTH then provides tools to execute and log results of these tests in an automated fashion.

Dietz, Dan [Oak Ridge National Laboratory (ORNL), ↗

DOE OSTI2025

System Noise Benchmarks

This project includes benchmarks to assess the presence of system noise on supercomputers. System noise is any activity that interferes with the execution of high-performance computing applications.

Moody, AdamT [Lawrence Livermore National Laborato↗

DOE OSTI2025

BULKI-Store v0.3.2

BULKI-Store is a distributed object storage system optimized for high-performance computing environments. Built with a Rust core and Python bindings, it efficiently manages scientific and machine learning datasets across HPC clusters. The system employs a client-server architecture with MPI integration, enabling seamless scaling on supercomputers like Perlmutter. BULKI-Store's object-oriented approach provides intuitive data organization with rich metadata support, contrasting with traditional file-based solutions. Key optimizations include selective checkpoint loading, unified checkpoint files, and object chunking for large data transfers. For machine learning workloads, BULKI-Store offers advantages through fine-grained access patterns, dynamic data sharing between training instances, and reduced memory pressure. Memory management features include strategic Python GC calls, minimized data copies, and batch processing capabilities. The system leverages Rayon's thread pool for asynchronous data prefetching and supports multiple CPU architectures (ARM64, x86, AMD, RISC-V). By combining performance optimizations with developer-friendly APIs, BULKI-Store addresses the complex data management challenges of modern HPC applications while maintaining compatibility across heterogeneous computing environments.

Zhang, Wei [Lawrence Berkeley National Laboratory ↗

DOE OSTI2025

DOE-ICoM/RIFT

Rapid Infrastructure Flood Tool (RIFT) is a two-dimensional hydrodynamic model based on the complete shallow water equations. RIFT has specifically been designed with rapid simulation in mind by utilizing commodity high performance computing technology and best-available nation-wide data. RIFT is used to predict the movement of water over land and resolve the spatial and temporal variability of flood depths, extent, and velocity. RIFT can be applied to many flood situations and has primarily been used to quantify flood extents from dam/levee failure or inland rainfall flooding.

Perkins, Bill [Pacific Northwest National Laborato↗

DOE OSTI2025

Maskman

SAND2025-04369O Maskman is a user-friendly tool designed to create hex masks, which are essential for optimizing application performance in high-performance computing environments. By converting a list of integers into binary and then hex masks, Maskman simplifies the process of setting application affinity. This ensures that software runs efficiently on specific nodes within a computing cluster. Ideal for researchers and developers, Maskman streamlines the preparation of inputs for HPC schedulers, enhancing resource management and improving overall system performance. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Pase, Douglas [Sandia National Lab. (SNL-CA), Live↗

DOE OSTI2025

MiniMOD

SAND2025-03854O MiniMod is a user-friendly software tool designed to assess the performance of high-performance computing (HPC) systems. Researchers can use the program to test communication methods and computational tasks to understand how different setups can affect application efficiency. This software is particularly useful for optimizing network performance in scientific research, simulations, and data analysis. MiniMod‘s flexible design allows users to make informed decisions about their computing environments, which can enhance productivity and results in real-world applications. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Dosanjh, Matthew [Sandia National Lab. (SNL-CA), L↗