SEARCH · Search NASA

Results for “HPC”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

All sources NASA NTRS DOE OSTI

DOE OSTI2025

dCache: The Storage System of Choice for Data-Intensive Applications

The ever-increasing volumes of data produced by modern scientific facilities like EuXFEL and LHC put significant stress on data management infrastructure operated by laboratories and research centers. The challenges to be addressed span the entire data life cycle, from ingest and efficient data analysis to long-term preservation, typically involving large tape libraries. dCache, a storage system developed in collaboration between the Deutsches Elektronen-Synchrotron (DESY), Fermi National Accelerator Laboratory, and Nordic e-Infrastructure Collaboration (NeIC), is designed to manage a large number of disk servers and to facilitate transparent data migration to and from archival storage. Its multifaceted approach offers a unified method to support a variety of scientific use cases with the same storage infrastructure, including high-throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters, and long-term data preservation on tertiary storage. Initially developed for high energy physics (HEP) experiments, dCache is now used by various scientific communities, including astrophysics, biomedical research, and life sciences, each having specific requirements. This paper presents architecture, deployment strategies, performance and scalability enhancements, and recent advancements in dCache addressing the needs of scientific communities. Finally, we touch on the development and release process, ensuring the software’s high quality.

DCache↗

DOE OSTI2024

Progressive Hedging Decomposition for Solutions of Large-Scale Process Family Design Problems

Rapid, wide-scale deployment of green process systems, such as carbon capture or water desalination systems, is essential for combatting climate change. Methods relying on traditional design or modularity fail to capture the benefits of both economies of numbers and economies of scale. We have proposed process family design, which designs a family of processes simultaneously exploiting opportunities for common elements. In previous work, we explored different optimization formulations to solve this problem. In this work, we develop a decomposition approach to tackle larger problems efficiently. We solve a water desalination case study, which is too large to solve within a reasonable timeframe with the discretization formulation. We exploit the block angular structure of the discretization problem to decompose and solve using Progressive Hedging (PH). We use the open-source Python package mpi-sppy to execute PH which allows us to leverage parallelization and a HPC cluster to further improve solution time.

Stinchfield, Georgia↗

DOE OSTI2024

Numerical simulation of involute-plate research reactor flow behavior using RANS, LES and DNS

This paper investigates the flow behavior of involute-plate research reactors by performing Reynolds-Averaged Navier Stokes simulation (RANS), Large Eddy Simulation (LES) and Direct Numerical Simulation (DNS) of the channel flow between fuel plates. By modeling turbulence with different numerical approaches, this study provides data with three levels of fidelity. For the RANS simulation, three widely used turbulence models, i.e., k-ε, k-ω, Reynolds Stress Turbulence model (RST) are applied by using the commercial CFD code STAR-CCM +. For LES and DNS, the open-source CFD code, Nek5000, is used given its outstanding scalability on High Performance Computer (HPC) and high-order technique. The results from RANS simulations are compared with that from LES and DNS for benchmarking. Both macroscale parameters and turbulence statistics, such as velocity magnitude, lateral velocity and turbulence kinetic energy, are presented and analyzed. The results from RANS simulation achieve good agreement with LES and DNS on velocity and turbulence kinetic energy prediction. The RST turbulence model predicts the most similar flow pattern of lateral velocity as compared to LES and DNS. The Lambda-2 (λ2) criterion with a reasonable threshold is used to demonstrate the instantaneous vortices distribution in the involute channel from both LES and DNS calculation. The DNS simulation captures more detailed turbulence especially near the corner, which explains the discrepancy between LES and DNS results near the corner. The normalized RMS error are defined and calculated to assess the performance of those turbulence models. The RST model captures the anisotropic feature of turbulence, which enable it to outperform other turbulence models for predicting the flow behavior in an involute channel. Although some discrepancies are found between LES and DNS results in the corner, the overall deviations between LES and DNS are found to be small. In conclusion, given that the computational cost of DNS calculation is an order of magnitude higher, using LES data for benchmarking RANS model is a cost-effective approach.

DNS↗

DOE OSTI2024

A parallel and performance portable implementation of a full-field crystal plasticity model

We have developed a parallel implementation of an Elasto-Viscoplastic Fast Fourier Transform-based (EVPFFT) micromechanical solver to enable computationally efficient crystal plasticity modeling for polycrystalline materials. Our primary focus lies in achieving performance portability, allowing a single EVPFFT implementation to run optimally on various homogeneous architectures, including multi-core Central Processing Units (CPUs), as well as on heterogeneous computer architectures comprising multi-core CPUs and Graphics Processing Units (GPUs) from different vendors. To accomplish this goal, we have leveraged MATAR, a C++ software library that simplifies the creation and utilization of multidimensional dense or sparse matrix and array data structures. These data structures are designed to be portable across diverse architectures through the use of Kokkos, a performance-portable library. Additionally, we have employed the Message Passing Interface (MPI) to efficiently distribute the computational workload among processors. The heFFTe (Highly Efficient FFT for Exascale) library is used to facilitate the performance portability of the fast Fourier transforms (FFTs) computation. The computational performance of EVPFFT is evaluated and presented in terms of parallel scalability and simulation runtime on different high-performance computing (HPC) architectures. As a result, the utility of the developed framework to efficiently simulate the micro-mechanical fields in polycrystalline microstructures in engineering applications is discussed.

36 MATERIALS SCIENCE↗

DOE OSTI2024

Efficient distributed continual learning for steering experiments in real-time

Deep learning has emerged as a powerful method for extracting valuable information from large volumes of data. However, when new training data arrives continuously (i.e., is not fully available from the beginning), incremental training suffers from catastrophic forgetting (i.e., new patterns are reinforced at the expense of previously acquired knowledge). Training from scratch each time new training data becomes available would result in extremely long training times and massive data accumulation. Rehearsal-based continual learning has shown promise for addressing the catastrophic forgetting challenge, but research to date has not addressed performance and scalability. To fill this gap, we propose an approach based on a distributed rehearsal buffer that efficiently complements data-parallel training on multiple GPUs to achieve high accuracy, short runtime, and scalability. It leverages a set of buffers (local to each GPU) and uses several asynchronous techniques for updating these local buffers in an embarrassingly parallel fashion, all while handling the communication overheads necessary to augment input minibatches using unbiased, global sampling. We further propose a generalization of rehearsal buffers to support both classification and generative learning tasks, as well as more advanced rehearsal strategies (notably Dark Experience Replay, leveraging knowledge distillation). We illustrate this approach with a real-life HPC streaming application from the domain of ptychographic image reconstruction. Furthermore, we run extensive experiments on up to 128 GPUs of the ThetaGPU supercomputer to compare our approach with baselines representative of training-from-scratch (the upper bound in terms of accuracy) and incremental training (the lower bound). Results show that rehearsal-based continual learning achieves a top-5 validation accuracy close to the upper bound, while simultaneously exhibiting a runtime close to the lower bound.

Asynchronous data management↗

DOE OSTI2025

Multi-physics melt pool modeling and process optimization for laser direct energy deposition of Nb-based refractory C103: Defect formation, geometric precision, and process mapping

Recent developments in additive manufacturing (AM) technology have reignited interest in the fabrication of the Nb-based refractory C103 alloy offering solutions to the challenges posed by traditional manufacturing methods. However, the limited numerical and experimental studies on laser direct energy deposition (DED) of C103 have hindered the understanding of the relationships between process parameters and build quality. This has made it challenging to consistently produce parts with the desired quality and microstructure suitable for critical applications. In this study, we focus on optimizing the laser DED process for C103 by employing a hybrid approach that combines experimental techniques and computational fluid dynamics (CFD). This approach facilitates the development of process maps for defect detection and geometric precision. To achieve this, multi-layer C103 samples were fabricated using laser DED under various process parameters, enabling the creation of a process map for defect detection. Additionally, a multi-physics, multiphase simulation framework was developed within a high-performance computing (HPC) environment to establish process maps for geometric precision. Using these process maps, printability windows were identified for achieving both the desired geometric accuracy and defect-free prints. It was observed that prints with a power-to-velocity (P/V) ratio close to unity resulted in defect-free outcomes. This study provides a foundation for reducing design lead time and rejected parts, ultimately optimizing the laser DED process for C103.

Defect formation and geometric precision↗

DOE OSTI2025

ED-cPSD: Fast Phase-Size Distribution via Sequential Erosion-Dilation

The Erosion-Dilation continuous Phase-Size Distribution, ED-cPSD, is an application for calculating continuous pore and particle-size distribution from digital reconstructions and/or image-based structural data. It is based on the erosion-dilation continuous phase-size distribution method. A continuous size distribution is a measure of the probability density of finding a particle or pore of a certain size. These distributions are of interest in any field of study involving porous media, including but not limited to electrochemistry, petroleum engineering, geology, and food science. The algorithm behind the software provides a computationally efficient way to calculate phase-size distributions for large domains. For a 3D battery electrode reconstruction with 1.3 x 10 8 voxels, the particle size distribution is derived in under 2 min on a desktop, while also retaining flexibility and computational efficiency for HPC-scale multi-threading. The software can handle structures with over 10 9 voxels. The algorithm is roughly 280 times faster than a previous version on the same task.

Characterization↗

DOE OSTI2025

Real-space Kohn–Sham density functional theory for complex energy applications

Real-space Kohn-Sham density functional theory (real-space KS-DFT) enables large-scale electronic structure simulations that is particularly well-suited for the modern high-performance computing (HPC) architectures. This feature article reviews its theoretical foundations, highlights the algorithmic advances and recent developments, and showcases applications in complex nano systems. We aim to provide a perspective on the trajectory of real-space KS-DFT as an emerging tool for computational chemistry and materials science in the exascale era.

Zhang, Zeyi↗

DOE OSTI2024

DUNE Database Development

The DUNE experiment will produce vast amounts of metadata, which describe the data coming from the read-out of the primary DUNE detectors. Various databases will make up the overall DB architecture for this metadata. ProtoDUNE at CERN is the largest existing prototype for DUNE and serves as a testing ground for - among other things - possible database solutions for DUNE. The subset of all metadata that is accessed during offline data reconstruction and analysis is referred to as ‘conditions data’ and it is stored in a dedicated database. As offline data reconstruction and analysis will be deployed on HTC and HPC resources, conditions data is expected to be accessed at very high rates. It is therefore crucial to store it in a granularity that matches the expected access patterns allowing for extensive caching. This requires a good understanding of the sources and use cases of conditions data. This contribution will briefly summarize the database architecture deployed at ProtoDUNE and explain the various sources of conditions data. We will present how the conditions data is retrieved and streamed from the databases and how it is handled to match expected access patterns.

Vizcaya Hernandez, Ana Paula↗

DOE OSTI2025

dCache project status and update

The dCache project delivers an open-source, massively scalable, distributed storage system deployed internationally to satisfy today’s scientists’ ever-demanding storage requirements. Its multifaceted approach supports different use cases with the same storage, from high throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters, and longterm data persistence on tertiary storage. Even though dCache was initially developed for HEP experiments, today, it is used by various scientific communities, including astrophysics, biomed, and life science, each with their specific requirements. To match the needs of these new communities and keep up with the scaling demands of existing experiments, dCache is permanently evolving. With this contribution, we would like to highlight the recent developments in dCache regarding integration with CERN Tape Archive (CTA), advanced metadata handling, token-based authorization support, bulk API for QoS transitions, REST API to control interaction with the tape system, and future development directions.

Mkrtchyan, Tigran [DESY]↗

DOE OSTI2025

Using the ATLAS experiment software on heterogeneous resources

With the large dataset expected from 2030 onwards by the HL-LHC at CERN, the ATLAS experiment is reaching the limits of the current data processing model in terms of traditional CPU resources based on x86_64 architectures and an extensive program for software upgrades towards the HL-LHC has been set up. The ARM CPU architecture is becoming a competitive and energy efficient alternative. Accelerators like GPUs are available in any recent HPC. In the past years ATLAS has successfully ported its full data processing and simulation software framework Athena to ARM and has invested significant effort in porting parts of the reconstruction and simulation algorithms to GPUs. We report on the successful usage of the ATLAS experiment offline and online software framework Athena on ARM and GPUs through the PanDA workflow management system at various WLCG sites. Furthermore we report on performance optimizations of the builds for ARM CPUs and the GPU integration efforts. We will discuss performance comparisons of different ARM and x86_64 architectures on WLCG resources and Cloud compute providers like GCP and AWS using ATLAS productions workflows as used in the Hep-Score23 benchmark suite.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

Optimization of distributed compute resources utilization in the CMS Global Pool

The CMS Submission Infrastructure is the primary system for managing computing resources for CMS workflows, including data processing, simulation, and analysis. It integrates geographically distributed resources from Grid, HPC, and cloud providers into federated pools managed by HTCondor and Glidein- WMS, for a total of around 500k CPU cores. This system dynamically manages workloads based on priorities defined by the collaboration. Additionally, CMS scheduling strategies must be flexible to handle multiple concurrent workloads while considering changing processing demands and resource availability from various providers.Efficient utilization of vast amounts of distributed compute resources is a key element for the success of the scientific programs of the LHC experiments. Optimizing the system is essential to maximize resource efficiency and fully utilize the distributed computing power. The CMS Submission Infrastructure team thus systematically investigates sources of inefficiency in workload scheduling to reduce their impact. In addition, a strategy of pilot overloading has been introduced to compensate for other inefficiency sources, thereby optimizing resource utilization and enhancing computational throughput.

Mascheroni, Marco [UC, San Diego (main)]↗

DOE OSTI2025

ROOT RNTuple and EOS: The Next Generation of Event Data I/O

For several years, the ROOT team is developing the new RNTuple I/O subsystem in preparation of the next generation of collider experiments. Both HL-LHC and DUNE are expected to start data taking by the end of this decade. They pose unprecedented challenges to event data I/O in terms of data rates, event sizes, and event complexity. At the same time, the I/O landscape is becoming more diverse. HPC cluster file systems and object stores, NVMe disk cache layers in analysis facilities, and S3 storage on cloud resources are mixing with traditional XRootD-managed spinning disk pools.The ROOT team will finalize a first production version of the RNTuple binary format by the end of 2024. After this point, ROOT will provide backward compatibility for RNTuple data. This contribution provides an overview of the RNTuple feature set, the related R&D activities and the long-term vision for RNTuple. We report on performance, interface design, tooling, robustness, integration with experiment frameworks, and validation results, as well as recent R&D on parallel reading and writing and exploitation of modern hardware and storage systems. We will give an outlook on possible future features after a first production release.Collaboratively, the IT and EP departments at CERN have launched a formal project within the Research and Computing sector to evaluate the novel data format for physics analysis data utilized in LHC experiments and other fields. This part of the project focuses on validating the scalability of the EOS storage backend during the transition from the over 25 years old TTree production format to the newly developed RNTuple format, using both replicated and erasure-coded storage profiles.

Blomer, Jakob [CERN]↗

DOE OSTI2024

Electronic structure simulations in the cloud computing environment

The transformative impact of modern computational paradigms and technologies, such as high-performance computing, quantum computing, and cloud computing, has opened up profound new opportunities for scientific simulations. Scalable computational chemistry is one beneficiary of this technological progress. The main focus of this paper is on the performance of various quantum chemical formulations, ranging from low-order methods to high-accuracy approaches, implemented in different computational chemistry packages, such as NWChem, NWChemEx, SPEC, ExaChem, and FLOSIC codes on the Azure Quantum Element (AQE) Microsoft cloud services. We pay particular attention to the intricate workflows for performing composite chemistry simulations, associated data curation, and mechanisms for accuracy assessment, as defined by the enabling cloud Computational Chemistry as a Service (CCaaS). Our focus also extends to Arrows' automated workflow for high throughput simulations. Finally, we provide a perspective on the role of cloud computing in supporting the mission of leadership computational facilities (LCFs).

computational chemistry, electronic structure, Clo↗

DOE OSTI2025

LCLS Big Data Handling – How I Learned to Stop Worrying and Love the Data Deluge

Advanced data and computing systems are vital to Linac Coherent Light Source (LCLS) operations, data interpretation and overall scientific productivity. The transition to MHz-era operation marks a fundamental change in scale that requires new infrastructure and architectures to link LCLS to the required scale of computing needed for scientific interpretation. The LCLS-II Data System meets big data challenges by implementing configurable data reduction that can adapt to multiple science areas, real-time analysis frameworks to provide visualization and fast feedback, and the ability to transfer data to local and remote computational facilities for near real time analysis at the appropriate scale. Feature extracted information generated in the data analysis pipeline - at the edge, local compute, or remote High-Performance Computing (HPC) resources - can be used to steer experiments and inform user decisions during beam time. Artificial Intelligence and Machine Learning (AI/ML) techniques present new opportunities to rapidly analyse large datasets and direct experiments, but create new challenges in scaling, adaptability, complexity, and trustworthiness. We describe how the LCLS-II Data System architecture addresses its data-driven challenges in the areas of data acquisition, data processing, data management, and workflow orchestration to decrease the overall time-to-science and provide a vision for future developments.

artificial intelligence↗

DOE OSTI2026

Mathematical Models and Numerical Methods for High-Fidelity Simulation of Ignition of Reactive Mixtures by Nanosecond Plasma Discharges in Realistic Configurations

We present a newly developed framework for the numerical simulation of ignition of reactive mixtures using single or repeated nanosecond discharge pulses. The framework builds upon the AMReX library, using the existing compressible solver PeleC and low-Mach solver PeleLMeX and allowing for adaptive mesh refinement, complex geometries, and execution on next-generation high-performance computing (HPC) systems. High-fidelity elementary models are adopted for weakly-ionised plasma discharges with significant energy deposition, consistent with nanosecond discharge pulses, and then implemented in the solver. The treatment of non-thermal electrons and charged species, thermodynamics of non-equilbrium species, plasma kinetics, limiting time scales, and boundary conditions for charged species are discussed and addressed for computational efficiency. The framework is demonstrated for three relevant applications: single and multi-pulse discharges in air, single pulse ignition of an ethylene/air mixture, and a three-dimensional plasma discharge in air with temperature stratification. The successful application of the framework demonstrates the feasibility of high-fidelity simulation of ignition of air/hydrocarbon mixtures in three-dimensions with multiple discharge pulses.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2026

SAIGE-GPU: accelerating genome- and phenome-wide association studies using GPUs

Genome-wide association studies (GWAS) at biobank scale are computationally intensive, especially for admixed populations requiring robust statistical models. SAIGE is a widely used method for generalized linear mixed-model GWAS but is limited by its CPU-based implementation, making phenome-wide association studies impractical for many research groups. We developed SAIGE-GPU, a GPU-accelerated version of SAIGE that replaces CPU-intensive matrix operations with GPU-optimized kernels. The core innovation is distributing genetic relationship matrix calculations across GPUs and communication layers. Applied to 2068 phenotypes from 635 969 participants in the Million Veteran Program, including diverse and admixed populations, SAIGE-GPU achieved a 5-fold speedup in mixed model fitting on supercomputing infrastructure and cloud platforms. We further optimized the variant association testing step through multi-core and multi-trait parallelization. Deployed on Google Cloud Platform and Azure, the method provided substantial cost and time savings. Source code and binaries are available for download at https://github.com/saigegit/SAIGE/tree/SAIGE-GPU-1.3.3. A code snapshot is archived at Zenodo for reproducibility (DOI: [10.5281/zenodo.17642591]). SAIGE-GPU is available in a containerized format for use across HPC and cloud environments and is implemented in R/C++ and runs on Linux systems.

Rodriguez, Alex [Argonne National Laboratory (ANL)↗

DOE OSTI2026

Meta2DB: Curated Shotgun Metagenomic Feature Sets and Metadata for Health State Prediction

Meta2DB is a curated metagenomic and metadata database that provides structurally consistent microbiome taxonomy feature count tables for 13 897 samples across 84 studies, 23 disease states, and 34 geographical locations. All samples were uniformly processed using a streamlined metagenomic classification pipeline that employs a unique and comprehensive reference database indexed to contain all sequences across all kingdoms of life that were present in the NCBI Nucleotide (nt) database retrieved on 4 January 2023. This pipeline leverages high-performance computing (HPC) resources at Lawrence Livermore National Laboratory and was used to process 50TB of publicly available raw metagenomic sequence data. Extensive metadata curation was carried out through a combination of manual curation and automated parsing, producing a consistent inter-study metadata table specifically structured to facilitate training of ML models for prediction of human health.

Kok, C [Lawrence Livermore National Laboratory (LL↗