Search NASA⌕ Search

SEARCH · Search NASA

Results for “software parallel computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

PPI DataHub Project Data Package: S. elongatus PCC 7942 Limited Proteolysis and Thermal Proteome Profiling Structural Proteomics (JM-PB-DP3)

The purpose of this experiment was to investigate structural alterations in proteins involved in central carbon metabolism and photosynthetic electron transfer pathways in Synechococcus elongatus PCC 7942. Sample data was obtained from S. elongatus cell lysates using three complementary mass spectrometry (MS) techniques using limited proteolysis (LiP-MS), thermal proteome profiling (TPP-MS), and redox enrichment (Redox-MS) in evaluating alterations solvent accessibility and structural stability caused by light perturbation at the molecular level. Experimentally processed sample data for LiP and TPP proteomic datasets were derived from the same cell culture stock, prepared simultaneously in parallel, and acquired by mass spectrometry. Processed datasets are openly accessible from the download button and contain secondary processed proteomic results files, computed outputs, and supporting metadata materials. Experimental samples processed for LiP-MS label-free quantification (LFQ) or TPP-MS tandem mass tag (TMT) 10-plex were acquired using a Q-Exactive HF-X mass spectrometer and processed/compiled using either MSGF+ (v2024.03.26) or ​​​​PlexedPiper for proteome evaluation. Additional software supporting downstream proteomic analysis include FragPipe (v.4.0), MSFragger (v.22.1), and an adapted Microbial Isolate LiP Analysis Workflow (located at Zenodo). Processed proteomic data downloads include a sample naming key, normalized quantification results files, and processed protein annotated abundance files.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

ChatHPC: Building the Foundations for a Productive and Trustworthy AI-Assisted HPC Ecosystem

ChatHPC democratizes large language models for the high-performance computing (HPC) community by providing the infrastructure, ecosystem, and knowledge needed to apply modern generative AI technologies to rapidly create specific capabilities for critical HPC components while using relatively modest computational resources. Our divide-and-conquer approach focuses on creating a collection of reliable, highly specialized, and optimized AI assistants for HPC based on the cost-effective and fast Code Llama fine-tuning processes and expert supervision. We target major components of the HPC software stack, including programming models, runtimes, I/O, tooling, and math libraries. Thanks to AI, ChatHPC provides a more productive HPC ecosystem by boosting important tasks related to portability, parallelization, optimization, scalability, and instrumentation, among others. With relatively small datasets (on the order of KB), the AI assistants, which are created in a few minutes by using one node with two NVIDIA H100 GPUs and the ChatHPC library, can create new capabilities with Meta’s 7-billion parameter Code Llama base model to produce high-quality software with a level of trustworthiness of up to 90% higher than the 1.8-trillion parameter OpenAI ChatGPT-4o model for critical programming tasks in the HPC software stack.

Young, Aaron [ORNL] (ORCID:0000000254484667)↗
DOE OSTI2021

HYPPO v1.0.0

HYPPO is a hyperparameter optimization (HPO) software that uses surrogate modeling and uncertainty quantification to provide reliable neural network architectures. Similar state-of-the-art HPO technologies do not use such features and mostly perform random searches to explore the parameter space, which can easily become computationally expensive. Using HYPPO, the user is able to reduce by an order of magnitude the number of evaluations necessary to identify the most optimal region in the hyperparameter space. Finally, using asynchronous nested parallelism, we are able to decrease by two orders of magnitude the amount of time required to complete the HPO process.

Dumont, Vincent↗
DOE OSTI2022

Building an Integrated Ecosystem of Computational and Observational Facilities to Accelerate Scientific Discovery

Future scientific discoveries will rely on flexible ecosystems that incorporate modern scientific instruments, high performance computing resources, parallel distributed data storage, and performant networks across multiple, independent facilities. In addition to connecting physical resources, such an ecosystem presents many challenges in logistics and accessibility, especially in orchestrating computations and experiments that span across leadership computing systems and experimental instruments. Past efforts have typically been application-specific or limited to interfaces for computing resources. This paper proposes a general framework for integrating computation resources and instrument operations, addressing challenges in code development/execution, data staging and collection, software stack, control mechanisms, resource authorization and governance, and hardware integration. We also describe a demonstration use case wherein a Bayesian optimization algorithm running on an edge computing resource guides a scanning probe microscope to autonomously and intelligently characterize a material sample. This science edge ecosystem framework will provide a blueprint for federating multi-institutional, disparate resources and orchestrating scientific workflows across them to enable next-generation discoveries.

Somnath, Suhas↗
DOE OSTI2020

Threadsafe Dynamic Neighbor Lists for Monte Carlo Ray Tracing

Monte Carlo (MC) transport codes offer high-fidelity modeling of particle transport physics, but their high computational cost makes them impractical for many applications. For some applications such as multiphysics and depletion that use finely discretized geometries, a large portion of this computational cost is attributable to ray tracing. Neighbor lists are a well-known method for accelerating ray-tracing calculations in a MC code, but despite their prevalence, little work has been published on the details of their implementation. The fine details can have a significant impact on performance, particularly when using shared-memory parallelism. This paper addresses these details of implementation with a discussion of different neighbor list schemes and their impact on software runtime. Performance tests were run by using OpenMC on a pin-cell problem discretized with up to 200 axial regions. The results demonstrate that switching from surface-based to cell-based neighbor lists leads to a 10 faster calculation rate for the most fine discretization. Finally, using a threadsafe shared-memory data structure results in a 20% faster calculation rate versus simple threadprivate neighbor lists. Results here show that a data structure that is contiguous in memory improves performance by only 1% to 2% over noncontiguous linked lists.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗
DOE OSTI2020

Understanding the use of message passing interface in exascale proxy applications

The Exascale Computing Project (ECP) focuses on the development of future exascale-capable applications. Most ECP applications use the message passing interface (MPI) as their parallel programming model with mini-apps serving as proxies. This paper explores the explicit usage of MPI in such ECP proxy applications. We empirically analyze 14 proxy applications from the ECP Proxy Apps Suite. We use the MPI profiling interface (PMPI) to collect MPI usage patterns in ECP proxy apps. Our analysis shows that a small subset of features from MPI is commonly used in the proxies of exascale-capable applications, even when they reference third-party libraries. Overall, this study is intended to provide a better understanding of the use of MPI in current exascale applications. The findings can help focus software investments made for exascale systems in the MPI middleware including optimization, fault-tolerance, tuning, and hardware-offload.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Scalable Data-Intensive Geocomputation: A Design for Real-Time Continental Flood Inundation Mapping

The convergence of data-intensive and extreme-scale computing enables an integrated software and data ecosystem for scientific discovery. Developments in this realm will fuel transformative research in data-driven interdisciplinary domains. Geocomputation provides computing paradigms in Geographic Information Systems (GIS) for interactive computing of geographic data, processes, models, and maps. Because GIS is data-driven, the computational scalability of a geocomputation workflow is directly related to the scale of the GIS data layers, their resolution and extent, as well as the velocity of the geo-located data streams to be processed. Unique in high user interactivity and low end-to-end latency requirements, geocomputation applications will dramatically benefit from the convergence of high-end data analytics (HDA) and high-performance computing (HPC). The application level challenge, however, is to identify and eliminate computational bottlenecks that arise along a geocomputation workflow. Indeed, poor scalability at any of the workflow components is detrimental to the entire end-to-end pipeline. Here, we study a large geocomputation use case in flood inundation mapping that handles multiple national-scale geospatial datasets and targets low end-to-end latency. We discuss benefits and challenges for harnessing both HDA and HPC for data-intensive geospatial data processing and intensive numerical modeling of geographic processes. We propose an HDA+HPC geocomputation architecture design that couples HDA (e.g., Spark)-based spatial data handling and HPC-based parallel data modeling. Key techniques for coupling HDA and HPC to bridge the two different software stacks are reviewed and discussed.

Liu, Yan↗
DOE OSTI2023

An accurate and efficient fragmentation approach via the generalized many-body expansion for density matrices

With relevant chemical space growing larger and larger by the day, the ability to extend computational tractability over that larger space is of paramount importance in virtually all fields of science. The solution we aim to provide here for this issue is in the form of the generalized many-body expansion for building density matrices (GMBE-DM) based on the set-theoretical derivation with overlapping fragments, through which the energy can be obtained by a single Fock build. In combination with the purification scheme and the truncation at the one-body level, the DM-based GMBE(1)-DM-P approach shows both highly accurate absolute and relative energies for medium-to-large size water clusters with about an order of magnitude better than the corresponding energy-based GMBE(1) scheme. Simultaneously, GMBE(1)-DM-P is about an order of magnitude faster than the previously proposed MBE-DM scheme [F. Ballesteros and K. U. Lao, J. Chem. Theory Comput. 18, 179 (2022)] and is even faster than a supersystem calculation without significant parallelization to rescue the fragmentation method. For even more challenging systems including ion–water and ion–pair clusters, GMBE(1)-DM-P also performs about 3 and 30 times better than the energy-based GMBE(1) approach, respectively. In addition, this work provides the first overlapping fragmentation algorithm with a robust and effective binning scheme implemented internally in a popular quantum chemistry software package. Thus, GMBE(1)-DM-P opens a new door to accurately and efficiently describe noncovalent clusters using quantum mechanics.

Chemistry↗
DOE OSTI2021

Transient cycling of nitrogen, organic carbon and oxygen within the free-flowing Columbia River corridor: Linking exposure time dependent biogeochemical reactions to river stage fluctuations (Final project report)

The objective of this project was to develop next-generation techniques for representing the transformations of complex reaction networks like those impacting transient river-corridors, then apply them to develop upscaling tools. The project contributed significantly to the development of Lagrangian “mass transfer particle tracking” (MTPT) tools that are the first numerical framework that explicitly separates mixing and spreading processes; a crucial distinction for accurate representations of reaction rates across scales. The MTPT approach uses an operator splitting scheme where physical transport processes are simulated using classical random walk methods and each particle is treated as a “container” that carries with it any number of chemical species. Mixing between particles is simulated using a colocation-probability based mass transfer kernel and reactions are evaluated on each particle after mixing. The method is stable, accurate, and also allows the explicit modeling of residence time distributions of the individual containers of mass. The MTPT scheme allowed us to explore several research questions related to reactive transport in river corridors and complex reaction networks. We found that it is possible to quantify the exposure time distributions (ETDs) of reactants and that this does suggest some pathways to upscaling. However, comparable approximations can be obtained using simplified (reduced dimensionality) MTPT simulations in less time with similar uncertainty, so development of extensive ETD-based methods was not productive. The major findings are that i) the MTPT schemes are robust and highly accurate across a range of arbitrarily complex reactions, ii) the methods can be efficiently parallelized and the parallel performance characteristics of MTPT are predictable, and iii) the MTPT tools allow simultaneous tracking of residence time. The development of these research tools into efficient software packages continues. The methods are already available to the community because open-source, working examples have been included with all publications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

An Algorithmic and Software Pipeline for Very Large Scale Scientific Data Compression with Error Guarantees

Efficient data compression is becoming increasingly critical for storing scientific data because many scientific applications produce vast amounts of data. This paper presents an end-to-end algorithmic and software pipeline for data compression that guarantees both error bounds on primary data (PD) and derived data, known as Quantities of Interest (QoI).We demonstrate the effectiveness of the pipeline by compressing fusion data generated by a large-scale fusion code, XGC, which produces tens of petabytes of data in a single day. We demonstrate that the compression is conducted by setting aside computational resources known as staging nodes, and does not impact the simulation performance. For efficient parallel I/O, the pipeline uses ADIOS2, which many codes such as XGC already use for their parallel I/O. We show that our approach can compress the data by two orders of magnitude while guaranteeing high accuracy on both the PD and the QoIs. Further, the amount of resources required by compression is a few percent of the resources required by simulation while ensuring that the compression time for each stage is less than the corresponding simulation time.This pipeline consists of three main steps. The first step decomposes the data using domain decomposition into small subdomains. Each subdomain is then compressed independently to achieve a high level of parallelism. The second step uses existing techniques that guarantee error bounds on the primary data for each subdomain. The third step uses a post-processing optimization technique based on Lagrange multipliers to reduce the QoI errors for data corresponding to each subdomain. The Lagrange multipliers generated can be further quantized or truncated to increase the compression level. All of the above characteristics of our approach make it highly practical to apply on-the-fly compression while guaranteeing errors on QoIs that are critical to the scientists.

Banerjee, Tania↗
DOE OSTI2022

An in-situ conductometric apparatus for physicochemical characterization of solutions and in-line monitoring of separation processes at elevated temperatures and pressures

Specific conductance and frequency-dependent resistance (impedance) data are widely utilized for understanding the physicochemical characteristics of aqueous and non-aqueous fluids and for evaluating the performance of chemical processes. However, the implementation of such an in-situ probe in high-temperature and high-pressure environments is not trivial. This work provides a description of both the hardware and software associated with implementing a parallel-type in-situ electrochemical sensor. The sensor can be used for in-line monitoring of thermal desalination processes and for impedance measurements in fluids at high temperature and pressure. Further, a comparison between the experimental measurements on the specific conductance in aqueous sodium chloride solutions and the conductance model demonstrate that the methodology yields reasonable agreement with both the model and literature data. A combination of hardware components, a software-based correction for experimental artifacts, and computational fluid dynamics (CFD) calculations used in this work provide a sound basis for implementing such in-situ electrochemical sensors to measure frequency-dependent resistance spectra.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Optimizing the hit finding algorithm for liquid argon TPC neutrino detectors using parallel architectures

Neutrinos are particles that interact rarely, so identifying them requires large detectors which produce lots of data. Processing this data with the computing power available is becoming even more difficult as the detectors increase in size to reach their physics goals. Liquid argon time projection chamber (LArTPC) neutrino experiments are expected to grow in the next decade to have 100 times more wires than in currently operating experiments, and modernization of LArTPC reconstruction code, including parallelization both at data- and instruction-level, will help to mitigate this challenge. The LArTPC hit finding algorithm is used across multiple experiments through a common software framework. In this paper we discuss a parallel implementation of this algorithm. Using a standalone setup we find speedup factors of two times from vectorization and 30–100 times from multi-threading on Intel architectures. The new version has been incorporated back into the framework so that it can be used by experiments. On a serial execution, the integrated version is about 10 times faster than the previous one and, once parallelization is enabled, more speedups comparable to the standalone program are achieved.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Reduced Models for Chemical Kinetics derived from Parallel Ensemble Simulations of Stirred Reactors

Detailed chemical kinetic models are essential to improve the predictive capability of computational fluid dynamics (CFD) simulations and to impact the development of next generation combustion technologies. However, such detailed kinetic models are beyond the reach of CFD codes due to the computational cost of transporting additional chemical degrees of freedom. Reduced order models for combustion kinetics exploit the presence of lower dimensional manifolds in the state space and require far fewer number of scalars to be transported as part of CFD simulations. In this paper, we describe the software infrastructure developed to allow large ensembles of stirred reactor configurations with detailed chemical kinetics. The canonical reactor simulations are incorporated in a workflow with sparse grid and data reduction methodologies aimed at development of reduced order models. We present results from analyzing the results of the stirred reactor calculations for syngas mixture using principal component analysis (PCA) and neural network methodologies. The results show that nonlinear autoencoder model can surpass PCA in representing the original dataset with far fewer degrees of freedom.

Zhang, Pei↗
DOE OSTI2024

Computational Analysis of Hydraulic Efficiency of Michigan DOT Cover C

Drainage structures are used to capture stormwater runoff in streets and highways in urban environments. These drainage structures, which typically consist of catch basins with grates, inlets, or combination grates/inlets, collect stormwater runoff and discharge through buried conveyance systems. They are strategically placed for public safety in curb and gutter systems to provide efficient drainage of water from roadways and thus reduce the risk of hydroplaning. The performance of drainage structures is measured in terms of hydraulic efficiency, which is defined as the percentage of flow captured by the basin as compared to the total flow drainage to the structure. Understanding of the performance of these drainage structures helps designers properly space inlets to promote an economic design that ensures the safety of the traveling public. The current design methodology used to determine drainage structure follows guidelines established in the current Michigan Department of Transportation (MDOT) Drainage Manual (2006). The guidelines in the MDOT Drainage Manual were modeled after the Federal Highway Administration’s (FHWA) Hydraulic Engineering Circular 22 “Urban Drainage Design” (HEC-22). HEC-22 includes empirically derived equations to calculate the interception capacity of drainage structures for several commonly used grate configurations, such as the parallel bar, curved vane and tilt bar grates, which are based on a research study performed by Burgi et al. in the 1970s. MDOT uses several drainage structures to capture runoff that are detailed as Standard Plans. Many of these drainage structures utilize sinusoidal type grates that are not described in HEC-22. Physical modeling of these structures has been limited, posing the need to have them analyzed to verify their capture efficiency. Current MDOT practice is to assume a similar sized reticuline grate, as described in HEC-22, for capture efficiencies. Until recently, evaluating the hydraulic performance of drainage structures was limited to physical modeling in a hydraulics laboratory. With advances in engineering software and computing power, computational fluid dynamics (CFD) modeling has become a more cost-effective alternative. The Federal Highway Administration (FHWA) provides states the option to evaluate their drainage structures using CFD through the Transportation Pooled Fund Program. This study, “Computational Analysis of Hydraulic Efficiency of Michigan DOT Cover C,” was carried out using the pooled fund. MDOT’s Cover C was chosen as the first test candidate, given its similar sinusoidal pattern to other MDOT grates, but it is typically used for high-volume, higher speed applications. A similar version, Cover CX, is used on interstate highways but does not have traverse bars for bicycle safety. Additional grates may be considered for evaluation in the future.

42 ENGINEERING↗
DOE OSTI2021

Axom

Axom is an open-source library of "building block" software components that provide core infrastructure capabilities for HPC applications. Such capabilities include: input file parsing and verification for various file formats, format specification for material shape input to multi-material simulations and tools for placing those shapes on meshes, parallel distributed coordination of diagnostic messages, geometric primitives, spatial queries and spatial search acceleration data structures, in memory key-value data store for managing simulation data and parallel file I/O. Axom components are used widely across LLNL Advanced Simulation and Computing program applications. It is also part of the LLNL institutionally-supported RADIUSS project, which promotes and funds its adoption by projects across LLNL.

Zagaris, George↗
DOE OSTI2025

VA EDH Advanced Software Pipeline Framework Report: Enhancing Automation and Scalability

The VA Environmental Determinants of Health (EDH) Advanced Software Pipeline Framework is designed to enhance the efficiency, scalability, and security of geospatial data processing workflows. This framework integrates modern data orchestration and containerization technologies, including Prefect for workflow automation, Docker for containerization, and PostgreSQL/PostGIS for geospatial data storage and analysis. It ensures standardized, reproducible, and automated data processing, supporting VA objectives related to substance use risk assessment and recovery research. The pipeline addresses key scalability and performance challenges through horizontal and vertical scaling, high-performance computing (HPC) integration, parallel processing, task caching, and dynamic resource allocation. These optimizations improve throughput and reduce latency, allowing the system to efficiently manage large and complex datasets. Additionally, security and compliance measures—such as data encryption (SSL), Role-Based Access Control (RBAC), and adherence to GDPR and HIPAA standards—safeguard sensitive information throughout data transmission and storage. A key implementation of this framework includes the automation of shelter list geolocation workflows, ensuring that up-to-date data is readily available for VA decision-making. Lessons learned from this project include the transition from in-memory processing to incremental storage writes, improving resource management and reliability. Future enhancements aim to expand automation, integrate AI-driven anomaly detection, and incorporate high-performance computing resources. This framework provides a scalable, secure, and adaptable solution for managing geospatial datasets, reinforcing the VA’s ability to support clinical and strategic initiatives through data-driven decision-making.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Evaluating Portable Parallelization Strategies for Heterogeneous Architectures in High Energy Physics

High-energy physics (HEP) experiments have developed millions of lines of code over decades that are optimized to run on traditional x86 CPU systems. However, we are seeing a rapidly increasing fraction of floating point computing power in leadership-class computing facilities and traditional data centers coming from new accelerator architectures, such as GPUs. HEP experiments are now faced with the untenable prospect of rewriting millions of lines of x86 CPU code, for the increasingly dominant architectures found in these computational accelerators. This task is made more challenging by the architecture-specific languages and APIs promoted by manufacturers such as NVIDIA, Intel and AMD. Producing multiple, architecture-specific implementations is not a viable scenario, given the available person power and code maintenance issues. The Portable Parallelization Strategies team of the HEP Center for Computational Excellence is investigating the use of Kokkos, SYCL, OpenMP, std::execution::parallel and alpaka as potential portability solutions that promise to execute on multiple architectures from the same source code, using representative use cases from major HEP experiments, including the DUNE experiment of the Long Baseline Neutrino Facility, and the ATLAS and CMS experiments of the Large Hadron Collider. This cross-cutting evaluation of portability solutions using real applications will help inform and guide the HEP community when choosing their software and hardware suites for the next generation of experimental frameworks. We present the outcomes of our studies, including performance metrics, porting challenges, API evaluations, and build system integration.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗