SEARCH · Search NASA

Results for “MPI”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2024

TeMPI Shim

This is the manual for the TeMPI Shim library, whose goal is to facilitate scientific apps being loosely coupled through MPI. TeMPI Shim can be used to create and manage Message Passing Interface (MPI, see [MPI]) Communicators for Single Program, Multiple Data (SPMD, see [SPMD]) and Multiple Programs, Multiple Data (MPMD, see [MPMD]) workflows. MPI provides the MPI_APPNUM command number to each of the processes within its context. This number, starting at 0, is the application number being executed. In a case where 3 applications are being run in MPMD mode, there would be MPI_APPNUM values of 0, 1, and 2. TeMPI Shim creates intra- and inter-communicators between each pair of MPI_APPNUM values. In the aforementioned case, application 0 would have intra- and inter-communicators to speak to itself, application 1, and application 2. This is replicated for each of the applications. Additionally, TeMPI Shim creates intra- and inter-communicators for the first MPI rank of each application to directly communicate only with each other. Finally, TeMPI Shim creates its own copy of the default world communicator, i.e., MPI_COMM_WORLD. In the case where there is only a single application, it will have the communicators to only speak with itself. TeMPI Shim is useful in this case since it is considered good practice for MPI applications to copy the default world communicator and reference this copy (see [Duplicate World]_) anyways. Ultimately, it provides value independent of the number of applications present.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2024

Multihead Attention U‐Net for Magnetic Particle Imaging–Computed Tomography Image Segmentation

Magnetic particle imaging (MPI) is an emerging noninvasive molecular imaging modality with high sensitivity and specificity, exceptional linear quantitative ability, and potential for successful applications in clinical settings. Computed tomography (CT) is typically combined with the MPI image to obtain more anatomical information. Herein, a deep learning‐based approach for MPI‐CT image segmentation is presented. The dataset utilized in training the proposed deep learning model is obtained from a transgenic mouse model of breast cancer following administration of indocyanine green (ICG)‐conjugated superparamagnetic iron oxide nanoworms (NWs‐ICG) as the tracer. The NWs‐ICG particles progressively accumulate in tumors due to the enhanced permeability and retention (EPR) effect. The proposed deep learning model exploits the advantages of the multihead attention mechanism and the U‐Net model to perform segmentation on the MPI‐CT images, showing superb results. In addition, the model is characterized with a different number of attention heads to explore the optimal number for our custom MPI‐CT dataset.

Juhong, Aniwat↗

DOE OSTI2024

Distributed Order Recording Techniques for Efficient Record-and-Replay of Multi-threaded Programs

After all these years and all these other shared memory programming frameworks, OpenMP is still the most popular one. However, its greater levels of non-deterministic execution makes debugging and testing more challenging. The ability to record and deterministically replay the program execution is key to address this challenge. However, scalably replaying OpenMP programs is still an unresolved problem. In this paper, we propose two novel techniques that use Distributed Clock (DC) and Distributed Epoch (DE) recording schemes to eliminate excessive thread synchronization for OpenMP record and replay. Our evaluation on representative HPC applications with ReOMP, which we used to realize DC and DE recording, shows that our approach is 2-5x more efficient than traditional approaches that synchronize on every shared-memory access. Furthermore, we demonstrate that our approach can be easily combined with MPI-level replay tools to replay non-trivial MPI+OpenMP applications. We achieve this by integrating ReOMP into ReMPI, an existing scalable MPI record-and-replay tool, with only a small MPI-scale-independent runtime overhead.

Fu, Xiang↗

DOE OSTI2025

Parallel Programming in MCNP6

Monte Carlo N-Particle (MCNP)1 is a general-purpose Monte Carlo particle transport code developed by Los Alamos National Laboratory (LANL). To efficiently handle long simulations, MCNP version 6 (MCNP6) supports parallel execution using two primary programming models: • Shared-memory task-based threading using OpenMP (Open Multi-Processing), and • Distributed-memory calculations using MPI (Message Passing Interface). The OpenMP and MPI programming models enable MCNP6 to scale from desktop systems to high-performance computing (HPC) clusters, allowing users to run MCNP in one of three parallel modes: • OpenMP-only, • MPI-only, and • Hybrid (MPI + OpenMP). The choice of parallelization mode depends on the underlying computer architecture and the characteristics of the simulation problem.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Multiplicity dependent 𝐽/𝜓 and 𝜓⁡(2⁢𝑆) production at forward and backward rapidity in 𝑝 + 𝑝 collisions at $\sqrt{𝑠}$ = 200 GeV

Recent measurements of 𝐽/𝜓 production as a function of event charged-particle multiplicity at the collision energies of both the Large Hadron Collider (LHC) and the Relativistic Heavy Ion Collider (RHIC) show enhanced 𝐽/𝜓 production yields with increasing multiplicity. One potential explanation for this type of dependence is multiparton interactions (MPI). We present the first study of potential autocorrelations at RHIC energies and forward and backward rapidity of self-normalized 𝐽/𝜓 yields and 𝜓⁡(2⁢𝑆) to 𝐽/𝜓 ratio, as a function of self-normalized multiplicity in 𝑝 + 𝑝 collisions. In addition, detailed pythia studies tuned to RHIC energies were performed to investigate the MPI impacts. We find that the PHENIX data at RHIC are consistent with recent LHC measurements and can only be described by pythia calculations that include MPI effects. The forward and backward 𝜓⁡(2⁢𝑆) to 𝐽/𝜓 ratio is found to be less dependent on the charged-particle multiplicity.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2026

Phloem

Phloem is a Message Passing Interface (MPI) micro-benchmarking suite featuring sub-communicator collectives, methods for finding slow links on MPI interconnects, and point-to-point MPI benchmarks, including a messaging rate benchmark. All of the benchmarks except for ones related exclusively to finding slow links are GPU-aware via the Umpire resource management library.

Moody, AdamT [Lawrence Livermore National Laborato↗

DOE OSTI2024

Evolution of the SLATE linear algebra library

SLATE (Software for Linear Algebra Targeting Exascale) is a distributed, dense linear algebra library targeting both CPU-only and GPU-accelerated systems, developed over the course of the Exascale Computing Project (ECP). While it began with several documents setting out its initial design, significant design changes occurred throughout its development. In some cases, these were anticipated: an early version used a simple consistency flag that was later replaced with a full-featured consistency protocol. In other cases, performance limitations and software and hardware changes prompted a redesign. Sequential communication tasks were parallelized; host-to-host MPI calls were replaced with GPU device-to-device MPI calls; more advanced algorithms such as Communication Avoiding LU and the Random Butterfly Transform (RBT) were introduced. Early choices that turned out to be cumbersome, error prone, or inflexible have been replaced with simpler, more intuitive, or more flexible designs. Applications have been a driving force, prompting a lighter weight queue class, nonuniform tile sizes, and more flexible MPI process grids. Of paramount importance has been building a portable library that works across several different GPU architectures – AMD, Intel, and NVIDIA – while keeping a clean and maintainable codebase. Here we explore the evolving design choices and their effects, both in terms of performance and software sustainability.

Gates, Mark↗

DOE OSTI2025

Preparing MPICH for exascale

The advent of exascale supercomputers heralds a new era of scientific discovery, yet it introduces significant architectural challenges that must be overcome for MPI applications to fully exploit its potential. Among these challenges is the adoption of heterogeneous architectures, particularly the integration of GPUs to accelerate computation. Additionally, the complexity of multithreaded programming models has also become a critical factor in achieving performance at scale. The efficient utilization of hardware acceleration for communication, provided by modern NICs, is also essential for achieving low latency and high throughput communication in such complex systems. In response to these challenges, the MPICH library, a high-performance and widely used Message Passing Interface (MPI) implementation, has undergone significant enhancements. Here, this paper presents four major contributions that prepare MPICH for the exascale transition. First, we describe a lightweight communication stack that leverages the advanced features of modern NICs to maximize hardware acceleration. Second, our work showcases a highly scalable multithreaded communication model that addresses the complexities of concurrent environments. Third, we introduce GPU-aware communication capabilities that optimize data movement in GPU-integrated systems. Finally, we present a new datatype engine aimed at accelerating the use of MPI derived datatypes on GPUs. These improvements in the MPICH library not only address the immediate needs of exascale computing architectures but also set a foundation for exploiting future innovations in high-performance computing. By embracing these new designs and approaches, MPICH-derived libraries from HPE Cray and Intel were able to achieve real exascale performance on OLCF Frontier and ALCF Aurora respectively.

Guo, Yanfei [Argonne National Laboratory (ANL), Ar↗

DOE OSTI2025

Historical and Future Windstorms in the Northeastern United States

Large-scale windstorms represent an important atmospheric hazard in the Northeastern US (NE) and are associated with substantial socioeconomic losses. Regional simulations performed with the Weather Research and Forecasting (WRF) model using lateral boundary conditions from three Earth System Models (ESMs: Geophysical Fluid Dynamics Laboratory (GFDL), Hadley Centre Global Environment Model (HadGEM) and Max Planck Institute (MPI)) are used to quantify possible future changes in windstorm characteristics and/or changes in the parent cyclone types responsible for windstorms. WRF nested within MPI ESM best represents important aspects of historical windstorms and the cyclone types responsible for generating windstorms compared with a reference simulation performed with the ERA-Interim reanalysis for the historical climate. The spatial scale and frequency of the largest windstorms in each simulation defined using the greatest extent of exceedance of local 99.9th percentile wind speeds (U > U999) plus 50-year return period wind speeds (U50,RP) do not exhibit secular trends. Projections of extreme wind speeds and windstorm intensity/frequency/geolocation and dominant parent cyclone type associated with windstorms vary markedly across the simulations. Only the MPI nested simulations indicate statistically significant differences in windstorm spatial scale, frequency and intensity over the NE in the future and historical periods. This model chain, which also exhibits the highest fidelity in the historical climate, yields evidence of future increases in 99.9th percentile 10 m height wind speeds, the frequency of simultaneous U > U999 over a substantial fraction (5–25%) of the NE and the frequency of maximum wind speeds above 22.5 ms−1. These geophysical changes, coupled with a projected doubling of population, leads to a projected tripling of a socioeconomic loss index, and hence risk to human systems, from future windstorms.

Pryor, Sara C. (ORCID:0000000348473440)↗

DOE OSTI2026

Exploration of Nirmatrelvir Derivatives as Optimized SARS‐CoV‐2 Antivirals

Nirmatrelvir (NMV) is a SARS‐CoV‐2 antiviral component of the approved COVID‐19 therapeutic Paxlovid. It is a reversible covalent inhibitor of SARS‐CoV‐2 main protease (M Pro ) that is effluxed from human cells by P‐glycoprotein (P‐gp). To identify NMV analogs with improved potency and reduced P‐gp efflux, a structure–activity relationship campaign was conducted. Warheads alternative to nitrile for engaging the active site cysteine were tested showing aldehyde and dichloroacetamide with better enzyme inhibition potency. Crystal structure of MPI‐136−M Pro shows its aldehyde warhead forming a thiohemiacetal with active Cys145 of M Pro . Several S4 binders were explored revealing that an O‐to‐S shift at the N ‐terminal amide leads to better enzyme inhibition. By exploring different combinations of S2, S3, and S4 binders, two inhibitors with better enzyme inhibition potency than NMV were found. Crystal structure of MPI‐148, with ( S )‐2‐azaspiro[4,5]decane‐3‐carboxylate as an alternative S2 binder, shows extensive hydrogen‐bond networks for locking the inhibitor in active site, explaining high affinity of NMV analogs. Further characterization of cellular M Pro engagement and antiviral potency against SARS‐CoV‐2 revealed four inhibitors with greater potency than NMV in P‐gp‐expressing cells. Studies with the P‐gp inhibitor CP‐100356 showed that these compounds were less sensitive to P‐gp inhibition than NMV, consistent with reduced P‐gp‐mediated efflux.

Alugubelli, Yugendar R. [Texas A&M Drug Discovery ↗

DOE OSTI2025

Lessons Learned and Scalability Achieved When Porting Uintah to DOE Exascale Systems

A key challenge faced when preparing codes for Department of Energy (DOE) exascale systems was designing scalable applications for systems featuring hardware and software not yet available at leadership-class scale. With such systems now available, it is important to evaluate scalability of the resulting software solutions on these target systems. One such code designed with the exascale DOE Aurora and DOE Frontier systems in mind is the Uintah Computational Framework, an open-source asynchronous many-task (AMT) runtime system. To prepare for exascale, Uintah adopted a portable MPI+X hybrid parallelism approach using the Kokkos performance portability library (i.e., MPI+Kokkos). This paper complements recent work with additional details and an evaluation of the resulting approach on Aurora and Frontier. Results are shown for a challenging benchmark demonstrating interoperability of 3 portable codes essential to Uintah-related combustion research. These results demonstrate single-source portability across Aurora and Frontier with scaling characteristics shown to 3,072 Aurora nodes and 9,216 Frontier nodes. In addition to showing results run to new scales on new systems, this paper also discusses lessons learned through efforts preparing Uintah for exascale systems.

Holmen, John [ORNL] (ORCID:0000000259342641)↗

DOE OSTI2025

General field evaluation in high-order meshes on GPUs

Robust and scalable function evaluation at any arbitrary point in the finite/spectral element mesh is required for querying the partial differential equation solution at points of interest, comparison of solution between different meshes, and Lagrangian particle tracking. This is a challenging problem, particularly for high-order unstructured meshes partitioned in parallel with MPI, as it requires identifying the element that overlaps a given point and computing the corresponding reference space coordinates. Here, we present a robust and efficient technique for general field evaluation in large-scale high-order meshes with quadrilaterals and hexahedra. In the proposed method, a combination of globally partitioned and processor-local maps are used to first determine a list of candidate MPI ranks, and then locally candidate elements that could contain a given point. Next, element-wise bounding boxes further reduce the list of candidate elements. Finally, Newton’s method with trust region is used to determine the overlapping element and corresponding reference space coordinates. Since GPU-based architectures have become popular for accelerating computational analyses using meshes with tensor-product elements, specialized kernels have been developed to utilize the proposed methodology on GPUs. The method is also extended to enable general field evaluation on surface meshes. The paper concludes by demonstrating the use of the proposed method in various applications ranging from mesh-to-mesh transfer during r-adaptivity to Lagrangian particle tracking.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Changing windstorm characteristics over the US Northeast in a single model large ensemble

Abstract Extreme windstorms pose a significant hazard to infrastructure and public safety, particularly in the highly populated US Northeast (NE). However, the influence climate change and changing land use will have on these events remains unclear. A large ensemble generated using the Max-Planck Institute (MPI) Earth system model is used to generate projections of NE windstorms under different shared socioeconomic pathways (SSPs) and to attribute changes to projected land use land cover (LULC) change, externally forced changes and internal climate variability. To reduce the influence of coarse grid cell resolution and uncertainties in surface roughness lengths, windstorms are identified using simultaneous widespread exceedance of local 99th percentile 10 m wind speeds (U 99 ). Projected declines in forest cover in the NE and the resulting reductions in surface roughness length under SSP3-7.0 lead to projections of large increases in U 99 and derived windstorm intensity and scale. However, these projected changes in regional LULC under SSP3-7.0 are unprecedented in a historical context and may not be realistic. After corrections are applied to remove the influence of LULC on wind speeds, regionally averaged U 99 exhibit declines for most of the single model initial-condition large ensemble (SMILE) members which are broadly proportional to the radiative forcing and global air temperature increase in the SSPs, with a median value of −0.15 ms −1 °C −1 . While weak cyclones are projected to decline in frequency in the NE, intense cyclones and the resulting windstorms and indices of socioeconomic loss do not. Where present, significant trends in these loss indices are positive, and some MPI SMILE members generate future windstorms that are unprecedented in the historical period.

Coburn, Jacob (ORCID:0000000309538117)↗

DOE OSTI2025

Enabling Scientific Applications with Performance-Portability and High-Productivity for Multi-GPU Programming with JACC.Multi

This work bridges the gap between multi-GPU computing and high-productivity, performance-portable programming solutions. Our goal is to enhance scientific applications with a productive and portable solution—program once, deploy everywhere—for multi-GPU programming with no cost to programmability. To accomplish this, we implemented JACC.Multi, which is part of the Julia for ACCelerators (JACC) performance-portable framework. JACC. Multi is the only high-level, portable metaprogramming solution that targets multi-GPU environments and is integrated in a readily accessible programming language (e.g., Julia language). With transparent GPU-to-GPU communication, JACC. Multi is optimized for scientific application workloads and is portable for NVIDIA and AMD accelerators. For the evaluation, we use two modern multi-GPU systems: Hudson, which features two NVIDIA H100 Hopper GPUs per node, and Frontier, which features four AMD MI250X GPUs per node, each with two Graphics Compute Dies (GCDs) for a total of eight GCDs per node. Additionally, as part of the evaluation, we use JACC (one GPU), MPI+JACC, and JACC. Multi codes that implement well-known and widely used scientific algorithms/kernels such as the conjugate gradient algorithm and an explicit forward Euler solver that requires GPU-to-GPU communication. Overall, JACC. Multi codes achieve better performance than MPI+JACC codes and significant speedups over JACC (one GPU), with up to 1.9× on Hudson and 6× on Frontier.

Valero Lara, Pedro [ORNL] (ORCID:0000000214794310)↗

DOE OSTI2025

Large-Message All-to-All Communication at Frontier Scale

Near the full scale of exascale supercomputers, latency can dominate the cost of all-to-all communication even for very large message sizes. We describe GPU-aware all-to-all implementations designed to reduce latency for large message sizes at extreme scales, and we present their performance using 65536 tasks (8192 nodes) on the Frontier supercomputer at the Oak Ridge Leadership Computing Facility. Two implementations perform best for different ranges of message size, and all outperform the vendor-provided MPI_Alltoall. Our results show promising options for improving implementations of MPI_Alltoall_init.

White, Trey [ORNL] (ORCID:000900052186075X)↗

DOE OSTI2025

Scalable multilevel Monte Carlo methods exploiting parallel redistribution on coarse levels

Here, we study an element agglomeration coarsening strategy that requires data redistribution at coarse levels when the number of coarse elements becomes smaller than the number of MPI processes used on the finest level. The overall procedure generates coarse elements (general unstructured unions of fine grid elements) within the framework of element-based algebraic multigrid methods (or AMGe) studied previously. The AMGe-generated coarse spaces have the ability to exhibit approximation properties of the same order as the fine-level spaces since by construction they contain the piecewise polynomials of the same order as on the fine level. These approximation properties are key for the successful use of AMGe in multilevel solvers for nonlinear partial differential equations as well as for multilevel Monte Carlo (MLMC) simulations. The ability to coarsen without being constrained by the number of MPI processes, as described in the present paper, allows to improve the scalability of these solvers as well as the overall MLMC method. The paper illustrates this latter fact with detailed scalability study of MLMC simulations applied to model Darcy equations with a stochastic log-normal permeability field.

AMGe↗

DOE OSTI2024

Venado acceptance: results and tips [Slides]

Nvidia compiler support is not available through cray-mpich/compiler wrapper interface. Adjust CMAKE files to use the correct COMPILER_ID in conditionals and explicit variables to package flags. Use pinned host memory in cray-libsci_acc and cublasXt calls. Set a large blockDim for cublasXt calls. Try MPS and/or explicit numactl binding if performance is lackluster. Use CRAY_MALLOPT_OFF=1 if unexpected OOM errors appear using cce. Use MPICH_SMP_SINGLE_COPY_MODE=CMA for xpmem issues. Use MPICH_OPT_THREAD_SYNC=0 for MPI_THREAD issues. Poor CUDA-aware MPI performance remains an issue.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Atmospheric H 2 observations from the NOAA Cooperative Global Air Sampling Network

Abstract. The NOAA Global Monitoring Laboratory (GML) measures atmospheric hydrogen (H2) in grab samples collected weekly as flask pairs at over 50 sites in the Cooperative Global Air Sampling Network. Measurements representative of background air sampling show higher H2 in recent years at all latitudes. The marine boundary layer (MBL) global mean H2 was 552.8 ppb in 2021, 20.2 ± 0.2 ppb higher compared to 2010. A 10 ppb or more increase over the 2010–2021 average annual cycle was detected in 2016 for MBL zonal means in the tropics and in the Southern Hemisphere. Carbon monoxide measurements in the same-air samples suggest large biomass burning events in different regions likely contributed to the observed interannual variability at different latitudes. The NOAA H2 measurements from 2009 to 2021 are now based on the World Meteorological Organization Global Atmospheric Watch (WMO GAW) H2 mole fraction calibration scale, developed and maintained by the Max Planck Institute for Biogeochemistry (MPI-BGC), Jena, Germany. GML maintains eight H2 primary calibration standards to propagate the WMO scale. These are gravimetric hydrogen-in-air mixtures in electropolished stainless steel cylinders (Essex Industries, St. Louis, MO), which are stable for H2. These mixtures were calibrated at the MPI-BGC, the WMO Central Calibration Laboratory (CCL) for H2, in late 2020 and span the range 250–700 ppb. We have used the CCL assignments to propagate the WMO H2 calibration scale to NOAA air measurements performed using gas chromatography and helium pulse discharge detector instruments since 2009. To propagate the scale, NOAA uses a hierarchy of secondary and tertiary standards, which consist of high-pressure whole-air mixtures in aluminum cylinders, calibrated against the primary and secondary standards, respectively. Hydrogen at the parts per billion level has a tendency to increase in aluminum cylinders over time. We fit the calibration histories of these standards with zero-, first-, or second-order polynomial functions of time and use the time-dependent mole fraction assignments on the WMO scale to reprocess all tank air and flask air H2 measurement records. The robustness of the scale propagation over multiple years is evaluated with the regular analysis of target air cylinders and with long-term same-air measurement comparison efforts with WMO GAW partner laboratories. Long-term calibrated, globally distributed, and freely accessible measurements of H2 and other gases and isotopes continue to be essential to track and interpret regional and global changes in the atmosphere composition. The adoption of the WMO H2 calibration scale and subsequent reprocessing of NOAA atmospheric data constitute a significant improvement in the NOAA H2 measurement records.

Pétron, Gabrielle↗