Search NASA⌕ Search

SEARCH · Search NASA

Results for “MPI”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

ChatMPI: LLM-Driven MPI Code Generation for HPC Workloads

The Message Passing Interface (MPI) standard plays a crucial role in enabling scientific applications for parallel computing and is an essential component in high-performance computing (HPC). However, implementing MPI code manually—especially applying a proper domain decomposition and communication pattern—is a challenging and error-prone task. We present ChatMPI, an AI assistant for MPI parallelization of sequential C codes. In our analysis, we focus on testing six essential HPC workloads, which are based on Basic Linear Algebra Subprograms levels 1, 2, and 3 as well as sparse, stencil, and iterative operations. We analyze the process of creating ChatMPI by using the ChatHPC library. This lightweight large language model (LLM)–based infrastructure enables HPC experts to efficiently create and supervise trustworthy AI capabilities for critical HPC software tasks. We study the data required for training (fine-tuning) ChatMPI to generate parallel codes that not only use MPI syntax correctly but also apply HPC techniques to reduce memory communication and maximize performance by using proper work decomposition. With a relatively small training dataset composed of a few dozen prompts and fewer than 15 minutes of fine-tuning on one node equipped with two NVIDIA H100 GPUs, ChatMPI elevates trustworthiness for MPI code generation of current LLMs (e.g., Code Llama, ChatGPT-4o and ChatGPT 5). Additionally, we evaluate the performance of the MPI codes generated by ChatMPI in comparison with the ones generated by ChatGPT-4o and ChatGPT-5. The codes generated by ChatMPI provide up to a 4 × boost in performance by using better problem decomposition, communication patterns, and HPC techniques (e.g., communication avoiding).

Valero Lara, Pedro [ORNL] (ORCID:0000000214794310)↗
DOE OSTI2025

Towards Seamless Interoperability of MPI-OpenMP Applications

A chasm exists between mathematical software libraries written for MPI-based applications and those written for OpenMP applications. Recently, however, PETSc enables the simple use of its MPI-based linear solvers from OpenMP applications. Separately, the MPICH MPI development team has started a new project to allow almost seamless MPI use in OpenMP applications. Both proposed approaches would result in a similar user experience. Here, we discuss the reasons for these projects and their potential for providing more numerical library choices for OpenMP applications, including the unlimited assortment of linear solvers available in PETSc. In addition, we present the performance of an application using the first approach, demonstrating its efficacy.

MPI↗
DOE OSTI2026

A Flexible Forwarding Scheme to Improve Latency-Bound Irregular P2P Communication in MPI

We propose an algorithm to efficiently perform latency-bound communication scenarios that consist of many small messages. In these parallel scenarios, processes typically pass around a lot of small-sized messages of a few KBs of size. Performing communication operations with P2P MPI routines or collective MPI routines (including neighborhood collectives) in such scenarios may not always yield the optimal results and may not resolve the latency bottleneck. To this end, we develop a regular structure called virtual process topology (VPT) on which the messages can be communicated in a structured and controlled manner. Using parameters of this topology, one can tune the rate of aggression in tackling the latency costs. We demonstrate that our communication algorithm is preferable to MPI P2P and collective routines for latency-bound communication and it can easily be adapted only by replacing calls to MPI routines in a parallel application. We show how to adapt existing topology-aware mapping heuristics to address the volume overhead due to communicating messages on the VPT. Moreover, we propose a novel swap-based mapping heuristic to address this overhead by optimizing the maximum volume handled by a process. Experiments on synthetic communication graphs as well as real-world applications such as parallel Canonical Polyadic sparse tensor decomposition and parallel sparse matrix-dense matrix multiplication show that our approach is a powerful way of overcoming the bottlenecks posed by sparse and latency-bound irregular communication.

communication algorithm↗
DOE OSTI2024

Implementing One Sided Partitioned Communication in Open MPI

This report introduces partitioned communication, a new MPI 4.0 interface that enables early bird communication by overlapping communication and computation. By partitioning messages into smaller sub-messages, MPI can start partial data transfers early. Performance studies show that the RMA implementation outperforms the Persistent implementation, despite some constraints. This report details a new opt-in RMA implementation, offering a high-performance option for partitioned communication that imposes some additional limitations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration

TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.

Layer-parallel↗
DOE OSTI2026

Open MPI for HPSF

Explore the source record for details and available documents.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

TeMPI Shim

This is the manual for the TeMPI Shim library, whose goal is to facilitate scientific apps being loosely coupled through MPI. TeMPI Shim can be used to create and manage Message Passing Interface (MPI, see [MPI]) Communicators for Single Program, Multiple Data (SPMD, see [SPMD]) and Multiple Programs, Multiple Data (MPMD, see [MPMD]) workflows. MPI provides the MPI_APPNUM command number to each of the processes within its context. This number, starting at 0, is the application number being executed. In a case where 3 applications are being run in MPMD mode, there would be MPI_APPNUM values of 0, 1, and 2. TeMPI Shim creates intra- and inter-communicators between each pair of MPI_APPNUM values. In the aforementioned case, application 0 would have intra- and inter-communicators to speak to itself, application 1, and application 2. This is replicated for each of the applications. Additionally, TeMPI Shim creates intra- and inter-communicators for the first MPI rank of each application to directly communicate only with each other. Finally, TeMPI Shim creates its own copy of the default world communicator, i.e., MPI_COMM_WORLD. In the case where there is only a single application, it will have the communicators to only speak with itself. TeMPI Shim is useful in this case since it is considered good practice for MPI applications to copy the default world communicator and reference this copy (see [Duplicate World]_) anyways. Ultimately, it provides value independent of the number of applications present.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2024

Multihead Attention U‐Net for Magnetic Particle Imaging–Computed Tomography Image Segmentation

Magnetic particle imaging (MPI) is an emerging noninvasive molecular imaging modality with high sensitivity and specificity, exceptional linear quantitative ability, and potential for successful applications in clinical settings. Computed tomography (CT) is typically combined with the MPI image to obtain more anatomical information. Herein, a deep learning‐based approach for MPI‐CT image segmentation is presented. The dataset utilized in training the proposed deep learning model is obtained from a transgenic mouse model of breast cancer following administration of indocyanine green (ICG)‐conjugated superparamagnetic iron oxide nanoworms (NWs‐ICG) as the tracer. The NWs‐ICG particles progressively accumulate in tumors due to the enhanced permeability and retention (EPR) effect. The proposed deep learning model exploits the advantages of the multihead attention mechanism and the U‐Net model to perform segmentation on the MPI‐CT images, showing superb results. In addition, the model is characterized with a different number of attention heads to explore the optimal number for our custom MPI‐CT dataset.

Juhong, Aniwat↗
DOE OSTI2024

Distributed Order Recording Techniques for Efficient Record-and-Replay of Multi-threaded Programs

After all these years and all these other shared memory programming frameworks, OpenMP is still the most popular one. However, its greater levels of non-deterministic execution makes debugging and testing more challenging. The ability to record and deterministically replay the program execution is key to address this challenge. However, scalably replaying OpenMP programs is still an unresolved problem. In this paper, we propose two novel techniques that use Distributed Clock (DC) and Distributed Epoch (DE) recording schemes to eliminate excessive thread synchronization for OpenMP record and replay. Our evaluation on representative HPC applications with ReOMP, which we used to realize DC and DE recording, shows that our approach is 2-5x more efficient than traditional approaches that synchronize on every shared-memory access. Furthermore, we demonstrate that our approach can be easily combined with MPI-level replay tools to replay non-trivial MPI+OpenMP applications. We achieve this by integrating ReOMP into ReMPI, an existing scalable MPI record-and-replay tool, with only a small MPI-scale-independent runtime overhead.

Fu, Xiang↗
DOE OSTI2025

Parallel Programming in MCNP6

Monte Carlo N-Particle (MCNP)1 is a general-purpose Monte Carlo particle transport code developed by Los Alamos National Laboratory (LANL). To efficiently handle long simulations, MCNP version 6 (MCNP6) supports parallel execution using two primary programming models: • Shared-memory task-based threading using OpenMP (Open Multi-Processing), and • Distributed-memory calculations using MPI (Message Passing Interface). The OpenMP and MPI programming models enable MCNP6 to scale from desktop systems to high-performance computing (HPC) clusters, allowing users to run MCNP in one of three parallel modes: • OpenMP-only, • MPI-only, and • Hybrid (MPI + OpenMP). The choice of parallelization mode depends on the underlying computer architecture and the characteristics of the simulation problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Multiplicity dependent 𝐽/𝜓 and 𝜓⁡(2⁢𝑆) production at forward and backward rapidity in 𝑝 + 𝑝 collisions at $\sqrt{𝑠}$ = 200 GeV

Recent measurements of 𝐽/𝜓 production as a function of event charged-particle multiplicity at the collision energies of both the Large Hadron Collider (LHC) and the Relativistic Heavy Ion Collider (RHIC) show enhanced 𝐽/𝜓 production yields with increasing multiplicity. One potential explanation for this type of dependence is multiparton interactions (MPI). We present the first study of potential autocorrelations at RHIC energies and forward and backward rapidity of self-normalized 𝐽/𝜓 yields and 𝜓⁡(2⁢𝑆) to 𝐽/𝜓 ratio, as a function of self-normalized multiplicity in 𝑝 + 𝑝 collisions. In addition, detailed pythia studies tuned to RHIC energies were performed to investigate the MPI impacts. We find that the PHENIX data at RHIC are consistent with recent LHC measurements and can only be described by pythia calculations that include MPI effects. The forward and backward 𝜓⁡(2⁢𝑆) to 𝐽/𝜓 ratio is found to be less dependent on the charged-particle multiplicity.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗