Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel computing algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83

Multigrid-Reduction-in-Time for the Rotating Shallow Water Equations

We consider multilevel time-parallel methods for the numerical solution of the rotating shallow water equations. In particular, the multigrid-reduction-in-time (MGRIT) algorithm is used for the parallel time integration. An asymptotic model is used at the coarse levels while the full model is employed at the finer levels. The asymptotic model is well-suited for highly oscillatory partial differential equations like the rotating shallow water equations because it can accurately and stably take the required large time-steps on coarse levels. Our work exploits the flexibility of the MGRIT algorithm in terms of the number of levels and relaxation schemes to show some computational benefits, especially with respect to FCF-relaxation and data reuse.

97 MATHEMATICS AND COMPUTING↗

Automatic Transformation of MPI Programs to Asynchronous, Graph-Driven Form

The goals of this project are to develop new, scalable, high-fidelity algorithms for atomic-level simulations and program transformations that automatically restructure existing applications, enabling them to scale forward to Petascale systems and beyond. The techniques enable legacy MPI application code to exploit greater parallelism though increased latency hiding and improved workload assignment. The techniques were successfully demonstrated on high-end scalable systems located at DOE laboratories. Besides the automatic MPI program transformations efforts, the project also developed several new scalable algorithms for ab-initio molecular dynamics, including new massively parallel algorithms for hybrid DFT and new parallel in time algorithms for molecular dynamics and ab-initio molecular dynamics. These algorithms were shown to scale to very large number of cores, and they were designed to work in the latency hiding framework developed in this project. The effectiveness of the developments was enhanced by the direct application to real grand challenge simulation problems covering a wide range of technologically important applications, time scales and accuracies. These included the simulation of the electronic structure of mineral/fluid interfaces, the very accurate simulation of chemical reactions in microsolvated environments, and the simulation of chemical behavior in very large enzyme reactions.

97 MATHEMATICS AND COMPUTING↗

PARTISN on Advanced/Heterogeneous Processing Systems [Slides]

This presentation describes both the importance and the difficulty of parallelizing deterministic neutron transport calculations using the LANL code PARTISN. A brief history of the standard "KBA" parallelization method used for deterministic SN transport is presented, then the modifications necessary to implement this on Roadrunner are covered. The use of the Roadrunner algorithm with OpenMP is then described, along with a brief mention of an alternative thermal radiation transport algorithm for GPUs. Finally, ongoing work with PARTISN on the MIC many-core architecture is noted.

97 MATHEMATICS AND COMPUTING↗

Performance Results on CPU/GPU Exascale Architectures for OMEGA: The Ocean Model for E3SM Global Applications

The US Department of Energy (DOE) conducts climate simulations on some of the world’s largest supercomputers. These exascale machines use heterogeneous architectures with both CPUs and GPUs, and scientific codes must adapt to make full use of this computing power. Los Alamos National Lab is developing Omega: The Ocean Model for E3SM Global Applications, which is specifically designed for modern exascale computers. It uses external libraries that have been optimized for a variety of architectures to run on different supercomputers. Omega is an unstructured-mesh ocean model based on TRiSK numerical methods. It will be the new ocean component of the DOE’s Energy Exascale Earth System Model (E3SM). The algorithms in Omega follow those of the current ocean component, MPAS-Ocean, but it will be written in C++ rather than Fortran to take advantage of the Kokkos performance portability library. Omega spatial operators are written as Kokkos kernels to run efficiently on both CPUs and GPUs. Work on Omega began in 2023 with a new C++ framework for unstructured mesh partitioning, halo exchanges, parallel IO, and Kokkos interfaces. The current version, Omega-0, is being developed to solve the shallow water equations and at present includes all of the tendency terms but not time stepping. Here we share the results of Omega-0 verification and performance testing. Verification includes unit tests implemented with CTest as well as convergence tests in Polaris, an in-house python package with a large suite of test problems. Performance tests compare simulations conducted on CPUs versus GPUs and across different architectures: tests are run on Frontier, which has AMD “Optimized 3rd Gen EPYC” CPUs and AMD MI250X GPUs, as well as Perlmutter, which is composed of AMD EPYC 7763 CPUs and NVIDIA A100 GPUs.

58 GEOSCIENCES↗

Coupled flow-polymer dynamics via statistical field theory: Modeling and computation

Field-theoretic models, which replace interactions between polymers with interactions between polymers and one or more conjugate fields, offer a systematic framework for coarse-graining of complex fluids systems. While this approach has been used successfully to investigate a wide range of polymer formulations at equilibrium, field-theoretic models often fail to accurately capture the non-equilibrium behavior of polymers, especially in the early stages of phase separation. Here the 'two-fluid' approach serves as a useful alternative, treating the motions of fluid components separately in order to incorporate asymmetries between polymer molecules. In this work we focus on the connection of these two theories, drawing upon the strengths of each of the approaches in order to couple polymer microstructure with the dynamics of the flow in a systematic way. For illustrative purposes we work with an inhomogeneous melt of elastic dumbbell polymers, though our methodology will apply more generally to a wide variety of inhomogeneous systems. First we derive the model, incorporating thermodynamic forces into a two-fluid model for the flow through the introduction of conjugate chemical potential and elastic strain fields for the polymer density and stress. The resulting equations are composed of a system of fourth order PDEs coupled with a non-linear, non-local optimization problem to determine the conjugate fields. The coupled system is severely stiff and with a high degree of computational complexity. Next, we overcome the formidable numerical challenges posed by the model by designing a robust semi-implicit method based on linear asymptotic behavior of the leading order terms at small scales, by exploiting the exponential structure of global (integral) operators, and by parallelizing the non-linear optimization problem. The semi-implicit method effectively removes the fourth order stability constraint associated with explicit methods and we observe only a first order time-step restriction. The algorithm for solving the non-linear optimization problem, which takes advantage of the form of the operators being optimized, reduces the overall simulation time by several orders of magnitude. We illustrate the methodology with several examples of phase separation in an initially quiescent flow.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

MrHyDE v.1.0

SAND2024-01324O MrHyDE, which stands for Multi-resolution Hybridized Differential Equations, is a general-purpose C++ package for the solution of coupled multiphysics and multiscale systems on massively parallel computing systems. MrHyDE is designed to enable moving beyond forward simulation for multiscale applications which includes optimization, control, uncertainty quantification, and stochastic inversion. The framework provides interfaces to several packages within the Trilinos framework and leverages automatic differentiation to enable adjoint capabilities for large-scale, gradient-based optimization. MrHyDE provides automated multiscale capabilities through a subgrid model interface and multiscale Dirichlet-to-Neumann maps. For extreme-scale applications, MrHyDE provides in situ data-compression algorithms to reduce memory requirements while maintaining performance. MrHyDE is a general-purpose, computational framework for the solution of multiscale and multiphysics applications. It uses a combination of structure-preserving, physics-compatible discretizations, fully implicit methods, multi-resolution schemes, or fully explicit methods. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗

Critical Path-Based Thread Placement for NUMA Systems

Multicore multiprocessors use Non Uniform Memory Architecture (NUMA) to improve their scalability. However,NUMA introduces performance penalties due to remote memory accesses. Without efficiently managing data layout and thread mapping to cores, scientific applications, even if they are optimized for NUMA, may suffer performance loss. In this paper, we present an algorithm that optimizes the placement of OpenMP threads on NUMA processors. By collecting information from hardware counters and defining new metrics to capture the effects of thread placement, the algorithm reduces NUMA performance penalty by minimizing the critical path of OpenMP parallel regions and by avoiding local memory resource contention. We evaluate our algorithm with NPB benchmarks and achieve performance improvement between 8.13% and 25.68%, compared to the OS default scheduling.

99 GENERAL AND MISCELLANEOUS↗

Combining machine-learned and empirical force fields with the parareal algorithm: application to the diffusion of atomistic defects

We numerically investigate an adaptive version of the parareal algorithm in the context of molecular dynamics. This adaptive variant has been originally introduced in [1]. We focus here on test cases of physical interest where the dynamics of the system is modelled by the Langevin equation and is simulated using the molecular dynamics software LAMMPS. In this work, the parareal algorithm uses a family of machine-learning spectral neighbor analysis potentials (SNAP) as fine, reference, potentials and embedded-atom method potentials (EAM) as coarse potentials. We consider a self-interstitial atom in a tungsten lattice and compute the average residence time of the system in metastable states. Our numerical results demonstrate significant computational gains using the adaptive parareal algorithm in comparison to a sequential integration of the Langevin dynamics. We also identify a large regime of numerical parameters for which statistical accuracy is reached without being a consequence of trajectorial accuracy.

36 MATERIALS SCIENCE↗

Characterization of a Pixelated Cadmium Telluride Detector System Using a Polychromatic X-Ray Source and Gold Nanoparticle-Loaded Phantoms for Benchtop X-Ray Fluorescence Imaging

In this paper, the imaging dose and scan time have been considered as the two major constraints for routine benchtop x-ray fluorescence computed tomography (XFCT) imaging. One way to address this issue is to acquire x-ray fluorescence (XRF) signals in parallel through a 2D array of single-crystal detectors or a pixelated detector along with the cone-beam x-ray source. To identify a detector system suitable for this purpose, a commercially available, fully spectroscopic cadmium telluride (CdTe) pixelated detector, HEXITEC (High-Energy X-ray Imaging Technology), was tested under the experimental conditions optimized for benchtop XFCT imaging of gold nanoparticles (GNPs). Specifically, two different parallel-hole stainless steel collimators were fabricated and coupled with the detector for seamless integration into our existing benchtop cone-beam XFCT system. After the detector deployment, this benchtop XFCT system was used to detect XRF photons from GNP-loaded phantoms. A pixel-merging algorithm was introduced to enhance the sensitivity of XRF photon detection thereby minimizing the scan time. The effect of pixel-level charge sharing correction algorithms was investigated within the context of benchtop XFCT imaging. The detector energy resolution, in terms of the full width at half maximum (FWHM) values at different gold K-shell XRF energies, was also determined. Of the two charge sharing correction algorithms examined, the charge sharing addition gave better sensitivity than the charge sharing discrimination (csd). On the other hand, under the current experimental conditions, the energy resolution of the HEXITEC detector was the best with the csd and estimated to be 1.56 keV FWHM at 66-69 keV photon energy. Overall, despite some degradation of the detector energy resolution (compared with typical single crystal CdTe detectors), the HEXITEC detector enabled parallel data acquisition under the experimental conditions typical of benchtop XFCT imaging and operated well within our benchtop XFCT setup.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Implementation of High-Order Multireference Coupled-Cluster Methods on Intel Many Integrated Core Architecture

Here we discuss the implementation of multireference coupled-cluster formalism with singles, doubles, and noniterative triples (MRCCSD(T)), which is capable of taking advantage of the processing power of the Intel Xeon Phi coprocessor. We discuss the integration of two levels of parallelism underlying the MRCCSD(T) implementation with computational kernels designed to offload the computationally intensive parts of the MRCCSD(T) formalism to Intel Xeon Phi coprocessors. Special attention is given to the enhancement of the parallel performance by task reordering that has improved load balancing in the noniterative part of the MRCCSD(T) calculations. We also discuss aspects regarding efficient optimization and vectorization strategies.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Towards a high performance geometry library for particle-detector simulations

Thread-parallelization and single-instruction multiple data (SIMD) ”vectorisation” of software components in HEP computing has become a necessity to fully benefit from current and future computing hardware. In this context, the Geant-Vector/GPU simulation project aims to re-engineer current software for the simulation of the passage of particles through detectors in order to increase the overall event throughput. As one of the core modules in this area, the geometry library plays a central role and vectorising its algorithms will be one of the cornerstones towards achieving good CPU performance. Here, we report on the progress made in vectorising the shape primitives, as well as in applying new C++ template based optimizations of existing code available in the Geant4, ROOT or USolids geometry libraries. We will focus on a presentation of our software development approach that aims to provide optimized code for all use cases of the library (e.g., single particle and many-particle APIs) and to support different architectures (CPU and GPU) while keeping the code base small, manageable and maintainable. We report on a generic and templated C++ geometry library as a continuation of the AIDA USolids project. As a result, the experience gained with these developments will be beneficial to other parts of the simulation software, such as for the optimization of the physics library, and possibly to other parts of the experiment software stack, such as reconstruction and analysis.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

An Algorithmic and Software Pipeline for Very Large Scale Scientific Data Compression with Error Guarantees

Efficient data compression is becoming increasingly critical for storing scientific data because many scientific applications produce vast amounts of data. This paper presents an end-to-end algorithmic and software pipeline for data compression that guarantees both error bounds on primary data (PD) and derived data, known as Quantities of Interest (QoI).We demonstrate the effectiveness of the pipeline by compressing fusion data generated by a large-scale fusion code, XGC, which produces tens of petabytes of data in a single day. We demonstrate that the compression is conducted by setting aside computational resources known as staging nodes, and does not impact the simulation performance. For efficient parallel I/O, the pipeline uses ADIOS2, which many codes such as XGC already use for their parallel I/O. We show that our approach can compress the data by two orders of magnitude while guaranteeing high accuracy on both the PD and the QoIs. Further, the amount of resources required by compression is a few percent of the resources required by simulation while ensuring that the compression time for each stage is less than the corresponding simulation time.This pipeline consists of three main steps. The first step decomposes the data using domain decomposition into small subdomains. Each subdomain is then compressed independently to achieve a high level of parallelism. The second step uses existing techniques that guarantee error bounds on the primary data for each subdomain. The third step uses a post-processing optimization technique based on Lagrange multipliers to reduce the QoI errors for data corresponding to each subdomain. The Lagrange multipliers generated can be further quantized or truncated to increase the compression level. All of the above characteristics of our approach make it highly practical to apply on-the-fly compression while guaranteeing errors on QoIs that are critical to the scientists.

Banerjee, Tania↗

Stochastic Simulation of Complex Fluid Flows (Progress Report for period 07/01/2016 - 06/30/2018)

At a microscopic scale, fluids are composed of molecules whose positions and velocities are random. This gives rise to thermal fluctuations that span the whole range of scales from the microscopic through the mesoscopic, and even the macroscopic. The inclusion of thermal fluctuations is crucial in multi-scale models, which are an important theme in the research program of the DOE Office of Science, and in particular the ASCR Applied Mathematics program's priority focus area on modeling of complex systems involving processes that span vastly different time and/or length scales. In this five-year Early Career project, the PI Aleksandar Donev and collaborators developed computational algorithms for modeling complex fluid mixtures at small scales using a formulation based on fluctuating hydrodynamics. Novel computational methods were developed to model complex fluids with increasing physical complexity, starting from binary miscible and immiscible mixtures, going through multispecies non-reactive and reactive mixtures, and culminating with reactive electrolytes mixtures of neutral molecules and ions. In close collaboration with the group of John Bell at Lawrence Berkeley National Laboratory, the methods were implemented in a scalable computational framework suitable for modern parallel supercomputers, and made publicly available on github. A number of physical examples in which giant nonequilibrium fluctuations are improtant were studied, with a special focus on instabilities at a liquid-liquid interface driven by gravity, diffusion, reactions, and/or electric fields. The methods and codes developed in this project are expected to enable other novel applications in the DOE Basic Energy Sciences program, and engineering sciences more broadly.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Terascale High-Fidelity Simulations of Turbulent Combustion with Detailed Chemistry: Spray Simulations

The Terascale High-Fidelity Simulations of Turbulent Combustion (TSTC) project is a multi-university collaborative effort to develop a high-fidelity turbulent reacting flow simulation capability utilizing terascale, massively parallel computer technology. The main paradigm of the approach is direct numerical simulation (DNS) featuring the highest temporal and spatial accuracy, allowing quantitative observations of the fine-scale physics found in turbulent reacting flows as well as providing a useful tool for development of sub-models needed in device-level simulations. Under this component of the TSTC program the simulation code named S3D, developed and shared with coworkers at Sandia National Laboratories, has been enhanced with new numerical algorithms and physical models to provide predictive capabilities for turbulent liquid fuel spray dynamics. Major accomplishments include improved fundamental understanding of mixing and auto-ignition in multi-phase turbulent reactant mixtures and turbulent fuel injection spray jets.

02 PETROLEUM↗

Evolution of the SLATE linear algebra library

SLATE (Software for Linear Algebra Targeting Exascale) is a distributed, dense linear algebra library targeting both CPU-only and GPU-accelerated systems, developed over the course of the Exascale Computing Project (ECP). While it began with several documents setting out its initial design, significant design changes occurred throughout its development. In some cases, these were anticipated: an early version used a simple consistency flag that was later replaced with a full-featured consistency protocol. In other cases, performance limitations and software and hardware changes prompted a redesign. Sequential communication tasks were parallelized; host-to-host MPI calls were replaced with GPU device-to-device MPI calls; more advanced algorithms such as Communication Avoiding LU and the Random Butterfly Transform (RBT) were introduced. Early choices that turned out to be cumbersome, error prone, or inflexible have been replaced with simpler, more intuitive, or more flexible designs. Applications have been a driving force, prompting a lighter weight queue class, nonuniform tile sizes, and more flexible MPI process grids. Of paramount importance has been building a portable library that works across several different GPU architectures – AMD, Intel, and NVIDIA – while keeping a clean and maintainable codebase. Here we explore the evolving design choices and their effects, both in terms of performance and software sustainability.

Gates, Mark↗

The 3-D Monte Carlo neutron and photon transport code MCMG and its algorithms

The 3-D Monte Carlo neutron and photon transport parallel code MCMG is developed. A new collision mechanism based on material but not nuclide is added in the code. Geometry cells and surfaces can be dynamically extended. Combination of multigroup and continuous cross-section transport is developed. The multigroup scattering is expansible to P5 and upper scattering is considered. Various multigroup libraries can be easily equipped in the code. The same results with the experiments and the MCNP code are obtained for a series of modes. The speedup of MCMG is a factor of 2-4 relative to the MCNP code in speed. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

GPU-Accelerated Solution of the Bethe–Salpeter Equation for Large and Heterogeneous Systems

We present a massively parallel GPU-accelerated implementation of the Bethe–Salpeter equation (BSE) for the calculation of the vertical excitation energies (VEEs) and optical absorption spectra of condensed and molecular systems, starting from single-particle eigenvalues and eigenvectors obtained with density functional theory. The algorithms adopted here circumvent the slowly converging sums over empty and occupied states and the inversion of large dielectric matrices through a density matrix perturbation theory approach and a low-rank decomposition of the screened Coulomb interaction, respectively. Further computational savings are achieved by exploiting the nearsightedness of the density matrix of semiconductors and insulators to reduce the number of screened Coulomb integrals. We scale our calculations to thousands of GPUs with a hierarchical loop and data distribution strategy. The efficacy of our method is demonstrated by computing the VEEs of several spin defects in wide-band-gap materials, showing that supercells with up to 1000 atoms are necessary to obtain converged results. We discuss the validity of the common approximation that solves the BSE with truncated sums over empty and occupied states. In conclusion, we then apply our GW-BSE implementation to a diamond lattice with 1727 atoms to study the symmetry breaking of triplet states caused by the interaction of a point defect with an extended line defect.

Absorption spectra↗

Moment representation in the lattice Boltzmann method on massively parallel hardware

The widely-used lattice Boltzmann method (LBM) for computational fluid dynamics is highly scalable, but also significantly memory bandwidth-bound on current architectures. This paper presents a new regularized LBM implementation that reduces the memory footprint by only storing macroscopic, moment-based data. We show that the amount of data that must be stored in memory during a simulation is reduced by up to 47%. We also present a technique for cache-aware data re-utilization and show that optimizing cache utilization to limit data motion results in a similar improvement in time to solution. These new algorithms are implemented in the hemodynamics solver HARVEY and demonstrated using both idealized and realistic biological geometries. We develop a performance model for the moment representation algorithm and evaluate the performance on Summit.

Vardhan, Madhurima↗