Search NASA⌕ Search

SEARCH · Search NASA

Results for “Parallelization efficiency”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

A Framework to Analyze the Performance of Load Balancing Schemes for Ensembles of Stochastic Simulations

Ensembles of simulations are employed to estimate the statistics of possible future states of a system, and are widely used in important applications such as climate change and biological modeling. Ensembles of runs can naturally be executed in parallel. However, when the CPU times of individual simulations vary considerably, a simple strategy of assigning an equal number of tasks per processor can lead to serious work imbalances and low parallel efficiency. This paper presents a new probabilistic framework to analyze the performance of dynamic load balancing algorithms for ensembles of simulations where many tasks are mapped onto each processor, and where the individual compute times vary considerably among tasks. Four load balancing strategies are discussed: most-dividing, all-redistribution, random-polling, and neighbor-redistribution. Simulation results with a stochastic budding yeast cell cycle model are consistent with the theoretical analysis. It is especially significant that there is a provable global decrease in load imbalance for the local rebalancing algorithms due to scalability concerns for the global rebalancing algorithms. The overall simulation time is reduced by up to 25 %, and the total processor idle time by 85 %.

ALGORITHMS↗

Simulation of Coupled Processes of Flow, Transport, and Storage of CO 2 in Saline Aquifers

This report is the final scientific one for the award DE- FE0000988 entitled “Simulation of Coupled Processes of Flow, Transport, and Storage of CO 2 in Saline Aquifers.” The work has been divided into six tasks. In task, “Development of a Three-Phase Non-Isothermal CO 2 Flow Module,” we developed a fluid property module for brine-CO 2 mixtures designed to handle all possible phase combinations of aqueous phase, sub-critical liquid and gaseous CO 2 , supercritical CO 2 , and solid salt. The thermodynamic and thermophysical properties of brine-CO 2 mixtures (density, viscosity, and specific enthalpy of fluid phases; partitioning of mass components among the different phases) use the same correlations as an earlier fluid property module that does not distinguish between gaseous and liquid CO 2 -rich phases. We verified the fluid property module using two leakage scenarios, one that involves CO 2 migration up a blind fault and subsequent accumulation in a secondary “parasitic” reservoir at shallower depth, and another investigating leakage of CO 2 from a deep storage reservoir along a vertical fault zone. In task, “Development of a Rock Mechanical Module,” we developed a massively parallel reservoir simulator for modeling THM processes in porous media brine aquifers. We derived, from the fundamental equations describing deformation of porous elastic media, a momentum conservation equation relating mean stress, pressure, and temperature, and incorporated it alongside the mass and energy conservation equations from the TOUGH2 formulation, the starting point for the simulator. In addition, rock properties, namely permeability and porosity, are functions of effective stress and other variables that are obtained from the literature. We verified the simulator formulation and numerical implementation using analytical solutions and example problems from the literature. For the former, we matched a one-dimensional consolidation problem and a two-dimensional simulation of the Mandel-Cryer effect. For the latter, we obtained a good match of temperature and gas saturation profiles, and surface uplift, after injection of hot fluid into a model of a caldera structure. In task, “Incorporation of Geochemical Reactions of Selected Important Species,” we developed a novel mathematical model of THMC processes in porous and fractured saline aquifers, simulating geo-chemical reactions associated with CO 2 sequestration in saline aquifers. Two computational frameworks, sequentially coupled and fully coupled, were used to simulate the reactions and transport. We verified capabilities of the THMC model to treat complex THMC processes during CO 2 sequestration by analytical solutions and we constructed reactive transport models to analyze the THMC process quantitatively. Three of these are 1D reactive transport under chemical equilibrium, a batch reaction model with equilibrium chemical reactions, and a THMC model with CO 2 dissolution. In task “Study of Instability in CO 2 Dissolution-Diffusion-Convection Processes,” We reviewed literature related to the study of density driven convective flows and on the instability of CO 2 dissolution-diffusion-convection processes. We ran simulations that model the density-driven flow instability that would occur during CO 2 sequestration. CO 2 diffused through the top of the system and dissolved in the aqueous phase there, increasing its density. Density fingers formed along the top boundary, and coalesced into a few prominent ones, causing convective flow that forced the fluid to the system bottom. These simulations were in two and three dimensions. We ran additional simulations of convective mixing with density contrast caused by variable dissolved CO 2 concentration in saline water, modeled after laboratory experiments in which supercritical CO2 was circulated in the headspace above a brine saturated packed sand in a pressure vessel. As CO 2 dissolved into the upper part of the saturated sand, liquid phase density increases causing instability and setting off convective mixing. We obtained good agreement with the laboratory experiments, which were characterized by finger development and associated mixing of dissolved CO 2 into the system. We then varied a wide range of parameters and conceptual models in order to analyze the possibility of convective mixing under different conditions, such as various boundary conditions, and chemical reaction conditions. The CO 2 fingers from different simulations showed great differences as time progressed, caused by permeability heterogeneity. The early time diffusive phenomenon was captured by fine grid resolution, and the permeability heterogeneity affected the pattern of the CO 2 fingers. In addition, the fingers from three-dimensional simulations tended to be larger and flatter than the two-dimensional ones. In task “Implementation of Efficient Parallel Computing Technologies,” we made enhancements and modifications to our code in order to substantially increase the grid size that could be run. We installed and ran it on various platforms, including a multi-core PC and a cluster, and verified the numerical implementation and parallel code using an example problem from the literature. This problem, with a grid size of sixty million, utilized the cluster’s entire memory and all of its processors. In task “Implementation of General Fracture Conceptual Models,” we used the MINC approach, a generalization of the double-porosity concept, to model flow through porous and fractured media. In this approach, flow within the matrix is described by subdividing the matrix into nested volumes, with flow occurring between adjacent nested matrix volumes as well as between the fractures and the outer matrix volume. We generalized Hooke’s law to a thermo-multi- poroelastic medium, and derived from the fundamental equations describing deformation of porous and fractured elastic media a momentum conservation equation for thermo-multi- poroelastic media. This equation is a generalization to multi-poroelastic media of the one derived in Task 3.0 for single porosity media. We describe two simulations to provide model verification and application examples. The first, one-dimensional consolidation of a double-porosity medium, is compared to an analytical solution. The second is a match of published results from the literature, a simulation of CO 2 injection into hypothetical aquifer-caprock systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A scalable compressible volume of fluid solver using a stratified flow model

Here, in this paper, a Volume of Fluid method which uses a Stratified Flow model for flux calculation is proposed to solve a compressible multiphase flow in a way that has high parallel efficiency. The solver is part of an open-source computing tool to study fuel entrainment and combustion in the application of paraffin-based hybrid rocket motors. This paper focuses on the compressible Volume of Fluid solver for the liquid and gaseous phases in this environment. This solver uses perfect gas and stiffened gas models for equation of state, with the ability to easily add other models as desired. Additionally a comparison of a Riemann solver versus an AUSM+up scheme is conducted, showing improvement with the later in the solution of multiphase flows. This solver successfully reproduces shock tube cases with the expected accuracy, as well as simulating two-dimensional shear and gravity-driven flows. Results also show that the code is able to utilize a Stratified Flow model to evaluate shear flow without the need for interface reconstruction or gradient calculations. Finally, it is shown that the solver has near-ideal scaling under strong scaling tests as well as good performance in static scaling, giving improved performance over current options and implying future advancement of high-performance computing efficiency for multiphase flow solvers.

97 MATHEMATICS AND COMPUTING↗

Fragme∩t: An Open‐Source Framework for Multiscale Quantum Chemistry Based on Fragmentation

Fragment-based quantum chemistry offers a means to circumvent the nonlinear computational scaling of conventional electronic structure calculations, by partitioning a large calculation into smaller subsystems then considering the many-body interactions between them. Variants of this approach have been used to parameterize classical force fields and machine learning potentials, applications that benefit from interoperability between quantum chemistry codes. However, there is a dearth of software that provides interoperability yet is purpose-built to handle the combinatorial complexity of fragment-based calculations. To fill this void we introduce “Fragme∩t”, an open-source software application that provides a tool for community validation of fragment-based methods, a platform for developing new approximations, and a framework for analyzing many-body interactions. Fragme∩t includes algorithms for automatic fragment generation and structure modification, and for distance- and energy-based screening of the requisite subsystems. Checkpointing, database management, and parallelization are handled internally and results are archived in a portable database. Interfaces to various quantum chemistry engines are easy to write and exist already for Q-Chem, PySCF, xTB, Orca, CP2K, MRCC, Psi4, NWChem, GAMESS, and MOPAC. Applications reported here demonstrate parallel efficiencies around 96% on more than 1000 processors but also showcase that the code can handle large-scale protein fragmentation using only workstation hardware, all with a codebase that is designed to be usable by non-experts. Fragme∩t conforms to modern software engineering best practices and is built upon well established technologies including Python, SQLite, and Ray. The source code is available under the Apache 2.0 license.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Flexible User-Defined Domain Decomposition in Kilometer-Scale E3SM Land Model Simulation

The Energy Exascale Earth System Model (E3SM) Land Model (ELM) has been extended to kilometer-scale (km-ELM) resolutions, enabling high-fidelity simulations of terrestrial processes at 1 km x 1 km grid spacing. In ELM, domain decomposition partitions the computational domain across processors, ensuring efficient parallel execution. Currently, round-robin decomposition is applied, providing a straightforward way to distribute computational workload. As ELM continues evolving at the kilometer-scale (km-scale), particularly with integrating lateral flow modeling, decomposition strategies must also account for the increased workload and data movement. This paper introduces a flexible user-defined domain decomposition framework, allowing users to customize domain partitioning based on application requirements. The impact of different decomposition strategies is evaluated across various applications concerning computation, communication, and I/O. Results demonstrate that while 1D partitioning yields superior I/O performance, k-nearest neighbors (KNN) clustering effectively reduces inter-process communication overhead. This study lays the groundwork for scalable partitioning in large-scale land surface simulations, enhancing next-generation Earth system modeling.

Wang, Dali [ORNL] (ORCID:0000000168065108)↗

Performance Characterization of De Novo Genome Assembly on Leading Parallel Systems.

De novo genome assembly is one of the most important and challenging computational problems in modern genomics; further, it shares algorithms and communication patterns important to other graph analytic and irregular applications. Unlike simulations, it has no floating point arithmetic and is dominated by small memory transactions within and between computing nodes. In this work, we focus on the highly scalable HipMer assembler and identify the dominant algorithms and communication patterns, also using microbenchmarks to capture the workload. We evaluate HipMer on a variety of platforms from the latest HPC systems to ethernet clusters. HipMer performs well on all single node systems, including the Xeon Phi manycore architecture. Given large enough problems, it also demonstrates excellent scaling across nodes in an HPC system, but requires a high speed network with low overhead and high injection rates. Our results shed light on the architectural features that are most important for achieving good parallel efficiency on this and related problems.

Ellis, Marquita↗

High Order Anchoring and Reinitialization of Level Set Function for Simulating Interface Motion

A second order interface anchoring method has been developed and used with fast sweeping algorithm for reinitialization of a level set function. The algebraic anchoring formulation ensures that the location of the actual interface is preserved, leading to better mass conservation property. It also provides high order accurate algebraic constraint for solving the Eikonal equation on a finite difference grid. Geometric properties of the interface such as surface normal and curvature are subsequently computed from the reinitialized distance function. Various analytical functions for modeling distortion in level set field are considered and accuracy of reinitialization is evaluated using first and second order anchoring schemes. It is also shown that accurate computation of interface curvature requires a high order anchor in addition to a high order fast sweeping method. Mass conservation property of reinitialization is also analyzed by considering test problems from literature including the classic Rider–Kothe single vortex problem. The formulation is suitable for efficient parallelization for both distributed memory and on-node shared memory parallel systems. In conclusion, scalability and performance of the reinitialization scheme on multiple architectures are demonstrated.

97 MATHEMATICS AND COMPUTING↗

On Eliminating Synchronous Communication in Molecular Simulations to Improve Scalability

Molecular dynamics simulation, as a complementary tool to experimentation, has become an important methodology for the understanding and design of molecular systems as it provides access to properties that are difficult, impossible or prohibitively expensive to obtain experimentally. Many of the available software packages have been parallelized to take advantage of modern massively concurrent processing resources. The challenge in achieving parallel efficiency is commonly attributed to the fact that molecular dynamics algorithms are communication intensive. This paper illustrates how an appropriately chosen data distribution and asynchronous one-sided communication approach can be used to effectively deal with the data movement within the Global Arrays/ARMCI programming model framework. A new put_notify capability is presented here, allowing the implementation of the molecular dynamics algorithm without any explicit global or local synchronization or global data reduction operations. In addition, this push-data model is shown to very effectively allow hiding data communication behind computation. Rather than data movement or explicit global reductions, the implicit synchronization of the algorithm becomes the primary challenge for scalability. Without any explicit synchronous operations, the scalability of molecular simulations is shown to depend only on the ability to evenly balance computational load.

Straatsma, TP↗

An efficient tensor transpose algorithm for multicore CPU, Intel Xeon Phi, and NVidia Tesla GPU

An efficient parallel tensor transpose algorithm is suggested for shared-memory computing units, namely, multicore CPU, Intel Xeon Phi, and NVidia GPU. The algorithm operates on dense tensors (multidimensional arrays) and is based on the optimization of cache utilization on x86 CPU and the use of shared memory on NVidia GPU. From the applied side, the ultimate goal is to minimize the overhead encountered in the transformation of tensor contractions into matrix multiplications in computer implementations of advanced methods of quantum many-body theory (e.g., in electronic structure theory and nuclear physics). A particular accent is made on higher-dimensional tensors that typically appear in the so-called multireference correlated methods of electronic structure theory. Depending on tensor dimensionality, the presented optimized algorithms can achieve an order of magnitude speedup on x86 CPUs and 2-3 times speedup on NVidia Tesla K20X GPU with respect to the na ve scattering algorithm (no memory access optimization). Furthermore, the tensor transpose routines developed in this work have been incorporated into a general-purpose tensor algebra library (TAL-SH).

97 MATHEMATICS AND COMPUTING↗

An extension to V ORO ++ for multithreaded computation of Voronoi cells

V ORO ++ is a software library written in C++ for computing the Voronoi tessellation, a technique in computational geometry that is widely used for analyzing systems of particles. V ORO ++ was released in 2009 and is based on computing the Voronoi cell for each particle individually. Here, we take advantage of modern computer hardware, and extend the original serial version to allow for multithreaded computation of Voronoi cells via the OpenMP application programming interface. We test the performance of the code, and demonstrate that it can achieve parallel efficiencies greater than 95% in many cases. Further, the multithreaded extension follows standard OpenMP programming paradigms, allowing it to be incorporated into other programs. We provide an example of this using the VoroTop software library, performing a multithreaded Voronoi cell topology analysis of up to 102.4 million particles.

97 MATHEMATICS AND COMPUTING↗

Billion-atom synchronous parallel kinetic Monte Carlo simulations of critical 3D Ising systems

An extension of the synchronous parallel kinetic Monte Carlo (spkMC) algorithm developed by Martinez et al. [J. Comp. Phys. 227 (2008) 3804] to discrete lattices is presented. The method solves the master equation synchronously by recourse to null events that keep all processors' time clocks current in a global sense. Boundary conflicts are resolved by adopting a chessboard decomposition into non-interacting sublattices. We find that the bias introduced by the spatial correlations attendant to the sublattice decomposition is within the standard deviation of serial calculations, which confirms the statistical validity of our algorithm. We have analyzed the parallel efficiency of spkMC and find that it scales consistently with problem size and sublattice partition. We apply the method to the calculation of scale-dependent critical exponents in billion-atom 3D Ising systems, with very good agreement with state-of-the-art multispin simulations.

97 MATHEMATICS AND COMPUTING↗

Parallel multiscale simulations of a brain aneurysm

Cardiovascular pathologies, such as a brain aneurysm, are affected by the global blood circulation as well as by the local microrheology. Hence, developing computational models for such cases requires the coupling of disparate spatial and temporal scales often governed by diverse mathematical descriptions, e.g., by partial differential equations (continuum) and ordinary differential equations for discrete particles (atomistic). However, interfacing atomistic-based with continuum-based domain discretizations is a challenging problem that requires both mathematical and computational advances. We present here a hybrid methodology that enabled us to perform the first multiscale simulations of platelet depositions on the wall of a brain aneurysm. The large scale flow features in the intracranial network are accurately resolved by using the high-order spectral element Navier–Stokes solver NεκTαr. The blood rheology inside the aneurysm is modeled using a coarse-grained stochastic molecular dynamics approach (the dissipative particle dynamics method) implemented in the parallel code LAMMPS. The continuum and atomistic domains overlap with interface conditions provided by effective forces computed adaptively to ensure continuity of states across the interface boundary. A two-way interaction is allowed with the time-evolving boundary of the (deposited) platelet clusters tracked by an immersed boundary method. The corresponding heterogeneous solvers (NεκTαr and LAMMPS) are linked together by a computational multilevel message passing interface that facilitates modularity and high parallel efficiency. Results of multiscale simulations of clot formation inside the aneurysm in a patient-specific arterial tree are presented. We also discuss the computational challenges involved and present scalability results of our coupled solver on up to 300 K computer processors. Validation of such coupled atomistic-continuum models is a main open issue that has to be addressed in future work.

60 APPLIED LIFE SCIENCES↗

A pseudospectral matrix method for time-dependent tensor fields on a spherical shell

We construct a pseudospectral method for the solution of time-dependent, non-linear partial differential equations on a three-dimensional spherical shell. The problem we address is the treatment of tensor fields on the sphere. As a test case we consider the evolution of a single black hole in numerical general relativity. A natural strategy would be the expansion in tensor spherical harmonics in spherical coordinates. Instead, we consider the simpler and potentially more efficient possibility of a double Fourier expansion on the sphere for tensors in Cartesian coordinates. As usual for the double Fourier method, we employ a filter to address time-step limitations and certain stability issues. We find that a tensor filter based on spin-weighted spherical harmonics is successful, while two simplified, non-spin-weighted filters do not lead to stable evolutions. The derivatives and the filter are implemented by matrix multiplication for efficiency. A key technical point is the construction of a matrix multiplication method for the spin-weighted spherical harmonic filter. As example for the efficient parallelization of the double Fourier, spin-weighted filter method we discuss an implementation on a GPU, which achieves a speed-up of up to a factor of 20 compared to a single core CPU implementation.

97 MATHEMATICS AND COMPUTING↗

A domain decomposition method for pseudo-spectral electromagnetic simulations of plasmas

Pseudo-spectral electromagnetic solvers (i.e. representing the fields in Fourier space) have extraordinary precision. In particular, Haber et al. presented in 1973 a pseudo-spectral solver that integrates analytically the solution over a finite time step, under the usual assumption that the source is constant over that time step. Yet, pseudo-spectral solvers have not been widely used, due in part to the difficulty for efficient parallelization owing to global communications associated with global FFTs on the entire computational domains. A method for the parallelization of electromagnetic pseudo-spectral solvers is proposed and tested on single electromagnetic pulses, and on Particle-In-Cell simulations of the wakefield formation in a laser plasma accelerator. The method takes advantage of the properties of the Discrete Fourier Transform, the linearity of Maxwell’s equations and the finite speed of light for limiting the communications of data within guard regions between neighboring computational domains. Although this requires a small approximation, test results show that no significant error is made on the test cases that have been presented. The proposed method opens the way to solvers combining the favorable parallel scaling of standard finite-difference methods with the accuracy advantages of pseudo-spectral methods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A high-order discontinuous Galerkin method for fluid–structure interaction with efficient implicit–explicit time stepping

We present a high-order accurate scheme for coupled fluid–structure interaction problems. The fluid is discretized using a discontinuous Galerkin method on unstructured tetrahedral meshes, and the structure uses a high-order volumetric continuous Galerkin finite element method. Standard radial basis functions are used for the mesh deformation. The time integration is performed using a partitioned approach based on implicit–explicit Runge–Kutta methods. The resulting scheme fully decouples the implicit solution procedures for the fluid and the solid parts, which we perform using two separate efficient parallel solvers. We demonstrate up to fifth order accuracy in time on a non-trivial test problem, on which we also show that additional subiterations are not required. We solve a benchmark problem of a cantilever beam in a shedding flow, and show good agreement with other results in the literature. Finally, we solve for the flow around a thin membrane at a high angle of attack in both 2D and 3D, and compare with the results obtained with a rigid plate.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Moments-based interface reconstruction, remap and advection

Here, we present a new moment-of-fluid (MOF 2 ) interface reconstruction method. It uses the zeroth, first, and second moments of the fragment of material inside a cell of the mesh to reconstruct a convex material polygon or a union of convex polygons that approximate the respective material fragment. The new method requires information about the material moments only for the cell under consideration. The MOF 2 method allows to exactly reproduce several convex shapes: corners, filaments, and some concave shapes: cell-complements to corners and filaments. Interface reconstruction is formulated as a local (for each cell), non-linear, equality constrained optimization problem, which does not require additional communication and allows for an efficient parallel implementation. We present an extensive set of test problems, both for interface reconstruction on a single cell, and for reconstruction of a variety of shapes on a variety of meshes. We describe how to perform two-material advection using the MOF 2 method and present the results for the classical advection tests. We also show the examples of material interface remapping needed in the framework of multi-material arbitrary Lagrangian-Eulerian methods, and give a brief description of a procedure that can be used to update the material moments on the Lagrangian stage of those methods.

97 MATHEMATICS AND COMPUTING↗

An adaptive moments-based interface reconstruction using intersection of the cell with one half-plane, two half-planes and a circle

We present a new adaptive moment-of-fluid (A-MOF) interface reconstruction method. It uses the zeroth, first, and second moments of the fragment of material inside a cell of the mesh to construct a shape that approximates the respective material fragment. The new method requires information about the material moments only for the cell under consideration. The adaptive method chooses between shapes obtained by the intersection of the cell with one half-plane, two half-planes, or a circle. The A-MOF method allows to exactly reproduce several convex shapes: corners, filaments, and their concave cell-complements; as well as pieces of the circles and its cell-compliments. Interface reconstruction is formulated as a local (for each cell), non-linear, equality constrained optimization problem, which does not require additional communication and allows for an efficient parallel implementation. In conclusion, we present an extensive set of test problems, both for interface reconstruction on a single cell, and for reconstruction of a variety of shapes on the entire mesh.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Efficient Heterogeneous Execution on Large Multicore and Accelerator Platforms: Case Study Using a Block Tridiagonal Solver

The algorithmic and implementation principles are explored in gainfully exploiting GPU accelerators in conjunction with multicore processors on high-end systems with large numbers of compute nodes, and evaluated in an implementation of a scalable block tridiagonal solver. The accelerator of each compute node is exploited in combination with multicore processors of that node in performing block-level linear algebra operations in the overall, distributed solver algorithm. Optimizations incorporated include: (1) an efficient memory mapping and synchronization interface to minimize data movement, (2) multi-process sharing of the accelerator within a node to obtain balanced load with multicore processors, and (3) an automatic memory management system to efficiently utilize accelerator memory when sub-matrices spill over the limits of device memory. Results are reported from our novel implementation that uses MAGMA and CUBLAS accelerator software systems simultaneously with ACML for multithreaded execution on processors. Overall, using 940 nVidia Tesla X2090 accelerators and 15,040 cores, the best heterogeneous execution delivers a 10.9-fold reduction in run time relative to an already efficient parallel multicore-only baseline implementation that is highly optimized with intra-node and inter-node concurrency and computation-communication overlap. Detailed quantitative results are presented to explain all critical runtime components contributing to hybrid performance.

Park, Alfred J↗