SEARCH · Search NASA
Results for “Parallelization efficiency”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Reconstructing householder vectors from Tall-Skinny QR
The Tall-Skinny QR (TSQR) algorithm is more communication efficient than the standard Householder algorithm for QR decomposition of matrices with many more rows than columns. However, TSQR produces a different representation of the orthogonal factor and therefore requires more software development to support the new representation. Further, implicitly applying the orthogonal factor to the trailing matrix in the context of factoring a square matrix is more complicated and costly than with the Householder representation. We show how to perform TSQR and then reconstruct the Householder vector representation with the same asymptotic communication efficiency and little extra computational cost. We demonstrate the high performance and numerical stability of this algorithm both theoretically and empirically. The new Householder reconstruction algorithm allows us to design more efficient parallel QR algorithms, with significantly lower latency cost compared to Householder QR and lower bandwidth and latency costs compared with Communication-Avoiding QR (CAQR) algorithm. Experiments on supercomputers demonstrate the benefits of the communication cost improvements: in particular, our experiments show substantial improvements over tuned library implementations for tall-and-skinny matrices. Furthermore, we also provide algorithmic improvements to the Householder QR and CAQR algorithms, and we investigate several alternatives to the Householder reconstruction algorithm that sacrifice guarantees on numerical stability in some cases in order to obtain higher performance.
A nonhydrostatic formulation for MPAS-Ocean
The Model for Prediction Across Scales-Ocean (MPAS-Ocean) is an open-source, global ocean model and is one component of a family of climate models within the MPAS framework, including atmosphere, sea-ice, and land-ice models. Here, in this work, a new formulation for the ocean model is presented that solves the nonhydrostatic, incompressible Boussinesq equations on an unstructured, staggered, z-level grid. The introduction of this nonhydrostatic capability is necessary for the resolution of internal wave dynamics and large eddy simulations. Compared to the standard, hydrostatic formulation, a nonhydrostatic pressure solver and a vertical momentum equation are added, where the PETSc (Portable Extensible Toolkit for Scientific Computation) library is used for the inversion of a large sparse system for the nonhydrostatic pressure. Numerical results on a stratified seiche, internal solitary wave, overflow and lock-exchange test cases are presented, and the parallel efficiency of the code is evaluated using up to 1024 processors.
A nonhydrostatic formulation for MPAS-Ocean
The Model for Prediction Across Scales-Ocean (MPAS-Ocean) is an open-source, global ocean model and is one component of a family of climate models within the MPAS framework, including atmosphere, sea-ice, and land-ice models. Here, in this work, a new formulation for the ocean model is presented that solves the nonhydrostatic, incompressible Boussinesq equations on an unstructured, staggered, z-level grid. The introduction of this nonhydrostatic capability is necessary for the resolution of internal wave dynamics and large eddy simulations. Compared to the standard, hydrostatic formulation, a nonhydrostatic pressure solver and a vertical momentum equation are added, where the PETSc (Portable Extensible Toolkit for Scientific Computation) library is used for the inversion of a large sparse system for the nonhydrostatic pressure. Numerical results on a stratified seiche, internal solitary wave, overflow and lock-exchange test cases are presented, and the parallel efficiency of the code is evaluated using up to 1024 processors.
Gyrokinetic particle-in-cell optimization on emerging multi- and manycore platforms
The next decade of high-performance computing (HPC) systems will see a rapid evolution and divergence of multi- and manycore architectures as power and cooling constraints limit increases in microprocessor clock speeds. Understanding efficient optimization methodologies on diverse multicore designs in the context of demanding numerical methods is one of the greatest challenges faced today by the HPC community. In this paper, we examine the efficient multicore optimization of GTC, a petascale gyrokinetic toroidal fusion code for studying plasma microturbulence in tokamak devices. For GTC’s key computational components (charge deposition and particle push), we explore efficient parallelization strategies across a broad range of emerging multicore designs, including the recently-released Intel Nehalem-EX, the AMD Opteron Istanbul, and the highly multithreaded Sun UltraSparc T2+. We also present the first study on tuning gyrokinetic particle-in-cell (PIC) algorithms for graphics processors, using the NVIDIA C2050 (Fermi). Our work discusses several novel optimization approaches for gyrokinetic PIC, including mixed-precision computation, particle binning and decomposition strategies, grid replication, SIMDized atomic floating-point operations, and effective GPU texture memory utilization. Overall, we achieve significant performance improvements of 1.3–4.7× on these complex PIC kernels, despite the inherent challenges of data dependency and locality. Finally, our work also points to several architectural and programming features that could significantly enhance PIC performance and productivity on next-generation architectures.
Exploring the interplay of resilience and energy consumption for a task-based partial differential equations preconditioner
Here, we discuss algorithm-based resilience to silent data corruptions (SDCs) in a task-based domain-decomposition preconditioner for partial differential equations (PDEs). The algorithm exploits a reformulation of the PDE as a sampling problem, followed by a solution update through data manipulation that is resilient to SDCs. The implementation is based on a server-client model where all state information is held by the servers, while clients are designed solely as computational units. Scalability tests run up to ~51K cores show a parallel efficiency greater than 90%. We use a 2D elliptic PDE and a fault model based on random single and double bit-flip to demonstrate the resilience of the application to synthetically injected SDC. We discuss two fault scenarios: one based on the corruption of all data of a target task, and the other involving the corruption of a single data point. We show that for our application, given the test problem considered, a four-fold increase in the number of faults only yields a 2% change in the overhead to overcome their presence, from 7% to 9%. We then discuss potential savings in energy consumption via dynamic voltage/frequency scaling, and its interplay with fault-rates, and application overhead.
Optimization of processor allocation for domain decomposed Monte Carlo calculations
Load imbalance plagues domain decomposed Monte Carlo calculations when sources are not uniform. Parallel efficiency for domain decomposed Monte Carlo transport calculations improves through a nonuniform allocation of processors over subdomains. In this work, we optimize the allocation with runtime diagnostics collected during a calibration step, then complete the full calculation. The diagnostic-based approach is compared to implicit filtering, an optimization algorithm for bound constrained noisy optimization problems. Finally, we consider both forward and hybrid radiation transport calculations to measure performance.
Numerical Analysis of the Dynamics of Two- and Three-Dimensional Fluidized Bed Reactors using an Euler-Lagrange Approach
Biomass thermochemical conversion, often done in fluidized beds, recently gained a lot of attention due to its potential to efficiently produce renewable liquid fuels. Optimization of reactor design and operating conditions, however, requires a fundamental understanding of bed dynamics. In this work, a numerical framework based on an Euler-Lagrange approach is developed and used to perform and analyze large-scale simulations of two- and three-dimensional periodic fluidized beds. Collisions are handled using a soft-sphere model. An efficient parallel implementation allows one to explicitly track over 30 million particles, which is representative of the number of particles found in lab-scale reactor, therefore demonstrating the capability of Lagrangian approaches to simulate realistic systems at that scale. An on-the-fly bubble identification and tracking algorithm is used to characterize bubble dynamics for inlet velocities up to 9 times the minimum fluidization velocity. Statistics for gas volume fraction, gas and particle velocities, bed expansion, and bubble size and velocity, is compared across the two- and three-dimensional configurations, and comparison with literature data generally shows good agreement. The wide distribution of gas residence times observed in the simulations is linked to the different gas hold-up characteristics of the gas-solid system.
Hands-on Performance Tuning of 3D Finite Difference Earthquake Simulation on GPU Fermi Chipset
3D simulation of earthquake ground motion is one of the most challenging computational problems in science. The emergence of graphic processing units (GPU) as an effective alternative to traditional general purpose processors has become increasingly capable in terms of accelerating scientific computing research. In this paper, we describe our experiences in porting AWP-ODC, a 3D finite difference seismic wave propagation code, to the latest GPU Fermi chipset. We completely rewrote this Fortran-based 13-point asymmetric stencil computation code in C and MPI-CUDA in order to take advantage of the powerful GPU computing capabilities. Our new CUDA code implemented the asymmetric 3D stencil on Fermi to make the best use of GPU on-chip memory for an aggressive parallel efficiency. Benchmark on NVIDIA Tesla M2090 demonstrated 10x speedup versus the original fully optimized AWP-ODC FORTRAN MPI code running on a single Intel Nehalem 2.4 GHz CPU socket (4 cores/CPU), and 15x speedup versus the same MPI code running on a single AMD Istanbul 2.6 GHz CPU socket (6 cores/CPU). Sustained single-GPU performance of 143.8 GFLOPS in single precision is benchmarked for the testing case of 128x128x960 mesh size.
PySpawn: Software for Nonadiabatic Quantum Molecular Dynamics
The ab initio multiple spawning (AIMS) method enables nonadiabatic quantum molecular dynamics simulations in an arbitrary number of dimensions, with potential energy surfaces provided by electronic structure calculations performed on-the-fly. However, the intricacy of the AIMS algorithm complicates software development, deployment on modern shared computer resources, and post-simulation data analysis. PySpawn is a nonadiabatic molecular dynamics software package that addresses these issues. Here, the program is designed to be easily interfaced with electronic structure software, and an interface to the TeraChem software package is described here. PySpawn introduces a task-based reorganization of the AIMS algorithm, allowing fine-grained restart capability and setting the stage for efficient parallelization in a future release. PySpawn includes a user-friendly and interactive Python analysis module that will enable novice users to painlessly adopt AIMS. As a demonstration of PySpawn’s simulation capability and analysis module, we report complete active space self-consistent field–based AIMS simulations of the 1,2- dithienyl-1,2-dicyanoethene molecule, a promising molecular photoswitch.
A Massively Parallel Implementation of the CCSD(T) Method Using the Resolution-of-the-Identity Approximation and a Hybrid Distributed/Shared Memory Parallelization Model
In this work, a parallel algorithm is described for the coupled-cluster singles and doubles method augmented with a perturbative correction for triple excitations [CCSD(T)] using the resolution-of-the-identity (RI) approximation for two-electron repulsion integrals (ERIs). The algorithm bypasses the storage of four-center ERIs by adopting an integral-direct strategy. The CCSD amplitude equations are given in a compact quasi-linear form by factorizing them in terms of amplitude-dressed three-center intermediates. A hybrid MPI/OpenMP parallelization scheme is employed, which uses the OpenMP-based shared memory model for intranode parallelization and the MPI-based distributed memory model for internode parallelization. Parallel efficiency has been optimized for all terms in the CCSD amplitude equations. Two different algorithms have been implemented for the rate-limiting terms in the CCSD amplitude equations that entail and -scaling computational costs, where N O and N V denote the number of correlated occupied and virtual orbitals, respectively. One of the algorithms assembles the four-center ERIs requiring N V 4 and N O 2 N V 2 -scaling memory costs in a distributed manner on a number of MPI ranks, while the other algorithm completely bypasses the assembling of quartic memory-scaling ERIs and thus largely reduces the memory demand. It is demonstrated that the former memory-expensive algorithm is faster on a few hundred cores, while the latter memory-economic algorithm shows a better strong scaling in the limit of a few thousand cores. The program is shown to exhibit a near-linear scaling, in particular for the compute-intensive triples correction step, on up to 8000 cores. The performance of the program is demonstrated via calculations involving molecules with 24–51 atoms and up to 1624 atomic basis functions. As the first application, the complete basis set (CBS) limit for the interaction energy of the π-stacked uracil dimer from the S66 data set has been investigated. This work reports the first calculation of the interaction energy at the CCSD(T)/aug-cc-pVQZ level without local orbital approximation. The CBS limit for the CCSD correlation contribution to the interaction energy was found to be -8.01 kcal/mol, which agrees very well with the value -7.99 kcal/mol reported by Schmitz, Hättig, and Tew [ Phys. Chem. Chem. Phys. 2014 , 16 , 22167-22178]. The CBS limit for the total interaction energy was estimated to be -9.64 kcal/mol.
The Importance of Three-Body Interactions in Molecular Dynamics Simulations of Water with the Fragment Molecular Orbital Method
The analytic first derivative with respect to nuclear coordinates is formulated and implemented in the framework of the three-body fragment molecular orbital (FMO) method. The gradient has been derived and implemented for restricted Hartree-Fock, second-order Møller-Plesset perturbation, and density functional theories. The importance of the three-body fully analytic gradient is illustrated through the failure of the two-body FMO method during molecular dynamics simulations of a small water cluster. The parallel implementation of the fragment molecular orbital method, its parallel efficiency, and its scalability on the Blue Gene/Q architecture up to 262,144 CPU cores, are also discussed.
Energy-Screened Many-Body Expansion for Protein–Ligand Interactions: Examining Convergence for Metalloenzymes Through Seven–Body Interactions
Fragment-based quantum chemistry is a powerful strategy for calculating protein−ligand interaction energies using quantum chemistry methods. Rigorous convergence often requires hundreds of atoms in the protein binding-site model, especially if that model is constructed using distance-based criteria to select amino acid residues, while three- and four-body calculations exhibit instability related to combinatorial proliferation in the number of subsystem calculations. Here, we report an energy-based screening protocol for the many-body expansion applied to protein−ligand interactions, implemented in the open-source FRAGME∩T code. Using a combination of aggressive screening based on semiempirical quantum chemistry, with an improved graph-theoretical algorithm to eliminate unimportant subsystems, we are able to perform n-body calculations up to n = 7 using density functional theory in triple-ζ basis sets. Distance cutoffs further reduce the cost without compromising accuracy. Rapid and stable convergence of the many-body expansion is obtained by n = 4, for a pair of metalloenzymes in which a divalent ion coordinates directly to the ligand. As compared to previous results that relied solely on distance cutoffs, oscillations in the n-body corrections are reduced or eliminated, although residual errors remain in one case. This work demonstrates that benchmark-quality protein−ligand interaction energies can be systematically converged using a method with excellent parallel efficiency and scalability.
Monte Carlo Explicitly Correlated Second-Order Many-Body Green’s Function Calculations of Semiconductor Band Gaps
A systematically converging series of ab initio, post-density-functional, size-consistent, electron-correlated approximations is desired for predictive computing of felectronic band structures of insulating, semiconducting, and metallic solids. A series that meets all of these desiderata (except the applicability to metals) is ab initio many-body Green's function theory based on Gaussian-type-orbital (GTO) basis sets. Here, its leading-order approximation, the second-order Green's function (GF2) method in the diagonal and frequency-independent approximations with the aug-cc-pVDZ basis set, is applied to the fundamental band gaps of three semiconductors (diamond, silicon, and silicon carbide in the zincblende structure) using cluster models. Corrections are made to the basis-set-incompleteness errors by the explicit-correlation (F12) ansatz (GF2-F12) for the valence band edges. The crystals are modeled as surface-passivated clusters of increasing sizes, whose wave functions are expanded by up to 2709 GTO basis functions. Immense computational costs of these calculations are overcome by the highly scalable stochastic algorithm of the Monte Carlo GF2-F12 method, whose operation cost per state increases only as a cubic power of system size, which has a tiny memory footprint and easily achieves near-perfect parallel efficiency on thousands of CPUs or on hundreds of GPUs. The correlated, F12-corrected highest-occupied and lowest-unoccupied molecular-orbital energy (HOMO-LUMO) gap is 5.78 ± 0.07 eV for C 87 H 76 as compared with the experimental value of the fundamental (indirect) band gap of bulk diamond at 5.48 eV. The correlated, F12-corrected HOMO-LUMO gaps for Si 75 H 76 and Si 32 C 43 H 76 are 2.56 ± 0.15 eV and 3.50 ± 0.12 eV, respectively, which are expected to decrease further with increasing cluster sizes. As a result, the experimental fundamental (indirect) band gaps of bulk silicon and silicon carbide are 1.17 eV and 2.42 eV, respectively.
Training data selection for accuracy and transferability of interatomic potentials
Abstract Advances in machine learning (ML) have enabled the development of interatomic potentials that promise the accuracy of first principles methods and the low-cost, parallel efficiency of empirical potentials. However, ML-based potentials struggle to achieve transferability, i.e., provide consistent accuracy across configurations that differ from those used during training. In order to realize the promise of ML-based potentials, systematic and scalable approaches to generate diverse training sets need to be developed. This work creates a diverse training set for tungsten in an automated manner using an entropy optimization approach. Subsequently, multiple polynomial and neural network potentials are trained on the entropy-optimized dataset. A corresponding set of potentials are trained on an expert-curated dataset for tungsten for comparison. The models trained to the entropy-optimized data exhibited superior transferability compared to the expert-curated models. Furthermore, the models trained to the expert-curated set exhibited a significant decrease in performance when evaluated on out-of-sample configurations.
An implicit-explicit time splitting strategy for the far SOL plasma fluid model with DG-FEM discretization
We consider a far scrape-off layer (SOL) plasma fluid model of ions that is governed by a Braginskiitype model: a one-dimensional, nonlinear system of advection-diffusion equations coupled with a diffusion equation for neutral particles. Our motivation for studying this system arises from the coupling between the edge plasma and radio-frequency (RF) heating, where solving a far SOL plasma fluid model provides critical insights into edge plasma dynamics. Numerical simulations of plasma fluid models require advanced computational techniques to achieve both efficiency and accuracy, especially when resolving the boundary layer in magnetically confined plasmas. In this work, we propose an implicit-explicit time operator splitting strategy that allows for an efficient solution algorithm, where the diffusive terms are treated semi-implicitly requiring only a linear solve, while the advection part is handled explicitly using a strong-stability-preserving Runge-Kutta (SSP-RK3) scheme. This leads to a fully decoupled system in which the diffusion and advection sub-problems can be solved separately, simplifying the overall solution procedure and allowing for efficient parallelization, which is particularly relevant for exploring the impact of RF heating on the SOL plasma. The main challenge of the discretization is due to the strong coupling between diffusion and advection, particularly through the boundary conditions. This makes implementation of such a scheme in an accurate and stable manner nontrivial. We discuss in detail how to split the equations and manage boundary conditions to maintain stability and well-posedness for each subsystem. We also describe a spatial discretization approach, based on the discontinuous Galerkin finite element method (DG-FEM) and present numerical results for a one-dimensional system.
Active-space completely-renormalized equation-of-motion coupled-cluster formalism: Excited-state studies of green fluorescent protein, free-base porphyrin, and oligoporphyrin dimer
The development of efficient parallel implementations of electronic structure methods enables not only the study of excited states of large molecular systems but also a unique opportunity to assess the role of various correlation effects in describing excitation energies for systems composed of hundreds of electrons. In this article, we discuss the impact of triply excited configurations in Equation-of-Motion Coupled Cluster (EOMCC) formalism. As a benchmark system we chose the fused porphyrin dimer, which is described by the basis set composed of 942 functions and where 270 electrons were correlated in the EOMCC calculations.
Computation of the free energy due to electron density fluctuation of a solute in solution: A QM/MM method with perturbation approach combined with a theory of solutions
We developed a perturbation approach to compute solvation free energy Δμ within the framework of QM (quantum mechanical)/MM (molecular mechanical) method combined with a theory of energy representation (QM/MM-ER). The energy shift η of the whole system due to the electronic polarization of the solute is evaluated using the second-order perturbation theory (PT2), where the electric field formed by surrounding solvent molecules is treated as the perturbation to the electronic Hamiltonian of the isolated solute. The point of our approach is that the energy shift η, thus obtained, is to be adopted for a novel energy coordinate of the distribution functions which serve as fundamental variables in the free energy functional developed in our previous work. The most time-consuming part in the QM/MM-ER simulation can be, thus, avoided without serious loss of accuracy. For our benchmark set of molecules, it is demonstrated that the PT2 approach coupled with QM/MM-ER gives hydration free energies in excellent agreements with those given by the conventional method utilizing the Kohn-Sham SCF procedure except for a few molecules in the benchmark set. A variant of the approach is also proposed to deal with such difficulties associated with the problematic systems. The present approach is also advantageous to parallel implementations. We examined the parallel efficiency of our PT2 code on multi-core processors and found that the speedup increases almost linearly with respect to the number of cores. Thus, it was demonstrated that QM/MM-ER coupled with PT2 deserves practical applications to systems of interest.