Search NASA⌕ Search

Engineering topics

Hammond, Jeff R.

Publications and source records attributed to Hammond, Jeff R..

LibERI—A portable and performant multi-GPU accelerated library for electron repulsion integrals via OpenMP offloading and standard language parallelism

A portable and performant graphics processing unit (GPU)-accelerated library for electron repulsion integral (ERI) evaluation, named LibERI, has been developed and implemented via directive-based (e.g., OpenMP and OpenACC) and standard language parallelism (e.g., Fortran DO CONCURRENT). Offloaded ERIs consist of integrals over low and high contraction s, p, and d functions using the rotated-axis and Rys quadrature methods. GPU codes are factorized based on previous developments with two layers of integral screening and quartet presorting. In this work, the density screening is moved to the GPU to enhance the computational efficacy for large molecular systems. Here, the L-shells in the Pople basis set are also separated into pure S and P shells to increase the ERI homogeneity and reduce atomic operations and the memory footprint. LibERI is compatible with any quantum chemistry drivers supporting the MolSSI Driver Interface. Benchmark calculations of LibERI interfaced with the GAMESS software package were carried out on various GPU architectures and molecular systems. The results show that the LibERI performance is comparable to other state-of-the-art GPU-accelerated codes (e.g., TeraChem and GMSHPC) and, in some cases, outperforms conventionally developed ERI CUDA kernels (e.g., QUICK) while fully maintaining portability.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NWChem: Recent and Ongoing Developments

In this paper we summarize developments in the NWChem computational chemistry suite since the last major release (NWChem 7.0). Specifically, we focus on functionalities, along with input blocks, that are currently accessible in the current stable release (NWChem 7.2) and master branches, interfaces to quantum computing simulators, interfaces to external libraries, the NWChem GitHub repository, and containerization of NWChem executable images. In conclusion, some of the ongoing developments that will be available in the near future are also discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Enabling Fortran Standard Parallelism in GAMESS for Accelerated Quantum Chemistry Calculations

The performance of Fortran 2008 DO CONCURRENT (DC) relative to OpenACC and OpenMP target offloading (OTO) with different compilers is studied for the GAMESS quantum chemistry application. Specifically, DC and OTO are used to offload the Fock build, which is a computational bottleneck in most quantum chemistry codes, to GPUs. The DC Fock build performance is studied on NVIDIA A100 and V100 accelerators and compared with the OTO versions compiled by the NVIDIA HPC, IBM XL, and Cray Fortran compilers. The results show that DC can speed up the Fock build by 3.0× compared with that of the OTO model. Finally, with similar offloading efforts, DC is a compelling programming model for offloading Fortran applications to GPUs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Transitioning NWChem to the Next Generation of Manycore Machines

The NorthWest Chemistry (NWChem) modeling software is a popular molecular chemistry simulation software that was designed from the start to work on massively parallel processing supercomputers[6, 28, 49]. It contains an umbrella of modules that today includes Self Consistent Field (SCF), second order Mller-Plesset perturbation theory (MP2), Coupled Cluster, multi-conguration selfconsistent eld (MCSCF), selected conguration interaction (CI), tensor contraction engine (TCE) many body methods, density functional theory (DFT), time-dependent density functional theory (TDDFT), real time time-dependent density functional theory, pseudopotential plane-wave density functional theory (PSPW), band structure (BAND), ab initio molecular dynamics, Car-Parrinello molecular dynamics, classical molecular dynamics (MD), QM/MM, AIMD/MM, GIAO NMR, COSMO, COSMO-SMD, and RISM solvation models, free energy simulations, reaction path optimization, parallel in time, among other capabilities[ 22]. Moreover new capabilities continue to be added with each new release.

Environmental Molecular Sciences Laboratory↗

Performance Evaluation of NWChem Ab-Initio Molecular Dynamics (AIMD) Simulations on the Intel® Xeon Phi™ Processor

Ab-initio Molecular Dynamics (AIMD) methods are an important class of algorithms, as they enable scientists to understand the chemistry and dynamics of molecular and condensed phase systems while retaining a first-principles-based description of their interactions. Many-core architectures such as the Intel® Xeon Phi™ processor are an interesting and promising target for these algorithms, as they can provide the computational power that is needed to solve interesting problems in chemistry. In this paper, we describe the efforts of refactoring the existing AIMD plane-wave method of NWChem from an MPI-only implementation to a scalable, hybrid code that employs MPI and OpenMP to exploit the capabilities of current and future many-core architectures. We describe the optimizations required to get close to optimal performance for the multiplication of the tall-and-skinny matrices that form the core of the computational algorithm. We present strong scaling results on the complete AIMD simulation for a test case that simulates 256 water molecules and that strong-scales well on a cluster of 1024 nodes of Intel Xeon Phi processors. We compare the performance obtained with a cluster of dual-socket Intel® Xeon® E5–2698v3 processors.

Environmental Molecular Sciences Laboratory↗

Coupled Cluster Studies of Ionization Potentials and Electron Affinities of Single-Walled Carbon Nanotubes

In this paper we apply equation-of-motion coupled cluster (EOMCC) methods in studies of vertical ionization potentials (IP) and electron affinities (EA) for sin- gled walled carbon nanotubes. EOMCC formulations for ionization potentials and electron affinities employing excitation manifolds spanned by single and double ex- citations (IP/EA-EOMCCSD) are used to study IPs and EAs of nanotubes as a function of nanotube length. Several armchair nanotubes corresponding to C20nH20 models with n = 2 - 6 have been used in benchmark calculations. In agreement with previous studies, we demonstrate that the electronegativity of C20nH20 systems remains, to a large extent, independent of nanotube length. We also compare IP/EA- EOMCCSD results with those obtained with the coupled cluster models with single and double excitations corrected by perturbative triples, CCSD(T), and density func- tional theory (DFT) using global and range-separated hybrid exchange-correlation functionals.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Hartree-Fock Application Using UPC++ and the New DArray Library

The Hartree-Fock (HF) method is the fundamental first step for incorporating quantum mechanics into many-electron simulations of atoms and molecules, and it is an important component of computational chemistry toolkits like NWChem. The GTFock code is an HF implementation that, while it does not have all the features in NWChem, represents crucial algorithmic advances that reduce communication and improve load balance by doing an up-front static partitioning of tasks, followed by work stealing whenever necessary. To enable innovations in algorithms and exploit next generation exascale systems, it is crucial to support quantum chemistry codes using expressive and convenient programming models and runtime systems that are also efficient and scalable. Here, this paper presents an HF implementation similar to GTFock using UPC++, a partitioned global address space model that includes flexible communication, asynchronous remote computation, and a powerful multidimensional array library. UPC++ offers runtime features that are useful for HF such as active messages, a rich calculus for array operations, hardware-supported fetch-and-add, and functions for ensuring asynchronous runtime progress. We present a new distributed array abstraction, DArray, that is convenient for the kinds of random-access array updates and linear algebra operations on block-distributed arrays with irregular data ownership. Finally, we analyze the performance of atomic fetch-and-add operations (relevant for load balancing) and runtime attentiveness, then compare various techniques and optimizations for each. Our optimized implementation of HF using UPC++ and the DArrays library shows up to 20% improvement over GTFock with Global Arrays at scales up to 24,000 cores.

97 MATHEMATICS AND COMPUTING↗

Theoretical studies of the global minima and polarizabilities of small lithium clusters

Lithium clusters Lin (n=1-20) have been investigated with density functional theory (DFT) and coupled—cluster (CC) methods. The global-minimum structures are located via an improved basin---hopping algorithm and the lowest energy Lin isomers are confirmed with DFT geometry optimizations, CCSD(T) energy calculations, and by comparing simulated and experimental polarizabilities. The tetrahedral Li4 structure is found to be the basic building block of lithium clusters Lin (n=6-20). Simulated polarizabilities, including thermal effects at room temperature, are in good agreement with measured isotropic polarizabilities.

clusters↗

MADNESS: A Multiresolution, Adaptive Numerical Environment for Scientific Simulation

MADNESS (multiresolution adaptive numerical environment for scientific simulation) is a high-level software environment for solving integral and differential equations in many dimensions that uses adaptive and fast harmonic analysis methods with guaranteed precision based on multiresolution analysis and separated representations. Underpinning the numerical capabilities is a powerful petascale parallel programming environment that aims to increase both programmer productivity and code scalability. This paper describes the features and capabilities of MADNESS and briefly discusses some current applications in chemistry and several areas of physics.

00A72↗

Improving concurrency and asynchrony in multithreaded MPI applications using software offloading

We present a new approach for multithreaded communication and asynchronous progress in MPI applications, wherein we offload communication processing to a dedicated thread. The central premise is that given the rapidly increasing core counts on modern systems, the improvements in MPI performance arising from dedicating a thread to drive communication outweigh the small loss of resources for application computation, particularly when overlap of communication and computation can be exploited. Our approach allows application threads to make MPI calls concurrently, enqueuing these as communication tasks to be processed by a dedicated communication thread. This not only guarantees progress for such communication operations, but also reduces load imbalance. Our implementation additionally significantly reduces the overhead of mutual exclusion seen in existing implementations for applications using MPI THREAD MULTIPLE. Our technique requires no modification to the application, and we demonstrate significant performance improvement (up to 2X) for QCD, 1-D FFT and deep learning CNN applications.

Vaidyanathan, Karthikeyan↗

Parallel scalability of Hartree–Fock calculations

Quantum chemistry is increasingly performed using large cluster computers consisting of multiple interconnected nodes. For a fixed molecular problem, the efficiency of a calculation usually decreases as more nodes are used, due to the cost of communication between the nodes. This paper empirically investigates the parallel scalability of Hartree–Fock calculations. The construction of the Fock matrix and the density matrix calculation are analyzed separately. For the former, we use a parallelization of Fock matrix construction based on a static partitioning of work followed by a work stealing phase. For the latter, we use density matrix purification from the linear scaling methods literature, but without using sparsity. When using large numbers of nodes for moderately sized problems, density matrix computations are network-bandwidth bound, making purification methods potentially faster than eigendecomposition methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NWChem: Quantum Chemistry Simulations at Scale

Methods based on quantum mechanics equations have been developed since the 1930's with the purpose of accurately studying the electronic structure of molecules. However, it is only during the last two decades that intense development of new computational algorithms has opened the possibility of performing accurate simulations of challenging molecular processes with high-order many-body methods. A wealth of evidence indicates that the proper inclusion of instantaneous interactions between electrons (or the so-called electron correlation effects) is indispensable for the accurate characterization of chemical reactivity, molecular properties, and interactions of light with matter. The availability of reliable methods for benchmarking of medium-size molecular systems provides also a unique chance to propagate high-level accuracy across spatial scales through the multiscale methodologies. Some of these methods have potential to utilize computational resources in an effi*cient way since they are characterized by high numerical complexity and appropriate level of data granularity, which can be effi*ciently distributed over multi-processor architectures. The broad spectrum of coupled cluster (CC) methods falls into this class of methodologies. Several recent CC implementations clearly demonstrated the scalability of CC formalisms on architectures composed of hundreds thousand computational cores. In this context NWChem provides a collection of Tensor Contraction Engine (TCE) generated parallel implementations of various coupled cluster methods capable of taking advantage of many thousand of cores on leadership class parallel architectures.

chemistry↗

Accurate Numerical Simulations Of Chemical Phenomena Involved in Energy Production and Storage with MADNESS and MPQC: ALCF-2 Early Science Program Technical Report

In order to solve the electronic structure of large molecular systems on petascale computers using MADNESS, a numerical tool kit, are required fast and accurate implementations for linear algebra. MADNESS uses multiresolution analysis and low separation rank which translates high dimensional functions in tensor products using Legendre polynomial. The multiple tensor products make to the singular value decomposition and matrix multiplication the most intense operations in MADNESS. This work discusses the interfacing of Eigen3 as a C++ substitute of LAPACK and introduces Elemental for the diagonalization of large matrices. Furthermore, the present paper shows the performance these libraries on Blue Gene/ Q.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Coupled Cluster Theory on Graphics Processing Units I. The Coupled Cluster Doubles Method

The coupled cluster (CC) ansatz is generally recognized as providing one of the best wave function-based descriptions of electronic correlation in small- and medium-sized molecules. The fact that the CC equations with double excitations (CCD) may be expressed as a handful of dense matrix-matrix multiplications makes it an ideal method to be ported to graphics processing units (GPUs). Here we present our implementation of the spin-free CCD equations in which the entire iterative procedure is evaluated on the GPU. The GPU-accelerated algorithm readily achieves a factor of 4-5 speedup relative to the multithreaded CPU algorithm on same-generation hardware. The GPU-accelerated algorithm is approximately 8-12 times faster than Molpro, 17-22 times faster than NWChem, and 21-29 times faster than GAMESS for each CC iteration. Single-precision GPU-accelerated computations are also performed, leading to an additional doubling of performance. Single-precision errors in the energy are typically on the order of 10 -6 hartrees and can be improved by about an order of magnitude by performing one additional iteration in double precision.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Predicted thermochemistry for chemical conversions of 5-hydroxymethylfurfural.

The thermochemistry of various chemical transformations of 5-hydroxy methyl furfural (HMF) were investigated using highly accurate Gaussian-4 (G4) theory. The conversion of HMF to nonane through aldol condensation, hydrogenation, and hydrogenolysis reactions was found to be thermodynamically favorable. The hydrogenation reactions involving the keto groups in the nonane reaction sequence were found to be enhanced at low temperatures and high pressures of H{sub 2}. The hydrogenation, selective oxidation, and hydration of HMF were also found to be thermodynamically favorable. Gas phase enthalpies of formation of all the intermediate compounds were calculated at the G4 level of theory and compared against existing experimental data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Computational Studies of the Thermochemistry for Conversion of Glucose to Levulinic Acid

The thermochemistry of the conversion of glucose to levulinic acid through fructofuranosyl intermediates is investigated using the high-level ab initio methods G4 and G4MP2. The calculated gas phase reaction enthalpies indicate that the first two steps involving water molecule elimination are highly endothermic, while the other steps, including additional water elimination and rehydration to form levulinic acid, are exothermic. The calculated gas phase free energies indicate that inclusion of entropic effects makes the dehydration steps more favorable, although the elimination of the first water is still endothermic. Elevated temperatures and aqueous reaction environments are also predicted to make the dehydration reaction steps thermodynamically more favorable. On the basis of these enthalpy and free energy calculations, the first dehydration step in conversion of glucose to levulinic acid is likely a key step in controlling the overall progress of the reaction. An assessment of density functional theories and other theoretical methods for the calculation of the dehydration and hydration reactions in the decomposition of glucose is also presented.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗