Search NASA⌕ Search

SEARCH · Search NASA

Results for “OpenMP parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

A Massively Parallel Adaptive Fast Multipole Method on Heterogeneous Architectures

We describe a parallel fast multipole method (FMM) for highly nonuniform distributions of particles. We employ both distributed memory parallelism (via MPI) and shared memory parallelism (via OpenMP and GPU acceleration) to rapidly evaluate two-body nonoscillatory potentials in three dimensions on heterogeneous high performance computing architectures. We have performed scalability tests with up to 30 billion particles on 196,608 cores on the AMD/CRAY-based Jaguar system at ORNL. On a GPU-enabled system (NSF's Keeneland at Georgia Tech/ORNL), we observed 30x speedup over a single core CPU and 7x speedup over a multicore CPU implementation. By combining GPUs with MPI, we achieve less than 10 ns/particle and six digits of accuracy for a run with 48 million nonuniformly distributed particles on 192 GPUs.

99 GENERAL AND MISCELLANEOUS↗

An efficient MPI/OpenMP parallelization of the Hartree–Fock–Roothaan method for the first generation of Intel® Xeon Phi™ processor architecture

The Hartree-Fock (HF) method in the quantum chemistry package GAMESS represents one of the most irregular algorithms in computation today. Major steps in the calculation are the irregular computation of electron repulsion integrals (ERIs) and the building of the Fock matrix. These are the central components of the main Self Consistent Field (SCF) loop, the key hotspot in Electronic Structure (ES) codes. By threading the MPI ranks in the official release of the GAMESS code, we not only speed up the main SCF loop (4x to 6x for large systems), but also achieve a significant (>2x) reduction in the overall memory footprint. These improvements are a direct consequence of memory access optimizations within the MPI ranks. We benchmark our implementation against the official release of the GAMESS code on the Intel R Xeon PhiTM supercomputer. Here, scaling numbers are reported on up to 7,680 cores on Intel Xeon Phi coprocessors.

97 MATHEMATICS AND COMPUTING↗

Direct NeTS sampling of nuclear graphite $S(α, β, T)$ in Serpent

For advanced reactor applications, Neural Thermal Scattering (NeTS) modules were developed to predict the thermal scattering law (TSL or $S(α, β, T)$) of a nuclear graphite neutron moderator. NeTS are multi-layer, feedforward artificial neural networks, which act as universal function approximators designed for TSL datasets. In this case, a 4-layer neural network with 164 neurons per layer is trained using FLASSH evaluated data in PyTorch and serialized as a torchscript dictionary to predict $S(α, β, T)$ on-the-fly. Relative, absolute and maximum percent deviations of NeTS from File 7 data generated using the FLASSH code are on the order of 0.01%, 0.1% and 1%, respectively, with low inference latencies of 0.000172 s per $S(α, β, T)$ at a given temperature. Capturing the full dimensionality of possible inelastic neutron-lattice interactions, NeTS functionality is embedded in the Serpent Monte Carlo code, where $S(α, β, T)_{NeTS}$ sampling is conducted on-the-fly and compared to ACE look-up-tables for predicting TREAT criticality. k-eff differences between sampling algorithms of 6 pcm are observed and are within the order of Monte Carlo uncertainty. Compared to discrete and continuous-energy ACE files (30 MB and 131 MB per temperature), the NeTS format is on the order of 200–300 kB for a continuous-temperature, interpolation-free representation of $S(α, β, T)$ and cross sections. NeTS-in-Serpent runtimes comparable with ACE look-up tables are achieved by scaling NeTS for high performance computing architectures with hybrid OpenMP + MPI parallelization. This work validates a novel, self-contained reactor physics framework for predictive cross sections, and demonstrates a general methodology for embedding modern machine learning libraries within existing neutronic analysis frameworks.

Nuclear Criticality Safety Program (NCSP)↗

Parallel kinetic Monte Carlo simulation framework incorporating accurate models of adsorbate lateral interactions

Ab initio kinetic Monte Carlo (KMC) simulations have been successfully applied for over two decades to elucidate the underlying physico-chemical phenomena on the surfaces of heterogeneous catalysts. These simulations necessitate detailed knowledge of the kinetics of elementary reactions constituting the reaction mechanism, and the energetics of the species participating in the chemistry. The information about the energetics is encoded in the formation energies of gas and surface-bound species, and the lateral interactions between adsorbates on the catalytic surface, which can be modeled at different levels of detail. The majority of previous works accounted for only pairwise-additive first nearest-neighbor interactions. More recently, cluster-expansion Hamiltonians incorporating long-range interactions and many-body terms have been used for detailed estimations of catalytic rate [C. Wu, D. J. Schmidt, C. Wolverton, and W. F. Schneider, J. Catal. 286, 88 (2012)]. In view of the increasing interest in accurate predictions of catalytic performance, there is a need for general-purpose KMC approaches incorporating detailed cluster expansion models for the adlayer energetics. We have addressed this need by building on the previously introduced graph-theoretical KMC framework, and we have developed Zacros, a FORTRAN2003 KMC package for simulating catalytic chemistries. To tackle the high computational cost in the presence of long-range interactions we introduce parallelization with OpenMP. We further benchmark our framework by simulating a KMC analogue of the NO oxidation system established by Schneider and co-workers [J. Catal. 286, 88 (2012)]. We show that taking into account only first nearest-neighbor interactions may lead to large errors in the prediction of the catalytic rate, whereas for accurate estimates thereof, one needs to include long-range terms in the cluster expansion.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Phoebe: a high-performance framework for solving phonon and electron Boltzmann transport equations

Understanding the electrical and thermal transport properties of materials is critical to the design of electronics, sensors, and energy conversion devices. Computational modeling can accurately predict material properties but, in order to be reliable, requires accurate descriptions of electron and phonon states and their interactions. While first-principles methods are capable of describing the energy spectrum of each carrier, using them to compute transport properties is still a formidable task, both computationally demanding and memory intensive, requiring integration of fine microscopic scattering details for estimation of macroscopic transport properties. To address this challenge, we present Phoebe—a newly developed software package that includes the effects of electron–phonon, phonon–phonon, boundary, and isotope scattering in computations of electrical and thermal transport properties of materials with a variety of available methods and approximations. This open source C++ code combines MPI-OpenMP hybrid parallelization with GPU acceleration and distributed memory structures to manage computational cost, allowing Phoebe to effectively take advantage of contemporary computing infrastructures. We demonstrate that Phoebe accurately and efficiently predicts a wide range of transport properties, opening avenues for accelerated computational analysis of complex crystals.

36 MATERIALS SCIENCE↗

Optimization of High-order Wave Equations for Multicore CPUs

This is a simple benchmark to gauge the performance of a high-order isotropic wave equation grid. The code is optimized for both SSE and AVX and is parallelized using OpenMP (see Optimization section). Structurally, the benchmark begins, reads a few command-line parameters, allocates and pads the four arrays (current, last, next wave fields, and the spatially varying but isotropic velocity), initializes these arrays, then runs the benchmark proper. The code then benchmarks the naïve, SSE (if supported), and AVX (if supported implementations) by applying the wave equation stencil 100 times and taking the average performance. Boundary conditions are ignored and would nominally be implemented by the user. Thus, the benchmark measures only the performance of the wave equation stencil and not a full simulation. The naïve implementation is a quadruply (z,y,x, radius) nested loop that can handle arbitrarily order wave equations. The optimized (SSE/AVX) implentations are somewhat more complex as they operate on slabs and include a case statement to select an optimized inner loop depending on wave equation order.

Williams, Samuel↗

High-performance epistasis detection in quantitative trait GWAS

epiSNP is a program for identifying pairwise single nucleotide polymorphism (SNP) interactions (epistasis) in quantitative-trait genome-wide association studies (GWAS). A parallel MPI version (EPISNPmpi) was created in 2008 to address this computationally expensive analysis on large data sets with many quantitative traits and SNP markers. However, the falling cost of genotyping has led to an explosion of large-scale GWAS data sets that challenge EPISNPmpi’s ability to compute results in a reasonable amount of time. Therefore, we optimized epiSNP for modern multi-core and highly parallel many-core processors to efficiently handle these large data sets. This paper describes the serial optimizations, dynamic load balancing using MPI-3 RMA operations, and shared-memory parallelization with OpenMP to further enhance load balancing and allow execution on the Intel Xeon Phi coprocessor (MIC). For a large GWAS data set, our optimizations provided a 38.43× speedup over EPISNPmpi on 126 nodes using 2 MICs on TACC’s Stampede Supercomputer. We also describe a Coarray Fortran (CAF) version that demonstrates the suitability of PGAS languages for problems with this computational pattern. We show that the Coarray version performs competitively with the MPI version on the NERSC Edison Cray XC30 supercomputer. Finally, the performance benefits of hyper-threading for this application on Edison (average 1.35× speedup) are demonstrated.

97 MATHEMATICS AND COMPUTING↗

Integration of Heat Transfer, Stress, and Particle Trajectory Simulation

Calabazas Creek Research, Inc. developed and currently markets Beam Optics Analyzer (BOA) in the United States and abroad. BOA is a 3D, charged particle optics code that solves the electric and magnetic fields with and without the presence of particles. It includes automatic and adaptive meshing to resolve spatial scales ranging from a few millimeters to meters. It is fully integrated with CAD packages, such as SolidWorks, allowing seamless geometry updates. The code includes iterative procedures for optimization, including a fully functional, graphical user interface. Recently, time dependent, particle in cell capability was added, pushing particles synchronically under quasistatic electromagnetic fields to obtain particle bunching under RF conditions. A heat transfer solver was added during this Phase I program. Completed tasks include: (1) Added a 3D finite element heat transfer solver with adaptivity; (2) Determined the accuracy of the linear heat transfer field solver to provide the basis for development of higher order solvers in Phase II; (3) Provided more accurate and smoother power density fields; and (4) Defined the geometry using the same CAD model, while maintaining different meshes, and interfacing the power density field between the particle simulator and heat transfer solvers. These objectives were achieved using modern programming techniques and algorithms. All programming was in C++ and parallelization in OpenMP, utilizing state-of-the-art multi-core technology. Both x86 and x64 versions are supported. The GUI design and implementation used Microsoft Foundation Class.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Monte Carlo based investigation of Berry phase for depth resolved characterization of biomedical scattering samples

The propagation of light in turbid media is an active area of research with relevance to numerous investigational fields, e.g., biomedical diagnostics and therapeutics. The statistical random-walk nature of photon propagation through turbid media is ideal for computational based modeling and simulation. Ready access to super computing resources provide a means for attaining brute force solutions to stochastic light-matter interactions entailing scattering by facilitating timely propagation of sufficient (>10million) photons while tracking characteristic parameters based on the incorporated physics of the problem. One such model that works well for isotropic but fails for anisotropic scatter, which is the case for many biomedical sample scattering problems, is the diffusion approximation. In this report, we address this by utilizing Berry phase (BP) evolution as a means for capturing anisotropic scattering characteristics of samples in the preceding depth where the diffusion approximation fails. We extend the polarization sensitive Monte Carlo method of Ramella-Roman, et al.,1 to include the computationally intensive tracking of photon trajectory in addition to polarization state at every scattering event. To speed-up the computations, which entail the appropriate rotations of reference frames, the code was parallelized using OpenMP. The results presented reveal that BP is strongly correlated to the photon penetration depth, thus potentiating the possibility of polarimetric depth resolved characterization of highly scattering samples, e.g., biological tissues.

back scattered Mueller matrix↗

MOCUM: A two-dimensional method of characteristics code based on unstructured meshing for general geometries

A transport theory code MOCUM based on the Method of Characteristics (MOC) as the flux solver with an advanced general geometry processor is developed for two-dimensional lattice and full core neutronics modeling. The core structure is represented by Constructive Solid Geometry (CSG) that uses Boolean operations to build complex geometries from simple polygons. Arbitrary-precision arithmetic is also used in the process of building CSG objects to eliminate the round-off error from the commonly used double precision numbers. Then, the constructed core frame will be decomposed and refined into a conforming Delaunay triangulation to ensure the quality of the meshes. The MOC solver kernel is fully paralleled using OpenMP and the developed numerical code is validated by several benchmarks representing various core geometries. For the cases modeled, the maximum percentage error for multiplication factor and the pin power compared to reference values are 0.1% and 0.7% respectively. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

The Kokkos OpenMPTarget Backend: Implementation and Lessons Learned

As the supercomputing landscape diversifies, solutions such as Kokkos to write vendor agnostic applications and libraries have risen in popularity. Kokkos provides a programming model designed for performance portability, which allows developers to write a single source implementation that can run efficiently on various architectures. At its heart, Kokkos maps parallel algorithms to architecture and vendor specific backends written in lower level programming models such as CUDA and HIP. Another approach to writing vendor agnostic parallel code is using OpenMP’s directives based approach, which lets developers annotate code to express parallelism. It is implemented at the compiler level and is supported by all major high performance computing vendors, as well as the primary Open Source toolchains GNU and LLVM. Since its inception, Kokkos has used OpenMP to parallelize on CPU architectures. In this paper, we explore leveraging OpenMP for a GPU backend and discuss the challenges we encountered when mapping the Kokkos APIs and semantics to OpenMP target constructs. As an exemplar workload we chose a simple conjugate gradient solver for sparse matrices. We find that performance on NVIDIA and AMD GPUs varies widely based on details of the implementation strategy and the chosen compiler. Furthermore, the performance of the OpenMP implementations decreases with increasing complexity of the investigated algorithms.

Gayatri, Rahulkumar↗

Exascale Programming Approaches for Accelerated Climate Modeling for Energy

The Accelerated Climate Modeling for Energy (ACME) program is a leading-edge climate and Earth system model designed to address the U.S. Department of Energy (DOE) mission needs. The ACME project involves collaboration between seven National Laboratories, the National Center for Atmospheric Research, four academic institutions, and one private-sector company. The ACME project funds ongoing efforts to ensure the model continues to make efficient use of DOE Leadership Computing Facilities (LCFs) in order to best address the three driving grand challenge science questions:Water cycle: How will more realistic portrayals of features important to the water cycle (resolution, clouds, aerosols, snowpack, river routing, land use) affect river flow and associated freshwater supplies at the watershed scale?Cryosphere systems: Could a dynamical instability in the Antarctic ice sheet be triggered within the next 40 years?Biogeochemistry: What are the contributions and feedbacks from natural and managed systems to current greenhouse gas fluxes, and how will those factors and associated fluxes evolve in the future?

Norman, Matthew↗

OpenMP in VASP: Threading and SIMD

Abstract The Vienna Ab initio Simulation Package (VASP) is a widely used electronic structure code that originally exploits process‐level parallelism through the Message Passing Interface (MPI) for work distribution within and across nodes. Architectural changes of modern parallel processors urge programmers to address thread‐ and data‐level parallelism as well to benefit most from the available compute resources within a node. We describe for VASP how to approach for an MPI + OpenMP parallelization including data‐level parallelism through OpenMP SIMD constructs together with a generic high‐level vector coding scheme. We can demonstrate an improved scalability of VASP and more than 20% gain over the MPI‐only version as well as a 2× increased performance of collective operations using the multiple‐endpoint MPI feature. The high‐level vector coding scheme applied to VASP's general gradient approximation routine gives up 9× performance gain on AVX512 platforms with the Intel compiler.

Wende, Florian↗

Scheduling and Performance of Asynchronous Tasks in Fortran 2018 with FEATS

Most parallel scientific programs contain compiler directives (pragmas) such as those from OpenMP (Hermanns in Parallel programming in Fortran 95 using openMP, 2002. School of Aeronautical Engineering, Universidad Politécnica de Madrid, España, 2011), explicit calls to runtime library procedures such as those implementing the Message Passing Interface (MPI) (in A message-passing interface standard version 4.0, 2021. https://www.mpi-forum.org/docs/mpi-4.0/mpi40-report.pdf), or compiler-specific language extensions such as those provided by CUDA (Ruetsch and Fatica in CUDA Fortran for scientists and engineers: best practices for efficient CUDA Fortran programming, Elsevier, 2013). By contrast, the recent Fortran standards empower developers to express parallel algorithms without directly referencing lower-level parallel programming models (Numrich in Parallel programming with co-arrays, CRC Press, 2018, and Curcic in Modern Fortran: building efficient parallel applications, Manning Publications, 2020). Fortran’s parallel features place the language within the Partitioned Global Address Space (PGAS) class of programming models. When writing programs that exploit data parallelism, application developers often find it straightforward to develop custom parallel algorithms. Problems involving complex, heterogeneous, staged calculations, however, pose much greater challenges. Such applications require careful coordination of tasks in a manner that respects dependencies prescribed by a directed acyclic graph. When rolling one’s own solution proves difficult, extending a customizable framework becomes attractive. Further, the paper presents the design, implementation, and use of the Framework for Extensible Asynchronous Task Scheduling (FEATS), which we believe to be the first task scheduling tool written in modern Fortran. We describe the benefits and compromises associated with choosing Fortran as the implementation language, and we propose ways in which future Fortran standards can best support the use case in this paper.

97 MATHEMATICS AND COMPUTING↗

Extending Automatic Parallelization to Optimize High-Level Abstractions for Multicore

Automatic introduction of OpenMP for sequential applications has attracted significant attention recently because of the proliferation of multicore processors and the simplicity of using OpenMP to express parallelism for shared-memory systems. However, most previous research has only focused on C and Fortran applications operating on primitive data types. C++ applications using high-level abstractions, such as STL containers and complex user-defined types, are largely ignored due to the lack of research compilers that are readily able to recognize high-level object-oriented abstractions and leverage their associated semantics. In this paper, we automatically parallelize C++ applications using ROSE, a multiple-language source-to-source compiler infrastructure which preserves the high-level abstractions and gives us access to their semantics. Several representative parallelization candidate kernels are used to explore semantic-aware parallelization strategies for high-level abstractions, combined with extended compiler analyses. Those kernels include an array-base computation loop, a loop with task-level parallelism, and a domain-specific tree traversal. Our work extends the applicability of automatic parallelization to modern applications using high-level abstractions and exposes more opportunities to take advantage of multicore processors.

99 GENERAL AND MISCELLANEOUS↗

Semantic-Aware Automatic Parallelization of Modern Applications Using High-Level Abstractions

Automatic introduction of OpenMP for sequential applications has attracted significant attention recently because of the proliferation of multicore processors and the simplicity of using OpenMP to express parallelism for shared-memory systems. However, most previous research has only focused on C and Fortran applications operating on primitive data types. Modern applications using high-level abstractions, such as C++ STL containers and complex user-defined class types, are largely ignored due to the lack of research compilers that are readily able to recognize high-level object-oriented abstractions and leverage their associated semantics. In this paper, we use a source-to-source compiler infrastructure, ROSE, to explore compiler techniques to recognize high-level abstractions and to exploit their semantics for automatic parallelization. Several representative parallelization candidate kernels are used to study semantic-aware parallelization strategies for high-level abstractions, combined with extended compiler analyses. Preliminary results have shown that semantics of abstractions can help extend the applicability of automatic parallelization to modern applications and expose more opportunities to take advantage of multicore processors.

99 GENERAL AND MISCELLANEOUS↗