Tuning parallel data compression and I/O for large-scale earthquake simulation
Explore the source record for details and available documents.
Mathematics and Computing, Computer science↗
SEARCH · Search NASA
Results for “parallel computations”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Porting HEP Parameterized Calorimeter Simulation Code to GPUs
The High Energy Physics (HEP) experiments, such as those at the Large Hadron Collider (LHC), traditionally consume large amounts of CPU cycles for detector simulations and data analysis, but rarely use compute accelerators such as GPUs. As the LHC is upgraded to allow for higher luminosity, resulting in much higher data rates, purely relying on CPUs may not provide enough computing power to support the simulation and data analysis needs. As a proof of concept, we investigate the feasibility of porting a HEP parameterized calorimeter simulation code to GPUs. We have chosen to use FastCaloSim, the ATLAS fast parametrized calorimeter simulation. While FastCaloSim is sufficiently fast such that it does not impose a bottleneck in detector simulations overall, significant speed-ups in the processing of large samples can be achieved from GPU parallelization at both the particle (intra-event) and event levels; this is especially beneficial in conditions expected at the high-luminosity LHC, where extremely high per-event particle multiplicities will result from the many simultaneous proton-proton collisions. We report our experience with porting FastCaloSim to NVIDIA GPUs using CUDA. A preliminary Kokkos implementation of FastCaloSim for portability to other parallel architectures is also described.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Development of a Library for Conducting Monte Carlo Tallies on Heterogeneous Systems
A library is introduced that enables users to conduct Monte Carlo tallies on heterogeneous computing systems. The library can be built with or linked to a Monte Carlo transport code to provide tally capabilities for the project. The library provides a set of tools and an interface for constructing tallies that can be computed on CPUs or GPUs regardless of where the sample data was generated. The library can be incorporated into existing Monte Carlo transport codes at both the research and production level with minimal intrusion into the transport code. The tally library is capable of computing a variety of tallies including track-length histogram (mesh) tallies, collisionbased kernel density estimators, functional expansion tallies, eigenvalue sensitivity coefficients via iterated fission probability, and sensitivities of reaction rates or adjoint-weighted quantities via differential operator sampling. The parallelization methods are discussed and speedups are shown for 2-D and 3-D problems with comparisons to both the tally library CPU implementations as well as the tally capabilities in the host transport code. Performance results and examples are shown for MCNP, MCATK, and OpenMC.
97 MATHEMATICS AND COMPUTING↗
Angle dependence of strong-field single and double ionization of carbonyl sulfide
We have studied, experimentally and theoretically, the ionization probability of carbonyl sulfide (OCS) molecules in intense linearly polarized 800-nm laser pulses as a function of the angle between the molecular axis and the laser polarization. Experimentally, the molecules are exposed to two laser pulses with a relative time delay. The first, weaker pulse induces a nuclear rotational wave packet within each molecule such that the ensemble exhibits preferential alignment in the laboratory frame at specific times. The second, stronger pulse induces ionization, and the variation in single and double ionization yields is measured as a function of the delay between the two pulses. The angular dependence of the ionization yield is extracted by fitting the delay-dependent yields to a sum of delay-dependent moments of the rotational wave packet's angular distribution. We compute these same angular-dependent strong-field ionization yields for OCS using time-dependent density-functional theory (TDDFT). For the single-ionization case, our measurements agree well with TDDFT calculations and with previous experiments. Furthermore, analysis of the simulated one-body density reveals that, when averaged over a laser cycle, the resulting hole is delocalized across the molecule for light polarized perpendicular to the molecular axis and mostly localized on the sulfur for parallel polarization. Furthermore, this suggests that preferential molecular alignment is a key parameter for controlling charge migration dynamics initiated by strong-field ionization. For double ionization, the agreement between experiment and theory is less compelling, reflecting the substantial challenges of computing double-ionization yields using TDDFT methods.
74 ATOMIC AND MOLECULAR PHYSICS↗
A mesoscale 3D model of irradiated concrete informed via a 2.5 U-Net semantic segmentation
The concrete biological shield in light-water reactors is exposed to neutron and gamma irradiation, which deteriorates the concrete’s mechanical properties in the long term. To assess the irradiation-induced damage, predictive mechanical models are developed and used in parallel with the characterization of irradiated concrete samples. Realistic 3D simulation domains can drastically improve a model’s prediction. In this work, we utilized x-ray computed tomography (XCT) data of a concrete specimen to reconstruct its 3D microstructure. The XCT data shows low contrast between the concrete’s aggregates and cement paste, resulting in poor image segmentation when using traditional unsupervised techniques. To address this issue, we developed and trained a 2.5D U-Net model on only 24 pre-labeled XCT layers to segment 651 layers of the XCT data. The overall F1-score of the model is approximately 96%. Then, we created a 3D finite element (FE) mesh based on the stack of segmented images. The FE model contains radiation-induced expansion, damage, and creep. The constitutive equations are adapted to each phase (aggregates and cement paste). Here, we simulated the effects of neutron irradiation in the concrete specimen as well as the specimen’s mechanical response to uniaxial compression. Finally, model validation was performed using experimental data on similar concrete specimens in the literature.
2.5D U-Net↗
A family of independent Variable Eddington Factor methods with efficient preconditioned iterative solvers
We present a family of discretizations for the Variable Eddington Factor (VEF) equations that have high-order accuracy on curved meshes and efficient preconditioned iterative solvers. The VEF discretizations are combined with the Discontinuous Galerkin transport discretization from to form effective high-order, linear transport methods. The VEF discretizations are derived by extending the unified analysis of Discontinuous Galerkin methods for elliptic problems presented by Arnold et al. to the VEF equations. This framework is used to define analogs of the interior penalty, second method of Bassi and Rebay, minimal dissipation local Discontinuous Galerkin, and continuous finite element methods. The analysis of subspace correction preconditioners, which use a continuous operator to iteratively precondition the discontinuous discretization, is extended to the case of the non-symmetric VEF system. Numerical results demonstrate that the VEF discretizations have arbitrary-order accuracy on curved meshes, preserve the thick diffusion limit, and are effective on a proxy problem from thermal radiative transfer in both outer transport iterations and inner preconditioned linear solver iterations. We demonstrate that the VEF solution converges to the S N transport solution as the mesh is refined on both problems with smooth and non-smooth behavior in angle. Parallel performance studies show that the interior penalty VEF discretization's linear solve weak scales out to 1024 processors and strong scales well on a single node. Particular attention is paid to the parallel performance of the VEF algorithm when used in combination with a parallel block Jacobi transport sweep.
97 MATHEMATICS AND COMPUTING↗
MetaStrider: Architectures for Scalable Memory-centric Reduction of Sparse Data Streams
Reduction is an operation performed on the values of two or more key-value pairs that share the same key. Reduction of sparse data streams finds application in a wide variety of domains such as data and graph analytics, cybersecurity, machine learning, and HPC applications. However, these applications exhibit low locality of reference, rendering traditional architectures and data representations inefficient. This article presents MetaStrider, a significant algorithmic and architectural enhancement to the state-of-the-art, SuperStrider. Furthermore, these enhancements enable a variety of parallel, memory-centric architectures that we propose, resulting in demonstrated performance that scales near-linearly with available memory-level parallelism.
97 MATHEMATICS AND COMPUTING↗
Quantum Dynamics at Scale: Ultrafast Control of Emergent Functional Materials
Confluence of extreme-scale quantum dynamics simulations (i.e. quantum@scale) and cutting-edge x-ray free-electron laser experiments are revolutionizing materials science. An archetypal example is the exciting concept of using picosecond light pulses to control emergent material properties on demand in atomically-thin layered materials. This paper describes efforts to scale our quantum molecular dynamics engine toward the United States' first exaflop/s computer, under an Aurora Early Science Program project named "Metascalable layered material genome". Key algorithmic and computing techniques incorporated are: (1) globally-scalable and locally-fast solvers within a linear-scaling divide-conquer-recombine algorithmic framework; (2) algebraic `BLASification' of computational kernels; and (3) data alignment and loop restructuring, along with register and cache blocking, for enhanced vectorization and efficient memory access. The resulting weak-scaling parallel efficiency was 0.93 on 131,072 Intel Xeon Phi cores for a 56.6 million atom (or 169 million valence-electron) system, whereas the various code transformations achieved 5-fold speedup. The optimized simulation engine allowed us for the first time to establish a significant effect of substrate on the dynamics of layered material upon electronic excitation.
Algebraic BLASification↗
Electrofracturing of Shale at Elevated Pressure
Electrofracturing deeply buried shale formations could be used to increase reservoir permeability and improve reservoir production without requiring large volumes of freshwater. This paper describes a novel experimental system and initial test results to electrofracture shale under high confining pressures. Core-scale laboratory testing was performed on twelve rock samples recovered from a shale gas reservoir. Each sample was subjected to confining pressures of 20.7 MPa (3000 psi) or 58.6 MPa (8000 psi), representative of overburden pressures at depth. Samples were then subjected to application of high voltage until specimen fracture. The experiments produced deformed samples with multiple fracture types, both parallel and oblique to bedding planes. Electrofracturing increased permeabilities by up to nine orders of magnitude for extended time periods. Rock fracture and throughgoing fractures were demonstrated. Computed tomography images revealed the creation of fractures and tube/tunnel flow channels, which resisted closure under hydrostatic pressures up to 58.6 MPa. The breakdown energy and permeability changes in the sample were independent of applied confining pressure. The cumulative energy input required for fracture depended on applied confining pressure and sample length. The energy required to fracture samples up to 9 cm in length is generally more than 0.5 kJ/cm, but no greater than 1 kJ/cm. Our results show that electrofracture of shales under confining pressure is possible and could be a possible water-free mechanism for reservoir stimulation.
04 OIL SHALES AND TAR SANDS↗
Priority-enabled Load Balancing for Dispersed Computing
Opportunistic managed access to local in-network compute resources can improve the performance of distributed applications and reduce the dependence on shared network resources. Instead of backhauling application data to a centralized cloud data center for processing, networked services may be adaptively and continuously dispersed into shared compute resources that are closer to the source of need. While this approach has several benefits, support for mission-aware access to computation is often an afterthought, and is implemented as a brittle extension over traditional load-balancer solutions.In this work, we investigate the design of two priority-aware resource allocation strategies and two load-balancing dispatching strategies as first class citizens in an open-source dispersed computing middleware. We present a control theoretic analysis of these load-balancing primitives to identify weaknesses and strengths in our design, and recommend future directions. In parallel, we prototype two priority-aware allocation algorithms to validate our priority predictions. In initial experiments our prototype shows substantial gains in processing prioritized load. Finally, we make our source-code and experimental configurations open source.
Paulos, Aaron↗
Mahakala: A Python-based Modular Ray-tracing and Radiative Transfer Algorithm for Curved Spacetimes
We introduce Mahakala, a Python-based, modular, radiative ray-tracing code for curved spacetimes. We employ Google's JAX framework for accelerated automatic differentiation, which can efficiently compute Christoffel symbols directly from the metric, allowing the user to easily and quickly simulate photon trajectories through non-Kerr spacetimes. JAX also enables Mahakala to run in parallel on both CPUs and GPUs. Mahakala natively uses the Cartesian Kerr–Schild coordinate system, which avoids numerical issues caused by the pole in spherical coordinate systems. We demonstrate Mahakala's capabilities by simulating 1.3 mm wavelength images (the wavelength of Event Horizon Telescope observations) of general relativistic magnetohydrodynamic simulations of low-accretion rate supermassive black holes. The modular nature of Mahakala allows us to quantitatively explore how different regions of the flow influence different image features. We show that most of the emission seen in 1.3 mm images originates close to the black hole and peaks near the photon orbit. We also quantify the relative contribution of the disk, forward jet, and counterjet to 1.3 mm images.
79 ASTRONOMY AND ASTROPHYSICS↗
Parallel-in-Time Solution of Hyperbolic PDE Systems via Characteristic-Variable Block Preconditioning
We consider the parallel-in-time solution of both linear and nonlinear hyperbolic partial differential equation (PDE) systems in one spatial dimension. In the nonlinear setting, the discretized equations are solved with a preconditioned residual iteration based on a global linearization. The linear(ized) equation systems are approximately solved parallel-in-time using a block preconditioner applied in the characteristic variables of the underlying linear(ized) hyperbolic PDE. This change of variables is motivated by the observation that intervariable coupling between characteristic variables is weak, at least locally where spatio-temporal variations in the eigenvectors of the associated flux Jacobian are sufficiently small, while that between the original variables is not. For an ℓ-dimensional system of PDEs, applying the preconditioner consists of solving a sequence of ℓ scalar linear(ized)-advection-like problems, each associated with a different characteristic wave-speed in the underlying linear(ized) PDE. Furthermore, we approximately solve these linear advection problems using multigrid reduction-in-time (MGRIT); however, any other suitable parallel-in-time method could be used. Numerical examples are shown for the (linear) acoustics equations in heterogeneous media and for the (nonlinear) shallow water equations and Euler equations of gas dynamics with shocks and rarefactions. For many test problems, the solver converges in just a handful of iterations and with mesh-independent convergence rates.
97 MATHEMATICS AND COMPUTING↗
The impact of non-local parallel electron transport on plasma-impurity reaction rates in tokamak scrape-off layer plasmas
Abstract Plasma-impurity reaction rates are a crucial part of modelling tokamak scrape-off layer (SOL) plasmas. To avoid calculating the full set of rates for the large number of important processes involved, a set of effective rates are typically derived which assume Maxwellian electrons. However, non-local parallel electron transport may result in non-Maxwellian electrons, particularly close to divertor targets. Here, the validity of using Maxwellian-averaged rates in this context is investigated by computing the full set of rate equations for a fixed plasma background from kinetic and fluid SOL simulations. We consider the effect of the electron distribution as well as the impact of the electron transport model on plasma profiles. Results are presented for lithium, beryllium, carbon, nitrogen, neon and argon. It is found that electron distributions with enhanced high-energy tails can result in significant modifications to the ionisation balance and radiative power loss rates from excitation, on the order of 50%–75% for the latter. Fluid electron models with Spitzer-Härm or flux-limited Spitzer-Härm thermal conductivity, combined with Maxwellian electrons for rate calculations, can increase or decrease this error, depending on the impurity species and plasma conditions. Based on these results, we also discuss some approaches to experimentally observing non-local electron transport in SOL plasmas.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Scalability of high-performance PDE solvers
Performance tests and analyses are critical to effective high-performance computing software development and are central components in the design and implementation of computational algorithms for achieving faster simulations on existing and future computing architectures for large-scale application problems. In this article, we explore performance and space-time trade-offs for important compute-intensive kernels of large-scale numerical solvers for partial differential equations (PDEs) that govern a wide range of physical applications. We consider a sequence of PDE-motivated bake-off problems designed to establish best practices for efficient high-order simulations across a variety of codes and platforms. We measure peak performance (degrees of freedom per second) on a fixed number of nodes and identify effective code optimization strategies for each architecture. In addition to peak performance, we identify the minimum time to solution at 80% parallel efficiency. The performance analysis is based on spectral and p-type finite elements but is equally applicable to a broad spectrum of numerical PDE discretizations, including finite difference, finite volume, and h-type finite elements.
97 MATHEMATICS AND COMPUTING↗
Parallelized and Vectorized Tracking Using Kalman Filters with CMS Detector Geometry and Events
The High-Luminosity Large Hadron Collider at CERN will be characterized by greater pileup of events and higher occupancy, making the track reconstruction even more computationally demanding. Existing algorithms at the LHC are based on Kalman filter techniques with proven excellent physics performance under a variety of conditions. Starting in 2014, we have been developing Kalman-filter-based methods for track finding and fitting adapted for many-core SIMD processors that are becoming dominant in high-performance systems. This paper summarizes the latest extensions to our software that allow it to run on the realistic CMS-2017 tracker geometry using CMSSW-generated events, including pileup. The reconstructed tracks can be validated against either the CMSSW simulation that generated the hits, or the CMSSW reconstruction of the tracks. In general, the code's computational performance has continued to improve while the above capabilities were being added. We demonstrate that the present Kalman filter implementation is able to reconstruct events with comparable physics performance to CMSSW, while providing generally better computational performance. Further plans for advancing the software are discussed.
Cerati, Giuseppe↗
PPINN: Parareal physics-informed neural network for time-dependent PDEs
Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree. While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time–space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposing a long-time problem into many independent short-time problems supervised by an inexpensive/fast coarse-grained (CG) solver. In particular, the serial CG solver is designed to provide approximate predictions of the solution at discrete times, while initiate many fine PINNs simultaneously to correct the solution iteratively. There is a two-fold benefit from training PINNs with small-data sets rather than working on a large-data set directly, i.e., training of individual PINNs with small-data is much faster, while training the fine PINNs can be readily parallelized. Consequently, compared to the original PINN approach, the proposed PPINN approach may achieve a significant speed-up for long-time integration of PDEs, assuming that the CG solver is fast and can provide reasonable predictions of the solution, hence aiding the PPINN solution to converge in just a few iterations. To investigate the PPINN performance on solving time-dependent PDEs, we first apply the PPINN to solve the Burgers equation, and subsequently we apply the PPINN to solve a two-dimensional nonlinear diffusion–reaction equation. Furthermore, our results demonstrate that PPINNs converge in a few iterations with significant speed-ups proportional to the number of time-subdomains employed.
42 ENGINEERING↗
Robustness of Deep Learning Classification to Adversarial Input on GPUs: Asynchronous Parallel Accumulation Is a Source of Vulnerability
The ability of machine learning (ML) classification models to resist small, targeted input perturbations—known as adversarial attacks—is a key measure of their safety and reliability. We show that floating-point non associativity (FPNA) coupled with asynchronous parallel programming on GPUs is sufficient to result in misclassification, without any perturbation to the input. Additionally, we show that this misclassification is particularly significant for inputs close to the decision boundary and that standard adversarial robustness results may be overestimated up to 4.6 when not considering machine-level details. We first study a linear classifier, before focusing on standard Graph Neural Network (GNN) architectures and datasets used in robustness assessments. We develop a novel black-box attack using Bayesian optimization to discover external workloads that can change the instruction scheduling which bias the output of reductions on GPUs and reliably lead to misclassification. Motivated by these results, we present a new learnable permutation (LP) gradient-based approach to learning floating-point operation orderings that lead to misclassifications. The LP approach provides a worst-case estimate in a computationally efficient manner, avoiding the need to run identical experiments tens of thousands of times over a potentially large set of possible GPU states or architectures. Finally, using instrumentation-based testing, we investigate parallel reduction ordering across different GPU architectures under external background workloads, when utilizing multi-GPU virtualization, and when applying power capping. Our results demonstrate that parallel reduction ordering varies significantly across architectures under the first two conditions, substantially increasing the search space required to fully test the effects of this parallel scheduler-based vulnerability. These results and the methods developed here can help to include machine-level considerations into adversarial robustness assessments, which can make a difference in safety and mission critical applications.
Shanmugavelu, Sanjif [Maxeler Technologies, a Groq↗
Energetic Substantiation of the Formation of Ytterbium Dimers in Single Forsterite Crystals
A structural computer simulation of ytterbium-containing forsterite crystals has been carried out. Atomistic simulation is performed using the GULP 4.1 software package (General Utility Lattice Program). Different mechanisms of ytterbium dissolution in forsterite crystals are considered, and the dissolution energies of isolated defects, as well as charged and neutral clusters of various configurations, are calculated. The results of the calculation manifest that the formation of ytterbium clusters with a magnesium vacancy gives a significant gain in the dissolution energy. The formation of neutral clusters (dimers) in the M1 site (Yb{sub Mg1}ν{sub Mg1}Yb{sub Mg1}) leads to an energy reduction by 1.7 eV compared with the statistical distribution of defects. As a result of simulation, it is shown the energetic substantiation of the formation of ytterbium dimers in forsterite crystals. A model of the most energetically favorable center in the M1 site is proposed—a dimer, consisting of a pair of trivalent ytterbium ions with a magnesium vacancy between them, forming a chain parallel to the crystallographic axis c.
75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗