Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,261 records · Page 70

FatMan vs. LittleBoy: Scaling Up Linear Algebraic Operations in Scale-Out Data Platforms

Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are highly desirable to enable efficient processing over millions of data points. To this end, we present a matrix manipulation approach to effectively scale-up each node in a scale-out data parallel platform such as Apache Spark. Specifically, we enable hardware acceleration for matrix multiplications in a distributed Spark setup without user intervention. Our approach supports both dense and sparse distributed matrices, and provides flexible control of acceleration by matrix density. We demonstrate the benefit of our approach for generalized matrix multiplication operations over large matrices with up to four billion elements. To connect the effectiveness of our approach with machine learning applications, we performed Gramian matrix computation via generalized matrix multiplications. Our experiments show that our approach achieves more than 2× performance speed-up, and up to 96.1% computation improvement, compared to a state of the art Spark MLlib for dense matrices.

97 MATHEMATICS AND COMPUTING↗

QCOR; A Language Extension Specification for the Heterogeneous Quantum-Classical Model of Computation

Quantum computing (QC) is an emerging computational paradigm that leverages the laws of quantum mechanics to perform elementary logic operations. Existing programming models for QC were designed with fault-tolerant hardware in mind, envisioning stand-alone applications. However, the susceptibility of near-term quantum computers to noise limits their stand-alone utility. To better leverage limited computational strengths of noisy quantum devices, hybrid algorithms have been suggested whereby quantum computers are used in tandem with their classical counterparts in a heterogeneous fashion. This modus operandi calls out for a programming model and a high-level programming language that natively and seamlessly supports heterogeneous quantum-classical hardware architectures in a single-source-code paradigm. Motivated by the lack of such a model, we introduce a language extension specification, called QCOR, which enables single-source quantum-classical programming. Programs written using the QCOR library–based language extensions can be compiled to produce functional hybrid binary executables. After defining QCOR’s programming model, memory model, and execution model, we discuss how QCOR enables variational, iterative, and feed-forward QC. Additionally, QCOR approaches quantum-classical computation in a hardware-agnostic heterogeneous fashion and strives to build on best practices of high-performance computing. The high level of abstraction in the language extension is intended to accelerate the adoption of QC by researchers familiar with classical high-performance computing.

97 MATHEMATICS AND COMPUTING↗

E4D_RT

Time-lapse ERT imaging for monitoring both natural and engineered subsurface processes has advanced rapidly over the past 15-20 years. However, imaging results generally required a significant amount of manual and computational effort, and therefore were not available while the process was occurring. Although the value of real-time imaging was recognized, several obstacles prevented it's implementation. E4D_RT addresses these obstacles by 1) providing specialized algorithms that negate the need for user intervention, thereby automating the time-lapse data processing steps, 2) linking field data collection systems with parallel supercomputing systems via wireless data transfer link, and 3) addressing the computational burdens by utilizing distributed memory supercomputing resources, thereby enabling rapid data processing and imaging results.

Source record↗

Discrete Fracture Network Modeling to Estimate Upscaled Parameters for the Topopah Spring, Lava Flow, and Tiva Canyon Aquifers at Pahute Mesa, Nevada National Security Site

This report describes the results of Discrete Fracture Network (DFN) simulations for the Topopah Spring Aquifer (TSA), Lava Flow Aquifer, and Tiva Canyon Aquifer (TCA), at Pahute Mesa on the Nevada National Security Site (NNSS), formerly the Nevada Test Site. The research focuses on calculating upscaled groundwater flow and contaminant transport parameters using DFNs generated according to fracture characteristics observed in the TSA, LFA and TCA at Pahute Mesa. The highly fractured and heterogeneous nature of these aquifers makes them candidates for stochastic DFN modeling of radionuclide transport on a small scale with subsequent upscaling. One hundred independent DFN realizations are generated for each aquifer, and the upscaled parameters for continuum simulations of subsurface flow and transport in fractured media at Pahute Mesa are calculated. Our goal is to implement a modeling approach that can translate parameters to larger-scale models that account for local-scale flow and transport processes, such as channelization of flow and transport along a few well connected, large fractures. Additionally, to simulate advective and advective-diffusive transport through the fracture networks, the Time Domain Random Walk (TDRW) approach is applied to account for matrix diffusion into a finite half-space. Moreover, a novel approach to calculate dynamic (active) fracture surface area to reflect flow channeling is implemented. This work will improve the representation of radionuclide transport processes in largescale, regulatory-focused models by providing estimates of hard-to-measure flow and contaminant transport parameters at large scales. In this report, we (1) show recent results of flow and transport simulations on multiple DFN realizations of the TSA, LFA, TCA; (2) discuss the resulting distributions of estimated upscaled parameters; (3) describe the estimation of upscaled parameters for an equivalent parallel-plate continuum model and (4) present a comparison between simulated transport from the equivalent continuum model and an actual DFN.

54 ENVIRONMENTAL SCIENCES↗

The Mechanism of Electron Injection and Acceleration in Transrelativistic Reconnection

Electron acceleration during magnetic reconnection is thought to play a key role in time-variable high-energy emission from astrophysical systems. By means of particle-in-cell simulations of transrelativistic reconnection, we investigate electron injection and acceleration mechanisms in low-β electron–proton plasmas. We set up a diversity of density and field structures (e.g., X-points and plasmoids) by varying the guide field strength and choosing whether to trigger reconnection or let it spontaneously evolve. We show that the number of X-points and plasmoids controls the efficiency of electron acceleration, with more X-points leading to a higher efficiency. Using on-the-fly acceleration diagnostics, we also show that the nonideal electric fields associated with X-points play a critical role in the first stages of electron acceleration. As a further diagnostic, we include two populations of test particles that selectively experience only certain components of electric fields. We find that the out-of-plane component of the parallel electric field determines the hardness of the high-energy tail of the electron energy distribution. These results further our understanding of electron acceleration in this regime of magnetic reconnection and have implications for realistic models of black hole accretion flows.

79 ASTRONOMY AND ASTROPHYSICS↗

FatMan vs. LittleBoy: Scaling up Linear Algebraic Operations in Scale-out Data Platforms

Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are highly desirable to enable efficient processing over millions of data points. To this end, we present a matrix manipulation approach to effectively scale-up each node in a scale-out data parallel platform such as Apache Spark. Specifically, we enable hardware acceleration for matrix multiplications in a distributed Spark setup without user intervention. Our approach supports both dense and sparse distributed matrices, and provides flexible control of acceleration by matrix density. We demonstrate the benefit of our approach for generalized matrix multiplication operations over large matrices with up to four billion elements. To connect the effectiveness of our approach with machine learning applications, we performed Gramian matrix computation via generalized matrix multiplications. Our experiments show that our approach achieves more than 2x performance speed-up, and up to 96.1% computation improvement, compared to a state of the art Spark MLlib for dense matrices.

Lim, Seung-Hwan↗

KIVA-4mpi

KIVA-4mpi is a transient, three-dimensional, multi-phase, multicomponent code for the analysis of chemically reacting flows with sprays has been under development at the Los Alamos National Laboratory for the past several years. The code uses an Arbitrary Lagrangian Eulerian (ALE) methodology on a staggered grid, and discretizes space using the finite-volume technique. This version is a parallel version which uses the Mpich libraries for communication of information as needed by the various distributed processing units. The domain is decomposed or formed into a distributed system using Metis or some local tools for specify which cells/nodes are on what processor. The code uses an implicit time-advancement with the exception of the advective terms that are cast in an explicit but second-order monotonicity-preserving manner. Also, the convection calculations can be subcycled in the desired regions to avoid restricting the time step due to Courant conditions. The range of validity of the code extends from low speeds to supersonic flows for both laminar and turbulent regimes. Arbitrary numbers of species and chemical reactions are allowed. A stochastic particle method is used to calculate evaporating liquid sprays, including the effects of droplet collisions and aerodynamic breakups. Although specifically designed for performing internal combustion engine calculations, the modularity of the code allows it for easy modifications for solving a variety of hydrodynamics problems involving chemical reactions. The code has found a widespread application in the automotive industry.

Torres, David↗

An Open-Source Parallel EMT Simulation Framework: Preprint

As the integration level of inverter-based resources (IBR) increases, ensuring the reliable operation of the bulk power systems requires the use of electromagnetic transient (EMT) simulation tools to identify and mitigate system-wide stability risks. Conducting EMT studies for large-scale, IBR-rich grids, however, is challenging due to the inherent computational bottleneck caused by the underlying high-fidelity models and required small time steps. This paper introduces ParaEMT: an open-source, generic EMT simulation framework designed to accelerate simulations by leveraging advanced parallel computational technologies, such as high-performance computers. This paper presents a comprehensive exposition of ParaEMT, covering its modeling library, simulation strategy, framework structure, operational procedures, and auxiliary features, alongside its extensible parallel computational architecture. Notably, ParaEMT is a publicly accessible and modularized framework written in Python, thereby facilitating future development and the integration of new models and algorithms. The accuracy and efficiency of ParaEMT are demonstrated by rigorous validations via multiple case studies.

electromagnetic transient simulation↗

Characterization and monitoring of subsurface processes using parallel computing and electrical resistivity imaging

This newsletter discusses recent advancement in subsurface resistivity characterization and monitoring capabilities. The BC Cribs field desiccation treatability test resistivity monitoring data is use an example to demonstrate near-real time 3D subsurface imaging capabilities. Electrical resistivity tomography (ERT) is a method of imaging the electrical resistivity distribution of the subsurface. An ERT data collection system consists of an array of electrodes, deployed on the ground surface or within boreholes, that are connected to a control unit which can access each electrode independently (Figure 1). A single measurement is collected by injecting current across a pair of current injection electrodes (source and sink), and measuring the resulting potential generated across a pair of potential measurement electrodes (positive and negative). An ERT data set is generated by collecting many such measurements using strategically selected current and potential electrode pairs. This data set is then processed using an inversion algorithm, which reconstructs an estimate (or image) of the electrical conductivity (i.e. the inverse of resistivity) distribution that gave rise to the measured data.

99 GENERAL AND MISCELLANEOUS↗

Distributed File System Utilities to Manage Large DatasetsVersion 0.5

FileUtils provides a suite of tools to manage large datasets typically created by large parallel MPI applications. They are written in C and use standard POSIX I/Ocalls. The current suite consists of tools to copy, compare, remove, and list. The tools provide dramatic speedup over existing Linux tools, which often run as a single process.

ECP↗

Situational Awareness of Grid Anomalies (SAGA)

The modern power industry becomes more vulnerable to cyber events due to the growing interconnectivity, interdependence, and complexity of the electric power grid. High-fidelity modeling and simulation tools that support the preventative risk analysis on potential cyber-relevant events is essential for ensuring the situational awareness of the system operator as it provides an inexpensive and risk-free environment to test the system responses under various cyber-relevant events and hereby can support research on cyber anomaly detection, optimal protective resource allocation, and mitigation measures. In this webinar, we will share NREL's cybersecurity research capabilities by highlighting the development of a scalable cyber-physical event test bed and demonstration with real hardware in the loop. The developed cyber-physical event test bed is backboned by an integrated transmission, distribution, and communication dynamic co-simulation framework and a plug-and-play cyber event generation module. It is designed to be modular and compatible with parallel computing, and thereby supports large-scale system simulations at an affordable computation cost. The test bed can capture millisecond-to-minutes dynamic frequency and voltage responses under cyber events from the bulk transmission system to the active distribution systems and distributed energy resources at the grid edge.

co-simulation↗

General properties of small-amplitude fluctuations in magnetized and unmagnetized collision poor plasmas. I. The dielectric tensor

The dielectric tensor of a collision poor plasma determines all the physical properties of small-amplitude fluctuations for given initial plasma particle distribution functions, as it enters the Maxwell operator as the only plasma-specific quantity. For the important class of gyrotropic initial particle distribution functions f{sub a}{sup (0)}(p{sub ||}),p{sub perpendicular} in a uniform magnetic field, we rigorously investigate the general properties of the plasma fluctuations without specifying the explicit momentum dependence of the gyrotropic distribution function. Two alternative forms of the relativistically correct dielectric tensor are derived which differ from nonrelativistic expressions in the literature. The first standard form is expressed in terms of infinite series of Bessel functions, whereas in the second form these infinite series are calculated with the Lerche-Newberger sum rules, yielding products of Bessel functions with complex indices for the individual elements of the Maxwell operator. The second form of the dielectric tensor is well suited to simplify the tensor in the special cases of parallel wave vectors and unmagnetized plasmas. For unmagnetized plasmas it is shown that aperiodic electrostatic and transverse fluctuations can only exist in symmetric distribution functions with f(-p{sub ||},p{sub perpendicular})=f(p{sub ||},p{sub perpendicular}). Because this includes isotropic distribution functions, the more thorough investigation of this special case reveals that no electrostatic and fluctuations with positive growth rates {gamma}=kcS>0 exist in an isotropic unmagnetized plasma, excluding both aperiodic (with R=0) and wave-like (with R{ne}0) instabilities, where R={omega}{sub R}/(kc) denotes the real part of the phase speed. The second form of the dielectric tensor is also most appropriate to investigate fluctuations in magnetized equal mass plasmas, such as electron-positron-pair and/or proton-antiproton plasmas. Here for arbitrary wave vector orientation the dispersion relation factorizes into three separate modes. For fluctuations with parallel wave vectors in isotropic plasmas of arbitrary composition, the electromagnetic stability of such isotropic plasma populations is proven, relativistically generalizing the known corresponding nonrelativistic theorem.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Distributed structure determination at the JCSG

The software suite Xsolve semi-exhaustively explores key parameters of the X-ray structure-determination process to compute multiple three-dimensional protein structures independently and in parallel from a set of diffraction images. An optimal consensus model for subsequent manual refinement is computed from these structures. The Joint Center for Structural Genomics (JCSG), one of four large-scale structure-determination centers funded by the US Protein Structure Initiative (PSI) through the National Institute for General Medical Sciences, has been operating an automated distributed structure-solution pipeline, Xsolve, for well over half a decade. During PSI-2, Xsolve solved, traced and partially refined 90% of the JCSG’s nearly 770 MAD/SAD structures at an average resolution of about 2 Å without human intervention. Xsolve executes many well established publicly available crystallography software programs in parallel on a commodity Linux cluster, resulting in multiple traces for any given target. Additional software programs have been developed and integrated into Xsolve to further minimize human effort in structure refinement. ConsensusModeler exploits complementarities in traces from Xsolve to compute a single optimal model for manual refinement. Xpleo is a powerful robotics-inspired algorithm to build missing fragments and qFit automatically identifies and fits alternate conformations.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Mass transport limitations in polymer electrolyte water electrolyzers using spatially-resolved current measurement

Here this work utilizes spatially-resolved current measurements to provide insight into mass transport limitations in electrolyzers that are not observable from traditional polarization measurement. In this study, two types of flow-fields (parallel and triple-serpentine) and two types of diffusion media (patterned porous thin titanium foil LGDLs and Ir-coated titanium felt PTLs) were examined. A non-uniform current distribution dominated by mass transport limitations was observed to be instigated by the restriction of liquid water transport to catalyst sites. Additionally, conditions are revealed which yield similar polarization performance but dissimilar current distributions. In such cases, the transport limitations for different architectures and porous media affect polarization in different regions of the active area. Furthermore, the triple-serpentine flow-field used in this study performs better than the parallel flow-field under mass transport limited operating conditions. This indicates that the parallel flow-field used in this study is more susceptible to starvation than the triple-serpentine flow-field for the electrolyzer studied.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Virtual Time III, Part 2: Combining Conservative and Optimistic Synchronization

This is Part 2 of a trio of works intended to provide a unifying framework in which conservative and optimistic synchronization for parallel discrete event simulations can be freely and transparently combined in the same logical process on an event-by-event basis. Here, in this article, we continue the outline of an approach called Unified Virtual Time (UVT) that was introduced in Part 1, showing in detail via two extended examples how conservative synchronization can be refactored and combined with optimistic synchronization in the UVT framework. We describe UVT versions of both a basic time windowing algorithm called Unified Simple Time Windows and a refactored version of the Chandy-Misra-Bryant Null Message algorithm called Unified CMB.

97 MATHEMATICS AND COMPUTING↗

Electron Bulk Acceleration and Thermalization at Earth’s Quasiperpendicular Bow Shock [Electron Acceleration and Thermalization at Earth's Quasi-Perpendicular Bow Shock]

Electron heating at Earth’s quasiperpendicular bow shock has been surmised to be due to the combined effects of a quasistatic electric potential and scattering through wave-particle interaction. Here we report the observation of electron distribution functions indicating a new electron heating process occurring at the leading edge of the shock front. Incident solar wind electrons are accelerated parallel to the magnetic field toward downstream, reaching an electron-ion relative drift speed exceeding the electron thermal speed. The bulk acceleration is associated with an electric field pulse embedded in a whistler-mode wave. The high electron-ion relative drift is relaxed primarily through a nonlinear current-driven instability. Here, the relaxed distributions contain a beam traveling toward the shock as a remnant of the accelerated electrons. Similar distribution functions prevail throughout the shock transition layer, suggesting that the observed acceleration and thermalization is essential to the cross-shock electron heating.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Trilinos for emerging parallel computing systems.

Trilinos is an object-oriented software framework to enabled the solution of large-scale, complex multiphysics engineering and scientific problems. Different Trilinos packages build on each other to create a stack providing the necessary capability: (1) Non-linear solver; (2) Linear solver/preconditioner; (3) Distributed linear algebra; and (4) Local linear algebra.

99 GENERAL AND MISCELLANEOUS↗