Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

Development of Ensemble Neural Network Convection Parametrizations for Climate Models Using ARM Data

The novel neural network (NN) approach has been formulated and used for development of a NN ensemble stochastic convection parametrization for climate models. This fast parametrization is built based on data from Cloud Resolving Model (CRM) simulations initialized with and forced by TOGA-COARE data. The SAM (System for Atmospheric Modeling), developed by D. Randall, M. Khairoutdinov, and their collaborators, has been used for CRM simulations. The observational data are also used for validation of model simulations. The SAM-simulated data have been averaged and projected onto the GCM space of atmospheric states to implicitly define a stochastic convection parametrization. This parametrization is emulated using an ensemble of NNs. An ensemble of NNs with different NN parameters has been trained and tested. The inherent uncertainty of the stochastic convection parametrization derived in such a way is estimated. Due to these inherent uncertainties, NN ensemble is used to constitute a stochastic NN convection parametrization. The developed NN convection parametrization have been validated in a diagnostic CAM (CAM-NN) run vs. the control CAM run. Actually, CAM inputs have been used, at every time step of the control/original CAM integration, for parallel calculations of the NN convection parametrization (CAM-NN) to produce its outputs as a diagnostic byproduct. Total precipitation (P) and cloudiness (CLD) time series, diurnal cycles, and P and CLD distributions for the large Tropical Pacific Ocean for the parallel CAM-NN and CAM runs show similarity and consistency with the NCEP reanalysis. The P and CLD distributions for the tropical area for the parallel runs have been analyzed first for the TOGA-COARE boreal winter season (November 1992 through February 1993) and then for the winter seasons of the follow-up parallel decadal simulations. The obtained results are encouraging and practically meaningful. They show the validity of the NN approach. This constitutes an important practical conclusion of the study: the obtained results on NN ensembles as a stochastic physics parametrization show a realistic possibility of development of NN convection parametrization for climate (and NWP) models based on learning cloud physics from CRM/SAM simulated data.

97 MATHEMATICS AND COMPUTING↗

Parallel Acceleration of Suprathermal Electrons Caused by Whistler‐Mode Hiss Waves

Abstract Suprathermal electrons (~0.1–10 keV) in the inner magnetosphere are usually observed in a 90°‐peaked pitch angle distribution, formed due to the conservation of the first and second adiabatic invariants as they are transported from the plasma sheet. We report a peculiar field‐aligned suprathermal electron (FASE) distribution measured by Van Allen Probes, where parallel fluxes are 1 order of magnitude higher than perpendicular fluxes. Those FASEs are found to be closely correlated with large‐amplitude hiss waves and are observed around the Landau resonant energies. We demonstrate, using quasilinear diffusion simulations, that hiss waves can rapidly accelerate suprathermal electrons through Landau resonance and create the observed FASE population. The proposed mechanism potentially has broad implications for suprathermal electron dynamics as well as whistler mode waves in the Earth's magnetosphere and has been demonstrated in the Jovian magnetosphere.

Li, Jinxing↗

Jali - Unstructured Mesh Infrastructure for Multi-Physics Applications

Jali is a parallel unstructured mesh infrastructure library designed for use by multi-physics simulations. It supports 2D and 3D arbitrary polyhedral meshes distributed over hundreds to thousands of nodes. Jali can read write Exodus II meshes along with fields and sets on the mesh and support for other formats is partially implemented or is (https://github.com/MeshToolkit/MSTK), an open source general purpose unstructured mesh infrastructure library from Los Alamos National Laboratory. While it has been made to work with other mesh frameworks such as MOAB and STKmesh in the past, support for maintaining the interface to these frameworks has been suspended for now. Jali supports distributed as well as on-node parallelism. Support of on-node parallelism is through direct use of the the mesh in multi-threaded constructs or through the use of "tiles" which are submeshes or sub-partitions of a partition destined for a compute node.

Garimella, RaoV↗

Jali - Unstructured Mesh Infrastructure for Multi-Physics Applications

Jali is a parallel unstructured mesh infrastructure library designed for use by multi-physics simulations. It supports 2D and 3D arbitrary polyhedral meshes distributed over hundreds to thousands of nodes. Jali can read write Exodus II meshes along with fields and sets on the mesh and support for other formats is partially implemented or is (https://github.com/MeshToolkit/MSTK), an open source general purpose unstructured mesh infrastructure library from Los Alamos National Laboratory. While it has been made to work with other mesh frameworks such as MOAB and STKmesh in the past, support for maintaining the interface to these frameworks has been suspended for now. Jali supports distributed as well as on-node parallelism. Support of on-node parallelism is through direct use of the the mesh in multi-threaded constructs or through the use of "tiles" which are submeshes or sub-partitions of a partition destined for a compute node.

Garimella, Rao V↗

TriC: Distributed-memory Triangle Counting by Exploiting the Graph Structure

Graph analytics has emerged as an important tool in the analysis of large scale data from diverse application domains such as social networks, cyber security and bioinformatics. Counting the number of triangles in a graph is a fundamental kernel with several applications such as detecting the community structure of a graph or in identifying important vertices in a graph. The ubiquity of massive datasets is driving the need to scale graph analytics on parallel systems. However, numerous challenges exist in efficiently parallelizing graph algorithms, especially on distributed-memory systems. Irregular memory accesses and communication patterns, low computation to communication ratios, and the need for frequent synchronization are some of the leading challenges. In this paper, we present TriC, our distributed-memory implementation of triangle counting in graphs using the Message Passing Interface (MPI), as a submission to the 2020 GraphChallenge competition. Using a set of synthetic and real-world inputs from the challenge, we demonstrate a speedup of up to 90x relative to previous work on 32 processor-cores of a NERSC Cori node. We also provide details from distributed runs with up to8192 processes along with strong scaling results. The observations presented in this work provide an understanding of the system-level bottlenecks at scale that specifically impact sparse-irregular workloads and will therefore benefit other efforts to parallelize graph algorithms.

Halappanavar, Mahantesh↗

Pushing configuration-interaction to the limit: Towards massively parallel MCSCF calculations

A new large-scale parallel multiconfigurational self-consistent field (MCSCF) implementation in the open-source NWChem computational chemistry code is presented. The generalized active space approach is used to partition large configuration interaction (CI) vectors and generate a sufficient number of batches that can be distributed to the available cores. Massively parallel CI calculations with large active spaces can be performed. The new parallel MCSCF implementation is tested for the chromium trimer and for an active space of 20 electrons in 20 orbitals, which can now routinely be performed. Unprecedented CI calculations with an active space of 22 electrons in 22 orbitals for the pentacene systems were performed and a single CI iteration calculation with an active space of 24 electrons in 24 orbitals for the chromium tetramer was possible. In conclusion, the chromium tetramer corresponds to a CI expansion of one trillion Slater determinants (914 058 513 424) and is the largest conventional CI calculation attempted up to date.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

EVIDENCE OF LANDAU AND CYCLOTRON RESONANCE BETWEEN PROTONS AND KINETIC WAVES IN SOLAR WIND TURBULENCE

The wave–particle interaction processes occurring in the solar wind provide crucial information to understand the wave dissipation and simultaneous particle heating in plasma turbulence. One requires observations of both wave fluctuations and particle kinetics near the dissipation range, which have, however, not yet been analyzed simultaneously. Here we show new evidence of wave–particle interactions by combining the diagnosis of wave modes with the analysis of particle kinetics on the basis of measurements from the WIND spacecraft with a high cadence of about 3 s. Solar wind protons appear to be highly dynamic in their velocity distribution consisting of varying anisotropic core and beam components. The basic scenario of solar wind proton heating through wave–particle interaction is suggested to be the following. Left-handed cyclotron resonance occurs continuously, and is evident from the observed proton core velocity distribution and the concurrent quasi-parallel left-handed Alfvén cyclotron waves. Landau and right-handed cyclotron resonances are persistent and indicated by the observed drifting anisotropic beam and the simultaneous quasi-perpendicular right-handed kinetic Alfvén waves in a general sense. The persistence of non-gyrotropic proton distributions may cast new light on the nature of the interaction between particles and waves near and beyond the proton gyro-frequency.

79 ASTRONOMY AND ASTROPHYSICS↗

Multi-water-bag models of ion temperature gradient instability in cylindrical geometry

Ion temperature gradient instabilities play a major role in the understanding of anomalous transport in core fusion plasmas. In the considered cylindrical geometry, ion dynamics is described using a drift-kinetic multi-water-bag model for the parallel velocity dependency of the ion distribution function. In a first stage, global linear stability analysis is performed. From the obtained normal modes, parametric dependencies of the main spectral characteristics of the instability are then examined. Comparison of the multi-water-bag results with a reference continuous Maxwellian case allows us to evaluate the effects of discrete parallel velocity sampling induced by the Multi-Water-Bag model. Differences between the global model and local models considered in previous works are discussed. Using results from linear, quasilinear, and nonlinear numerical simulations, an analysis of the first stage saturation dynamics of the instability is proposed, where the divergence between the three models is examined.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Automatic Parallelization of Numerical Python Applications using the Global Arrays Toolkit

Global Arrays is a software system from Pacific Northwest National Laboratory that enables an efficient, portable, and parallel shared-memory programming interface to manipulate distributed dense arrays. The NumPy module is the de facto standard for numerical calculation in the Python programming language, a language whose use is growing rapidly in the scientific and engineering communities. NumPy provides a powerful N-dimensional array class as well as other scientific computing capabilities. However, like the majority of the core Python modules, NumPy is inherently serial. Using a combination of Global Arrays and NumPy, we have reimplemented NumPy as a distributed drop-in replacement called Global Arrays in NumPy (GAiN). Serial NumPy applications can become parallel, scalable GAiN applications with only minor source code changes. Scalability studies of several different GAiN applications will be presented showing the utility of developing serial NumPy codes which can later run on more capable clusters or supercomputers.

99 GENERAL AND MISCELLANEOUS↗

Spectral quadrature for the first principles study of crystal defects: Application to magnesium

In this work, we present an accurate and efficient finite-difference formulation and parallel implementation of Kohn-Sham Density (Operator) Functional Theory (DFT) for non periodic systems embedded in a bulk environment. Specifically, employing non-local pseudopotentials, local reformulation of electrostatics, and truncation of the spatial Kohn-Sham Hamiltonian, and the Linear Scaling Spectral Quadrature method to solve for the pointwise electronic fields in real-space and the non-local component of the atomic force, we develop a parallel finite difference framework suitable for distributed memory computing architectures to simulate non-periodic systems embedded in a bulk environment. Choosing examples from magnesium-aluminum alloys, we first demonstrate the convergence of energies and forces with respect to spectral quadrature polynomial order, and the width of the spatially truncated Hamiltonian. Next, we demonstrate the parallel scaling of our framework, and show that the computation time and memory scale linearly with respect to the number of atoms. Next, we use the developed framework to simulate isolated point defects and their interactions in magnesium-aluminum alloys. Our findings conclude that the binding energies of divacancies, Al solute-vacancy and two Al solute atoms are anisotropic and are dependent on cell size. Furthermore, the binding is favorable in all three cases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Distributed Halide

Many image processing tasks are naturally expressed as a pipeline of small computational kernels known as stencils. Halide is a popular domain-specific language and compiler designed to implement image processing algorithms. Halide uses simple language constructs to express what to compute and a separate scheduling co-language for expressing when and where to perform the computation. This approach has demonstrated performance comparable to or better than hand-optimized code. Until now, however, Halide has been restricted to parallel shared memory execution, limiting its performance for memory-bandwidth-bound pipelines or large-scale image processing tasks. We present an extension to Halide to support distributed-memory parallel execution of complex stencil pipelines. These extensions compose with the existing scheduling constructs in Halide, allowing expression of complex computation and communication strategies. Existing Halide applications can be distributed with minimal changes, allowing programmers to explore the tradeoff between recomputation and communication with little effort. Approximately 10 new of lines code are needed even for a 200 line, 99 stage application. On nine image processing benchmarks, our extensions give up to a 1.4× speedup on a single node over regular multithreaded execution with the same number of cores, by mitigating the effects of non-uniform memory access. The distributed benchmarks achieve up to 18× speedup on a 16 node testing machine and up to 57× speedup on 64 nodes of the NERSC Cori supercomputer.

97 MATHEMATICS AND COMPUTING↗

Aces4: A Platform for Computational Chemistry Calculations with Extremely Large Block-Sparse Arrays

Aces4 is a parallel programming platform comprising a DSL for Computational Chemistry and its runtime system. It offers a convenient way to express parallelism together with extensive support for extremely large, possibly sparse, distributed arrays. It aids scientists in the creation of performant, scalable, massively parallel programs that can effectively take advantage of leadership class computing systems to address important scientific questions. Aces4 has enabled the development and implementation of new methods in electronic structure theory which are breaking new ground in their ability to perform highly accurate calculations on ever larger molecular systems. In this paper the design of Aces4, which is based on the the Super Instruction Architecture approach, is described. Experimental scaling results for Molecular Cluster Perturbation Theory, a new method enabled by Aces4, and CCSD, a widely used computational chemistry method are given.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

An Adaptive Memory Interface Controller for Improving Bandwidth Utilization of Hybrid and Reconfigurable Systems

Emerging applications such as data mining, bioinformatics, knowledge discovery, social network analysis are irregular. They use data structures based on pointers or linked lists, such as graphs, unbalanced trees or unstructures grids, which generates unpredictable memory accesses. These data structures usually are large, but difficult to partition. These applications mostly are memory bandwidth bounded and have high synchronization intensity. However, they also have large amounts of inherent dynamic parallelism, because they potentially perform a task for each one of the element they are exploring. Several efforts are looking at accelerating these applications on hybrid architectures, which integrate general purpose processors with reconfigurable devices. Some solutions, which demonstrated significant speedups, include custom-hand tuned accelerators or even full processor architectures on the reconfigurable logic. In this paper we present an approach for the automatic synthesis of accelerators from C, targeted at irregular applications. In contrast to typical High Level Synthesis paradigms, which construct a centralized Finite State Machine, our approach generates dynamically scheduled hardware components. While parallelism exploitation in typical HLS-generated accelerators is usually bound within a single execution flow, our solution allows concurrently running multiple execution flow, thus also exploiting the coarser grain task parallelism of irregular applications. Our approach supports multiple, multi-ported and distributed memories, and atomic memory operations. Its main objective is parallelizing as many memory operations as possible, independently from their execution time, to maximize the memory bandwidth utilization. This significantly differs from current HLS flows, which usually consider a single memory port and require precise scheduling of memory operations. A key innovation of our approach is the generation of a memory interface controller, which dynamically maps concurrent memory accesses to multiple ports. We present a case study on a typical irregular kernel, Graph Breadth First search (BFS), exploring different tradeoffs in terms of parallelism and number of memories.

BFS↗

Design and Analysis of 10” Parallel Plate Relief Device

All Cryomodule (CM) and Cryogenic Distribution System (CDS) relieving into Helium Low Pressure (LP) return header, which is connected to compressor suction so, helium can be preserved during small flow relieving event and recirculated to system. However, during worst case scenario, Helium LP header requires a parallel plate relief device to relieve excess pressure from header. To complete the CDS Warm piping header, a new design for a 10 parallel plate relief device is necessary to relieve outside of the tunnel into atmosphere.

Chicas, Kelly↗

Study of Electromagnetic Electron Cyclotron Waves for Kappa Distribution with AC Field in the Magnetosphere of Saturn

Parallel propagating electromagnetic electron cyclotron (EMEC) waves in the extended plasma sheet (~12R{sub S}) and in the outer magnetosphere (~18R{sub S}) of Saturn have been studied. A dispersion relation for parallel propagating relativistic EMEC waves has been applied to the magnetosphere of Saturn, and comparisons have been made with the data of Voyager 1 at these radial distances. The detailed investigations for EMEC waves have been done in the presence of the perpendicular AC electric field, using the kappa distribution function. The relativistic temporal growth rate is calculated by the method of characteristic solution with the data provided by Voyager 1. The effect of the suprathermal electron density, temperature anisotropy, frequency of AC electric field, thermal energy of ions, and relativistic factor on the temporal growth rate of EMEC wave emission has been studied. The simulation results show that the growth of parallel propagating EMEC waves is significantly affected by variations in the temperature anisotropy, electron density, ion thermal energy, and relativistic factor in both the extended plasma sheet and the outer magnetosphere of Saturn. The temperature anisotropy (T⊥/T{sub ║}), ion thermal energy (K{sub B}T{sub ║i}), and electron density (n{sub 0}) have been found to be a major source of free energy for parallel propagating EMEC waves in both regions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Water bag modeling of a multispecies plasma

We report in the present paper a new modeling method to study multiple species dynamics in magnetized plasmas. Such a method is based on the gyrowater bag modeling, which consists in using a multistep-like distribution function along the velocity direction parallel to the magnetic field. The choice of a water bag representation allows an elegant link between kinetic and fluid descriptions of a plasma. The gyrowater bag model has been recently adapted to the context of strongly magnetized plasmas. We present its extension to the case of multi ion species magnetized plasmas: each ion species being modeled via a multiwater bag distribution function. The water bag modelization will be discussed in details, under the simplification of a cylindrical geometry that is convenient for linear plasma devices. As an illustration, results obtained in the linear framework for ion temperature gradient instabilities are presented, that are shown to agree qualitatively with older works.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Electron Scattering by High-frequency Whistler Waves at Earth’s Bow Shock

Electrons are accelerated to non-thermal energies at shocks in space and astrophysical environments. While different mechanisms of electron acceleration have been proposed, it remains unclear how non-thermal electrons are produced out of the thermal plasma pool. Here, we report in situ evidence of pitch-angle scattering of non-thermal electrons by whistler waves at Earth’s bow shock. On 2015 November 4, the Magnetospheric Multiscale (MMS) mission crossed the bow shock with an Alfvén Mach number ∼11 and a shock angle ∼84°. In the ramp and overshoot regions, MMS revealed bursty enhancements of non-thermal (0.5–2 keV) electron flux, correlated with high-frequency (0.2–0.4 Ω{sub ce}, where Ω{sub ce} is the cyclotron frequency) parallel-propagating whistler waves. The electron velocity distribution (measured at 30 ms cadence) showed an enhanced gradient of phase-space density at and around the region where the electron velocity component parallel to the magnetic field matched the resonant energy inferred from the wave frequency range. The flux of 0.5 keV electrons (measured at 1 ms cadence) showed fluctuations with the same frequency. These features indicate that non-thermal electrons were pitch-angle scattered by cyclotron resonance with the high-frequency whistler waves. However, the precise role of the pitch-angle scattering by the higher-frequency whistler waves and possible nonlinear effects in the electron acceleration process remains unclear.

79 ASTRONOMY AND ASTROPHYSICS↗

Adaptive Protection and Validated Models to Enable Deployment of High Penetrations of Solar PV (PV-MOD)

The availability and validation of various PV models in commercial tools differ, with some models not yet thoroughly validated for advanced inverter functionalities and reliable performance under weak system conditions. Many existing models do not fully incorporate new inverter control functions, which can affect system stability. The increasing deployment of solar PV and other inverter-based resources (IBRs), including distributed energy resources (DERs), is influencing the reliable operation of protection schemes in distribution systems and microgrids. Emerging adaptive protection schemes (APS) offer new opportunities for protecting these systems during varying configurations and DER operating conditions, though their demonstration and validation remain limited. Adaptive protection schemes face similar challenges, as they are typically designed for specific configurations. There is a growing need for tools and methodologies to streamline the deployment of adaptive protection for safe and reliable DER integration. The project main objective was to develop and validate high-fidelity generic models of solar PV facilities for stability, protection, EMT, and QSTS analyses. This objective was achieved, and these models can now be integrated into commercial software tools, enabling utilities, vendors, and developers to study high-penetration PV systems more confidently. The project also demonstrated advanced applications of these models, including the design and deployment of adaptive protection schemes in high-penetration field applications and microgrids, supporting grid safety and reliability. Several milestones were reached by the end of the project. A sophisticated inverter test plan was developed, and inverters representative of the North American marketplace were selected. EPRI and NREL tested various inverters, conforming to IEEE standards. Improvements were made to existing generic models of IBR units, IBR plants, and aggregated feeders for various analyses. The first generic electromagnetic transient (EMT) model for a solar PV plant was developed, conforming to IEEE Std 2800™-2022 and validated against laboratory measurements of a 2.2 MVA large-scale battery energy storage system (BESS) inverter. That model was then used to produce reference responses illustrating examples of validated and verified IBR plant models that pass or fail tests for technical minimum capability and performance as specified in the IEEE standard. The developed, tested, and validated generic models can be used for transmission planning, stability assessments, expansion planning, and evaluating potential future IBR interconnection requirements. They can also support interconnection screens and conformity assessments of IBR plants, including solar PV. The project significantly contributed to the ongoing standardization and model-based representation and verification of IBR responses. The project further addressed challenges of common distribution protection schemes with increasing deployment of DER by developing, validating, and demonstrating adaptive protection schemes (APS) that can improve the reliable and safe integration of DER into distribution systems. New APS were designed using improved DER models for three common distribution systems: a radial feeder, a meshed network, and a microgrid. Modeling and hardware-in-the-loop (HIL) testing of the APS were conducted, successfully showing their effectiveness and selectivity. Proof-of-concept field demonstration was achieved for two APS, i.e., one on a radial feeder and another one in a microgrid. Field demonstration could not be achieved for the APS on a meshed network, primarily due apprehension of one utility partner and also due to limited access to the protective algorithms in the network protectors. Guidelines developed from the lessons learned in the project lay out the general process followed in the design, installation, and commissioning of APS for various distribution systems. Distribution utility partners’ apprehension about field demonstration of the new APS were addressed—with varying success—by taking a stepped risk-management approach of modeling of a wide range of sensitivities first, performing in-depth proof-of-concept testing in the laboratory including HIL next, and finally deliberately implementing and commissioning the actual protection equipment and algorithms into parts of—or in parallel operation to—the three real distribution systems. Future work should include pilot projects that further show the acceptable performance of the developed APS before these schemes be rolled out more widely. Inclusion of both utility and original equipment manufacturers (OEMs) in future projects could increase chances of successful field demonstration. Despite challenges in achieving the field demonstration goal of the project for all three APS, the research significantly contributed to the innovation of adaptive protection solutions for scalable and reliable DER integration into distribution systems. This project significantly enhances the understanding of the impact of using appropriate inverter models on distribution and transmission (T&D) systems. By addressing the limitations of existing generic models, the project introduces high-fidelity models for stability, protection, electromagnetic transient (EMT), and quasi-static time series (QSTS) analyses. These models, integrated into commercial software tools, enable utilities, vendors, and developers to confidently study high-penetration PV systems. The project also demonstrates advanced applications, including adaptive protection schemes (APS) for distribution systems and microgrids, ensuring grid safety and reliability. The technical effectiveness and economic feasibility of the methods are evident through the development and validation of sophisticated inverter test plans and the selection of representative inverters. Testing by EPRI and NREL on retail, commercial, and utility-scale inverters, conforming to IEEE standards, underscores the robustness of the models. Improvements to existing generic models for various analyses further enhance their validity and applicability. The project also identifies gaps in common distribution protection schemes and designed new APS using improved DER models, demonstrating their effectiveness through modeling and hardware-in-the-loop (HIL) testing. The project’s benefits to the public are manifold. By advancing the standardization and model-based representation of IBR response, it supports transmission planning, stability assessments, and future IBR interconnection requirements. The generic models can facilitate better communication between transmission planners and developers, supporting expected IBR plant capability and performance. Additionally, the development of APS for radial feeders, meshed networks, and microgrids supports the integration of distributed energy resources (DERs) into distribution systems, enhancing grid reliability and safety. The project’s emphasis on thorough testing and simplicity in design ensures practical and scalable solutions for DER integration.

14 SOLAR ENERGY↗