Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

TESS

Computing a Voronoi or Delaunay tessellation from a set of points is a core part of the analysis of many simulated and measured datasets. As the scale of simulations and observations surpasses billions of particles, a distributed-memory scalable parallel algorithm is the only feasible approach. The primary contribution of this software is a distributed-memory parallel Delaunay and Voronoi tessellation algorithm based on existing serial computational geometry libraries that automatically determines which neighbor points need to be exchanged among the subdomains of a spatial decomposition. Other contributions include the addition of periodic and wall boundary conditions.

Dmitriy Morozov, Tom Peterka↗

Gradient-Based Multi-Area Distribution System State Estimation

The increasing distributed and renewable energy resources and controllable devices in distribution systems make fast distribution system state estimation (DSSE) crucial in system monitoring and control. We consider a large multi-phase distribution system and formulate DSSE as a weighted least squares (WLS) problem. We divide the large distribution system into smaller areas of subtree structure, and by jointly exploring the linearized power flow model and the network topology, we propose a gradient-based multi-area algorithm to exactly and efficiently solve the WLS problem. The proposed algorithm enables distributed and parallel computation of the state estimation problem without compromising any performance. Numerical results on a 4,521-node test feeder show that the designed algorithm features fast convergence and accurate estimation results. Comparison with traditional Gauss-Newton method shows that the proposed method has much better performance in distribution systems with a limited amount of reliable measurement. The real-time implementation of the algorithm tracks time-varying system states with high accuracy.

41 EE - Solar Energy Technologies Office (EE-4S)↗

Reducing Communication Costs for Sparse Matrix Multiplication within Algebraic Multigrid

We consider the sequence of sparse matrix-matrix multiplications performed during the setup phase of algebraic multigrid. In particular, we show that the most commonly used parallel algorithm is often not the most communication-efficient one for all of the matrix-matrix multiplications involved. By using an alternative algorithm, we show that the communication costs are reduced (in theory and practice), and we demonstrate the performance benefit for both model (structured) and more realistic unstructured problems on large-scale distributed-memory parallel systems. Our theoretical analysis shows that we can reduce communication by a factor of up to 5.4 for a model problem, and we observe in our empirical evaluation communication reductions of factors up to 4.7 for structured problems and 3.7 for unstructured problems. These reductions in communication translate to run-time speedups of up to factors of 2.3 and 2.5, respectively.

97 MATHEMATICS AND COMPUTING↗

Plasma Science and Innovation Center at Washington, Wisconsin, and Utah State: Final Scientific Report for the University of Wisconsin-Madison

The University of Wisconsin-Madison component of the Plasma Science and Innovation Center (PSI Center) contributed to modeling capabilities and algorithmic efficiency of the Non-Ideal Magnetohydrodynamics with Rotation (NIMROD) Code, which is widely used to model macroscopic dynamics of magnetically confined plasma. It also contributed to the understanding of direct-current (DC) injection of electrical current for initiating and sustaining plasma in three spherical torus experiments: the Helicity Injected Torus-II (HIT-II), the Pegasus Toroidal Experiment, and the National Spherical Torus Experiment (NSTX). The effort was funded through the PSI Center's cooperative agreement with the University of Washington and Utah State University over the period of March 1, 2005 - August 31, 2016. In addition to the computational and physics accomplishments, the Wisconsin effort contributed to the professional education of four graduate students and two postdoctoral research associates. The modeling for HIT-II and Pegasus was directly supported by the cooperative agreement, and contributions to the NSTX modeling were in support of work by Dr. Bickford Hooper, who was funded through a separate grant. Our primary contribution to model development is the implementation of detailed closure relations for collisional plasma. Postdoctoral associate Adam Bayliss implemented the temperature-dependent effects of Braginskii's parallel collisional ion viscosity. As a graduate student, John O'Bryan added runtime options for Braginskii's models and Ji's K2 models of thermal conduction with magnetization effects and thermal equilibration. As a postdoctoral associate, O'Bryan added the magnetization effects for ion viscosity. Another area of model development completed through the PSI-Center is the implementation of Chodura's phenomenological resistivity model. Finally, we investigated and tested linear electron parallel viscosity, leveraged by support from the Center for Extended Magnetohydrodynamic Modeling (CEMM). Work on algorithmic efficiency improved NIMROD's element-based computations. We reordered arrays and eliminated a level of looping for computations over the data points that are used for numerical integration over elements. Moreover, the reordering allows fewer and larger communication calls when using distributed-memory parallel computation, thereby avoiding a data starvation problem that limited parallel scaling over NIMROD's Fourier components for the periodic coordinate. Together with improved parallel preconditioning, work that was supported by CEMM, these developments allowed NIMROD's first scaling to over 10,000 processor cores. Another algorithm improvement supported by the PSI Center is nonlinear numerical diffusivities for implicit advection. We also developed the Stitch code to enhance the flexibility of NIMROD's preprocessing. Our simulations of HIT-II considered conditions with and without fluctuation-induced amplification of poloidal flux, but our validation efforts focused on conditions without amplification. A significant finding is that NIMROD reproduces the dependence of net plasma current as the imposed poloidal flux is varied. The modeling of Pegasus startup from localized DC injectors predicted that development of a tokamak-like configuration occurs through a sequence of current-filament merger events. Comparison of experimentally measured and numerically computed cross-power spectra enhance confidence in NIMROD's simulation of magnetic fluctuations; however, energy confinement remains an open area for further research. Our contributions to the NSTX study include adaptation of the helicity-injection boundary conditions from the HIT-II simulations and support for linear analysis and computation of 3D current-driven instabilities.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Zonal magnetic fields regulate nonlinear edge-localized-mode dynamics via self-consistent force balance

Edge-localized modes (ELMs) eject intense bursts of heat and particles that threaten plasma-facing components in fusion reactors. Nonlinear full-torus BOUT++ simulations show that turbulence-driven zonal magnetic fields (ZMFs) play an essential role in nonlinear ELM evolution by maintaining self-consistent force balance. Zonal flows mitigate the initial crash through shear but do not prevent continued radial transport. When ZMFs are self-consistently included, turbulence-driven zonal currents modify the parallel current distribution and magnetic tension and are associated with a reduction of the axisymmetric (𝑛 = 0) perturbed radial force imbalance. This coincides with a transition from convective, bursty propagation to more localized, diffusive transport. Similar behavior is observed across the regimes considered, including both resistive-ballooning and peeling-ballooning cases. Finally, associated signatures, including radial electric field shear and parallel current redistribution, provide experimentally accessible diagnostics for present devices and ITER-relevant conditions.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Unraveling the excitation mechanisms of highly oblique lower-band chorus waves

Excitation mechanisms of highly oblique, quasi-electrostatic lower band chorus waves are investigated using Van Allen Probes observations near the equator of the Earth's magnetosphere. Linear growth rates are evaluated based on in situ, measured electron velocity distributions and plasma conditions and compared with simultaneously observed wave frequency spectra and wave normal angles. Accordingly, two distinct excitation mechanisms of highly oblique lower band chorus have been clearly identified for the first time. The first mechanism relies on cyclotron resonance with electrons possessing both a realistic temperature anisotropy at keV energies and a plateau at 100–500 eV in the parallel velocity distribution. The second mechanism corresponds to Landau resonance with a 100–500 eV beam. In both cases, a small low-energy beam-like component is necessary for suppressing an otherwise dominating Landau damping. In conclusion, our new findings suggest that small variations in the electron distribution could have important impacts on energetic electron dynamics.

58 GEOSCIENCES↗

Poster — Thur Eve — 74: Distributed, asynchronous, reactive dosimetric and outcomes analysis using DICOMautomaton

Many have speculated about the future of computational technology in clinical radiation oncology. It has been advocated that the next generation of computational infrastructure will improve on the current generation by incorporating richer aspects of automation, more heavily and seamlessly featuring distributed and parallel computation, and providing more flexibility toward aggregate data analysis. In this report we describe how a recently created — but currently existing — analysis framework (DICOMautomaton) incorporates these aspects. DICOMautomaton supports a variety of use cases but is especially suited for dosimetric outcomes correlation analysis, investigation and comparison of radiotherapy treatment efficacy, and dose-volume computation. We describe: how it overcomes computational bottlenecks by distributing workload across a network of machines; how modern, asynchronous computational techniques are used to reduce blocking and avoid unnecessary computation; and how issues of out-of-date data are addressed using reactive programming techniques and data dependency chains. We describe internal architecture of the software and give a detailed demonstration of how DICOMautomaton could be used to search for correlations between dosimetric and outcomes data.

60 APPLIED LIFE SCIENCES↗

LDRD final report : managing shared memory data distribution in hybrid HPC applications.

MPI is the dominant programming model for distributed memory parallel computers, and is often used as the intra-node programming model on multi-core compute nodes. However, application developers are increasingly turning to hybrid models that use threading within a node and MPI between nodes. In contrast to MPI, most current threaded models do not require application developers to deal explicitly with data locality. With increasing core counts and deeper NUMA hierarchies seen in the upcoming LANL/SNL 'Cielo' capability supercomputer, data distribution poses an upper boundary on intra-node scalability within threaded applications. Data locality therefore has to be identified at runtime using static memory allocation policies such as first-touch or next-touch, or specified by the application user at launch time. We evaluate several existing techniques for managing data distribution using micro-benchmarks on an AMD 'Magny-Cours' system with 24 cores among 4 NUMA domains and argue for the adoption of a dynamic runtime system implemented at the kernel level, employing a novel page table replication scheme to gather per-NUMA domain memory access traces.

99 GENERAL AND MISCELLANEOUS↗

Distributed Data-Driven Optimization for Voltage Regulation in Distribution Systems

Here, this paper proposes a distributed data-driven optimization framework for voltage regulation in distribution systems. The recursive kernel regression and alternating direction method of multipliers (ADMM) are selected to cover the system learning and distributed optimization tasks. The proposed distributed data-driven framework is capable of having a rapid response to system or load changes while considering the operation optimality. Besides, the distributed algorithm parallels the computation tasks and reduces the computational expense of a single agent. To validate the performance of the proposed method, a hypothetical 7-Bus system and the IEEE 123-Bus system are selected to show the effectiveness of the proposed data-driven framework. According to the numerical study results, the proposed method offers great flexibility for selecting customized kernel models for different regions and can effectively improve the system voltage profile in a distributed manner.

24 POWER TRANSMISSION AND DISTRIBUTION↗

MASSIVE HYBRID PARALLELISM FOR FULLY IMPLICIT MULTIPHYSICS

As hardware advances continue to modify the supercomputing landscape, traditional scientific software development practices will become more outdated, ineffective, and inefficient. The process of rewriting/retooling existing software for new architectures is a Sisyphean task, and results in substantial hours of development time, effort, and money. Software libraries which provide an abstraction of the resources provided by such architectures are therefore essential if the computational engineering and science communities are to continue to flourish in this modern computing environment. The Multiphysics Object Oriented Simulation Environment (MOOSE) framework enables complex multiphysics analysis tools to be built rapidly by scientists, engineers, and domain specialists, while also allowing them to both take advantage of current HPC architectures, and efficiently prepare for future supercomputer designs. MOOSE employs a hybrid shared-memory and distributed-memory parallel model and provides a complete and consistent interface for creating multiphysics analysis tools. In this paper, a brief discussion of the mathematical algorithms underlying the framework and the internal object-oriented hybrid parallel design are given. Representative massively parallel results from several applications areas are presented, and a brief discussion of future areas of research for the framework are provided.

97 MATHEMATICS AND COMPUTING↗

Massive hybrid parallelism for fully implicit multiphysics

As hardware advances continue to modify the supercomputing landscape, traditional scientific software development practices will become more outdated, ineffective, and inefficient. The process of rewriting/retooling existing software for new architectures is a Sisyphean task, and results in substantial hours of development time, effort, and money. Software libraries which provide an abstraction of the resources provided by such architectures are therefore essential if the computational engineering and science communities are to continue to flourish in this modern computing environment. The Multiphysics Object Oriented Simulation Environment (MOOSE) framework enables complex multiphysics analysis tools to be built rapidly by scientists, engineers, and domain specialists, while also allowing them to both take advantage of current HPC architectures, and efficiently prepare for future supercomputer designs. MOOSE employs a hybrid shared-memory and distributed-memory parallel model and provides a complete and consistent interface for creating multiphysics analysis tools. In this paper, a brief discussion of the mathematical algorithms underlying the framework and the internal object-oriented hybrid parallel design are given. Representative massively parallel results from several applications areas are presented, and a brief discussion of future areas of research for the framework are provided. (authors)

97 MATHEMATICS AND COMPUTING↗

Class network routing

Class network routing is implemented in a network such as a computer network comprising a plurality of parallel compute processors at nodes thereof. Class network routing allows a compute processor to broadcast a message to a range (one or more) of other compute processors in the computer network, such as processors in a column or a row. Normally this type of operation requires a separate message to be sent to each processor. With class network routing pursuant to the invention, a single message is sufficient, which generally reduces the total number of messages in the network as well as the latency to do a broadcast. Class network routing is also applied to dense matrix inversion algorithms on distributed memory parallel supercomputers with hardware class function (multicast) capability. This is achieved by exploiting the fact that the communication patterns of dense matrix inversion can be served by hardware class functions, which results in faster execution times.

97 MATHEMATICS AND COMPUTING↗

A fast parallel Poisson solver on irregular domains applied to beam dynamics simulations

We discuss the scalable parallel solution of the Poisson equation within a Particle-In-Cell (PIC) code for the simulation of electron beams in particle accelerators of irregular shape. The problem is discretized by Finite Differences. Depending on the treatment of the Dirichlet boundary the resulting system of equations is symmetric or 'mildly' nonsymmetric positive definite. In all cases, the system is solved by the preconditioned conjugate gradient algorithm with smoothed aggregation (SA) based algebraic multigrid (AMG) preconditioning. We investigate variants of the implementation of SA-AMG that lead to considerable improvements in the execution times. We demonstrate good scalability of the solver on distributed memory parallel processor with up to 2048 processors. We also compare our iterative solver with an FFT-based solver that is more commonly used for applications in beam dynamics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

SOMAFOAM: An OpenFOAM based solver for continuum simulations of low-temperature plasmas

Here, we report the development of SOMAFOAM, a finite volume framework for performing continuum simulations of low-temperature plasmas. The primary goal of this work is to discuss the features of SOMAFOAM along with representative results provided as examples for a range of operating conditions and geometries. This includes plasma and plasma–dielectric systems operating in direct current, radio frequency, and microwave regimes from pressures as low as 100 mTorr to atmospheric pressure. The code has several useful features including the ability to run massively parallel simulations using arbitrary geometries, structured/unstructured meshes, choice of models such as drift–diffusion/full-momentum at runtime, and species-dependent timesteps to name a few. The verification/validation studies presented include comparison with previously published continuum simulations (low-pressure direct current and radio frequency plasma), with experiments (Gaseous Electronics Conference Reference Cell and microwave microplasma ignited in a split ring resonator), and previously published kinetic simulations (low-pressure radio frequency plasma). Other examples provided include a direct current atmospheric pressure microplasma bounded by dielectric sidewalls and a helium–nitrogen plasma ignited using a needle electrode facing a dielectric. The performance of the code is also discussed with serial and distributed memory parallel runs demonstrated up to 512 cores. The design and implementation of the code in a modular object-oriented framework allows for easy extension and seamless coupling with other codes and can be expected to play an important role in both academia and industry.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

The YNP metagenome project: environmental parameters responsible for microbial distribution in the Yellowstone geothermal ecosystem

The Yellowstone geothermal complex contains over 10,000 diverse geothermal features that host numerous phylogenetically deeply rooted and poorly understood archaea, bacteria, and viruses. Microbial communities in high-temperature environments are generally less diverse than soil, marine, sediment, or lake habitats and therefore offer a tremendous opportunity for studying the structure and function of different model microbial communities using environmental metagenomics. One of the broader goals of this study was to establish linkages among microbial distribution, metabolic potential, and environmental variables. Twenty geochemically distinct geothermal ecosystems representing a broad spectrum of Yellowstone hot-spring environments were used for metagenomic and geochemical analysis and included approximately equal numbers of: (1) phototrophic mats, (2) “filamentous streamer” communities, and (3) archaeal-dominated sediments. The metagenomes were analyzed using a suite of complementary and integrative bioinformatic tools, including phylogenetic and functional analysis of both individual sequence reads and assemblies of predominant phylotypes.This volume identifies major environmental determinants of a large number of thermophilic microbial lineages, many of which have not been fully described in the literature nor previously cultivated to enable functional and genomic analyses. Moreover, protein family abundance comparisons and in-depth analyses of specific genes and metabolic pathways relevant to these hot-spring environments reveal hallmark signatures of metabolic capabilities that parallel the distribution of phylotypes across specific types of geochemical environments.

59 BASIC BIOLOGICAL SCIENCES↗

Parallel Fokker–Planck-DSMC algorithm for rarefied gas flow simulation in complex domains at all Knudsen numbers

A major challenge for the conventional Direct Simulation Monte Carlo (DSMC) technique lies in the fact that its computational cost becomes prohibitive in the near continuum regime, where the Knudsen number (Kn)—characterizing the degree of rarefaction—becomes small. In contrast, the Fokker–Planck (FP) based particle Monte Carlo scheme allows for computationally efficient simulations of rarefied gas flows in the low and intermediate Kn regime. The Fokker–Planck collision operator—instead of performing binary collisions employed by the DSMC method—integrates continuous stochastic processes for the phase space evolution in time. This allows for time step and grid cell sizes larger than the respective collisional scales required by DSMC. Dynamically switching between the FP and the DSMC collision operators in each computational cell is the basis of the combined FP-DSMC method, which has been proven successful in simulating flows covering the whole Kn range. Until recently, this algorithm had only been applied to two-dimensional test cases. In this contribution, we present the first general purpose implementation of the combined FP-DSMC method. Utilizing both shared- and distributed-memory parallelization, this implementation provides the capability for simulations involving many particles and complex geometries by exploiting state of the art computer cluster technologies.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Unit commitment with wind power generation: integrating wind forecast uncertainty and stochastic programming.

We present a computational framework for integrating the state-of-the-art Weather Research and Forecasting (WRF) model in stochastic unit commitment/energy dispatch formulations that account for wind power uncertainty. We first enhance the WRF model with adjoint sensitivity analysis capabilities and a sampling technique implemented in a distributed-memory parallel computing architecture. We use these capabilities through an ensemble approach to model the uncertainty of the forecast errors. The wind power realizations are exploited through a closed-loop stochastic unit commitment/energy dispatch formulation. We discuss computational issues arising in the implementation of the framework. In addition, we validate the framework using real wind speed data obtained from a set of meteorological stations. We also build a simulated power system to demonstrate the developments.

17 WIND ENERGY↗

Reassessing energy deposition for the ITER 5 MA vertical displacement event with an improved DINA model

The beryllium (Be) main chamber wall interaction during a 5 MA/1.8 T upward, unmitigated VDE scenario, first analysed in [J. Coburn et al., Phys. Scr. T171 (2020) 014076] for ITER, has been re-evaluated using the latest energy deposition analysis software. Updates to the DINA disruption model are summarized, including an improved numerical convergence for the 0D power balance, limitations on the safety factor within the plasma core, and the choice to maintain a constant plasma +halo poloidal cross-section. Such updates result in a broad halo region and higher radiated power fractions compared to previous models. The new scenario lasts for ~75 ms and deposits ~29 MJ of energy, with the radial distribution of parallel heat flux q‖(r)resembling an exponential falloff with an effective λ_q=75-198 mm. A maximum halo width w_h of 0.52 m at the outboard midplane is observed. SMITER field line tracing and energy deposition simulations calculate a q_{⊥,max} of ~83 MW/m^2 on the upper first wall panels (FWP). Heat transfer calculations with the MEMOS-U code show that the FWP surface temperature reaches ~1000 K, well below the Be melt threshold. Variations of this 5 MA scenario with Be im-purity densities from 0 to 3∙10^19 m^-3 also remain below the melt threshold despite differences in energy deposition and duration. These results are in contrast to the early study which predicted melt damage to the first wall [J. Coburn et al., Phys. Scr. T171 (2020) 014076], and emphasize the importance of accurate models for the halo width w_h and the heat flux distribution q‖(r)within that halo width. The 2020 halo model in DINA has been compared with halo current experiments on COMPASS, JET, and Alcator C-Mod, and the preliminary results build confidence in the broad halo width predictions. Results for the 5 MA VDE are compared with those for a 15 MA equivalent, generated using the new DINA model. At the higher current, significant melting of the upper FWP is to be expected.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗