Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

Programmability of the HPCS Languages: A Case Study with a Quantum Chemistry Kernel (Extended Version)

As high-end computer systems present users with rapidly increasing numbers of processors, possibly also incorporating attached co-processors, programmers are increasingly challenged to express the necessary levels of concurrency with the dominant parallel programming model, Fortran+MPI+OpenMP (or minor variations). In this paper, we examine the languages developed under the DARPA High-Productivity Computing Systems (HPCS) program (Chapel, Fortress, and X10) as representatives of a different parallel programming model which might be more effective on coming high-performance systems. The application used in this study is the Hartree-Fock method from quantum chemistry, which combines access to distributed data with a task-parallel algorithm and is characterized by significant irregularity in the computational tasks. We present several different implementation strategies for load balancing of the task parallel computation, as well as distributed array operations, in each of the three languages. We conclude that the HPCS languages provide a wide variety of mechanisms for expressing parallelism, which can be combined at multiple levels, making them quite expressive for this problem.

97 MATHEMATICS AND COMPUTING↗

Exploring Container Technologies for Large Scientific Libraries: Docker and Trilinos

The Trilinos Project produces, maintains and distributes a large collection of reusable, parallel scientific libraries. Docker provides container technologies that support compilation, packaging, distribution and execution of software on Linux, Mac OS and Windows systems, with emerging support for Cray platforms. In this short article we describe recent efforts to explore the potential for using Docker in a variety of settings to enhance several Trilinos Project workflows. The technical foundation for this article is presented in an Honors thesis of one of the authors.

97 MATHEMATICS AND COMPUTING↗

Uranium Chemical and Radiological Risk Assessment for Freshwater Ecosystems Receiving Ore Mining Releases: Principles, Equations and Parameters

Uranium is an element that has the solely characteristic to behave as significant hazard both from a chemical and radiological point of view. Exclusively of natural occurrence, its distribution into the environment may be influenced by human activities, such as nuclear fuel cycle, military use of depleted uranium, or coal and phosphate fertilizer use, which finally may impact freshwater ecosystems. Until now, the associated environmental impact and risk assessments were conducted separately. We propose here to apply the same methodology to evaluate the ecological risk due to potential chemotoxicity and radiotoxicity of uranium. This methodology is articulated into the classical four steps (EC, 2003: problem formulation, effect and exposure analysis, risk characterisation). The problem formulation dealt both with uranium viewed as a chemical element and as the three isotopes 234, 235 and 238 of uranium and their main daughters. Then, the exposure analysis of non-human species was led on the basis of a common conceptual model of the fluxes occurring in freshwater ecosystems. No-effect values for the ecosystem were derived using the same effect data treatment in parallel. A Species Sensitivity Distribution was fitted: (1) to the ecotoxicity data sets illustrating uranium chemotoxicity and allowing the estimation of a Predicted-No-Effect-Concentration for uranium in water expressed in {mu}g/L; (2) to radiotoxicity effect data as it was done within the ERICA project, allowing the estimation of a Predicted No-Effect-Dose-Rate (in {mu}Gy{center_dot}h{sup -1}). Two methods were then applied to characterize the risk to the ecosystem: a screening method using the risk quotient approach, involving for the radiological aspect back calculation of the water limiting concentration from the PNEDR for each isotope taken into account and a probabilistic risk assessment. A former uranium ore mining case-study will help in demonstrating the application of the whole methodology.

54 ENVIRONMENTAL SCIENCES↗

COHORT: Coordination of Heterogeneous Thermostatically Controlled Loads for Demand Flexibility

Demand flexibility is increasingly important for power grids. Careful coordination of thermostatically controlled loads (TCLs) can modulate energy demand, decrease operating costs, and increase grid resiliency. We propose a novel distributed control framework for the Coordination Of HeterOgeneous Residential Thermostatically controlled loads (COHORT). COHORT is a practical, scalable, and versatile solution that coordinates a population of TCLs to jointly optimize a grid-level objective, while satisfying each TCL’s end-use requirements and operational constraints. To achieve that, we decompose the grid-scale problem into subproblems and coordi- nate their solutions to find the global optimum using the alternating direction method of multipliers (ADMM). The TCLs’ local problems are distributed to and computed in parallel at each TCL, making COHORT highly scalable and privacy-preserving. While each TCL poses combinatorial and non-convex constraints, we characterize these constraints as a convex set through relaxation, thereby making COHORT computationally viable over long planning horizons. After coordination, each TCL is responsible for its own control and tracks the agreed-upon power trajectory with its preferred strategy. In this work, we translate continuous power back to discrete on/off actuation, using pulse width modulation. COHORT is generalizable to a wide range of grid objectives, which we demonstrate through three distinct use cases: generation following, minimizing ramping, and peak load curtailment. In a notable experiment, we validated our approach through a hardware-in-the-loop simulation, including a real-world air conditioner (AC) controlled via a smart thermostat, and simulated instances of ACs modeled after real-world data traces. During the 15-day experimental period, COHORT reduced daily peak loads by an average of 12.5% and maintained comfortable temperatures.

demand response↗

Method and structure for skewed block-cyclic distribution of lower-dimensional data arrays in higher-dimensional processor grids

A method and structure of distributing elements of an array of data in a computer memory to a specific processor of a multi-dimensional mesh of parallel processors includes designating a distribution of elements of at least a portion of the array to be executed by specific processors in the multi-dimensional mesh of parallel processors. The pattern of the designating includes a cyclical repetitive pattern of the parallel processor mesh, as modified to have a skew in at least one dimension so that both a row of data in the array and a column of data in the array map to respective contiguous groupings of the processors such that a dimension of the contiguous groupings is greater than one.

97 MATHEMATICS AND COMPUTING↗

On the Efficient Evaluation of the Exchange Correlation Potential on Graphics Processing Unit Clusters

The predominance of Kohn–Sham density functional theory (KS-DFT) for the theoretical treatment of large experimentally relevant systems in molecular chemistry and materials science relies primarily on the existence of efficient software implementations which are capable of leveraging the latest advances in modern high-performance computing (HPC). With recent trends in HPC leading toward increasing reliance on heterogeneous accelerator-based architectures such as graphics processing units (GPU), existing code bases must embrace these architectural advances to maintain the high levels of performance that have come to be expected for these methods. In this work, we purpose a three-level parallelism scheme for the distributed numerical integration of the exchange-correlation (XC) potential in the Gaussian basis set discretization of the Kohn–Sham equations on large computing clusters consisting of multiple GPUs per compute node. In addition, we purpose and demonstrate the efficacy of the use of batched kernels, including batched level-3 BLAS operations, in achieving high levels of performance on the GPU. We demonstrate the performance and scalability of the implementation of the purposed method in the NWChemEx software package by comparing to the existing scalable CPU XC integration in NWChem.

97 MATHEMATICS AND COMPUTING↗

Final report on LDRD project : advanced optical trigger systems.

Advanced optically-activated solid-state electrical switch development at Sandia has demonstrated multi-kA/kV switching and the path for scalability to even higher current/power. Realization of this potential requires development of new optical sources/switches based on key Sandia photonic device technologies: vertical-cavity surface-emitting lasers (VCSELs) and photoconductive semiconductor switch (PCSS) devices. The key to increasing the switching capacity of PCSS devices to 5kV/5kA and higher is to distribute the current in multiple parallel line filaments triggered by an array of high-brightness line-shaped illuminators. Commercial mechanically-stacked edge-emitting lasers have been used to trigger multiple filaments, but they are difficult to scale and manufacture with the required uniformity. In VCSEL arrays, adjacent lasers utilize identical semiconductor material and are lithographically patterned to the required dimensions. We have demonstrated multiple-line filament triggering using VCSEL arrays to approximate line generation. These arrays of uncoupled circular-aperture VCSELs have fill factors ranging from 2% to 30%. Using these arrays, we have developed a better understanding of the illumination requirements for stable triggering of multiple-filament PCSS devices. Photoconductive semiconductor switch (PCSS) devices offer advantages of high voltage operation (multi-kV), optical isolation, triggering with laser pulses that cannot occur accidentally in nature, low cost, high speed, small size, and radiation hardness. PCSS devices are candidates for an assortment of potential applications that require multi-kA switching of current. The key to increasing the switching capacity of PCSS devices to 5kV/5kA and higher is to distribute the current in multiple parallel line filaments triggered by an array of high-brightness line-shaped illuminators. Commercial mechanically-stacked edge-emitting lasers have been demonstrated to trigger multiple filaments, but they are difficult to scale and manufacture with the required uniformity. As a promising alternative to multiple discrete edge-emitting lasers, a single wafer of vertical-cavity surface-emitting lasers (VCSELs) can be lithographically patterned to achieve the desired layout of parallel line-shaped emitters, in which adjacent lasers utilize identical semiconductor material and thereby achieve a degree of intrinsic optical uniformity. Under this LDRD project, we have fabricated arrays of uncoupled circular-aperture VCSELs to approximate a line-shaped illumination pattern, achieving optical fill factors ranging from 2% to 30%. We have applied these VCSEL arrays to demonstrate single and dual parallel line-filament triggering of PCSS devices. Moreover, we have developed a better understanding of the illumination requirements for stable triggering of multiple-filament PCSS devices using VCSEL arrays. We have found that reliable triggering of multiple filaments requires matching of the turn-on time of adjacent VCSEL line-shaped-arrays to within approximately 1 ns. Additionally, we discovered that reliable triggering of PCSS devices at low voltages requires more optical power than we obtained with our first generation of VCSEL arrays. A second generation of higher-power VCSEL arrays was designed and fabricated at the end of this LDRD project, and testing with PCSS devices is currently underway (as of September 2008).

36 MATERIALS SCIENCE↗

FTK: A Simplicial Spacetime Meshing Framework for Robust and Scalable Feature Tracking

In this work, we present the Feature Tracking Kit (FTK), a framework that simplifies, scales, and delivers various feature-tracking algorithms for scientific data. The key of FTK is our simplicial spacetime meshing scheme that generalizes both regular and unstructured spatial meshes to spacetime while tessellating spacetime mesh elements into simplices. The benefits of using simplicial spacetime meshes include (1) reducing ambiguity cases for feature extraction and tracking, (2) simplifying the handling of degeneracies using symbolic perturbations, and (3) enabling scalable and parallel processing. The use of simplicial spacetime meshing simplifies and improves the implementation of several feature-tracking algorithms for critical points, quantum vortices, and isosurfaces. As a software framework, FTK provides end users with VTK/ParaView filters, Python bindings, a command line interface, and programming interfaces for feature-tracking applications. We demonstrate use cases as well as scalability studies through both synthetic data and scientific applications including tokamak, fluid dynamics, and superconductivity simulations. We also conduct end-to-end performance studies on the Summit supercomputer. FTK is open sourced under the MIT license: https://github.com/hguo/ftk.

97 MATHEMATICS AND COMPUTING↗

PhytoOracle: Scalable, modular phenomics data processing pipelines

As phenomics data volume and dimensionality increase due to advancements in sensor technology, there is an urgent need to develop and implement scalable data processing pipelines. Current phenomics data processing pipelines lack modularity, extensibility, and processing distribution across sensor modalities and phenotyping platforms. To address these challenges, we developed PhytoOracle (PO), a suite of modular, scalable pipelines for processing large volumes of field phenomics RGB, thermal, PSII chlorophyll fluorescence 2D images, and 3D point clouds. PhytoOracle aims to ( i ) improve data processing efficiency; ( ii ) provide an extensible, reproducible computing framework; and ( iii ) enable data fusion of multi-modal phenomics data. PhytoOracle integrates open-source distributed computing frameworks for parallel processing on high-performance computing, cloud, and local computing environments. Each pipeline component is available as a standalone container, providing transferability, extensibility, and reproducibility. The PO pipeline extracts and associates individual plant traits across sensor modalities and collection time points, representing a unique multi-system approach to addressing the genotype-phenotype gap. To date, PO supports lettuce and sorghum phenotypic trait extraction, with a goal of widening the range of supported species in the future. At the maximum number of cores tested in this study (1,024 cores), PO processing times were: 235 minutes for 9,270 RGB images (140.7 GB), 235 minutes for 9,270 thermal images (5.4 GB), and 13 minutes for 39,678 PSII images (86.2 GB). These processing times represent end-to-end processing, from raw data to fully processed numerical phenotypic trait data. Repeatability values of 0.39-0.95 (bounding area), 0.81-0.95 (axis-aligned bounding volume), 0.79-0.94 (oriented bounding volume), 0.83-0.95 (plant height), and 0.81-0.95 (number of points) were observed in Field Scanalyzer data. We also show the ability of PO to process drone data with a repeatability of 0.55-0.95 (bounding area).

59 BASIC BIOLOGICAL SCIENCES↗

Reinforcement Learning for Load-balanced Parallel Particle Tracing

We explore an online reinforcement learning (RL) paradigm to dynamically optimize parallel particle tracing performance in distributed-memory systems. Our method combines three novel components: (1) a work donation algorithm, (2) a high-order workload estimation model, and (3) a communication cost model. First, we design an RL-based work donation algorithm. Our algorithm monitors workloads of processes and creates RL agents to donate data blocks and particles from high-workload processes to low-workload processes to minimize program execution time. The agents learn the donation strategy on the fly based on reward and cost functions designed to consider processes' workload changes and data transfer costs of donation actions. Second, we propose a workload estimation model, helping RL agents estimate the workload distribution of processes in future computations. Third, we design a communication cost model that considers both block and particle data exchange costs, helping RL agents make effective decisions with minimized communication costs. We demonstrate that our algorithm adapts to different flow behaviors in large-scale fluid dynamics, ocean, and weather simulation data. Our algorithm improves parallel particle tracing performance in terms of parallel efficiency, load balance, and costs of I/O and communication for evaluations with up to 16,384 processors.

Distributed and parallel particle tracing↗

Concurrent Collections (CnC): A new approach to parallel programming

A common approach in designing parallel languages is to provide some high level handles to manipulate the use of the parallel platform. This exposes some aspects of the target platform, for example, shared vs. distributed memory. It may expose some but not all types of parallelism, for example, data parallelism but not task parallelism. This approach must find a balance between the desire to provide a simple view for the domain expert and provide sufficient power for tuning. This is hard for any given architecture and harder if the language is to apply to a range of architectures. Either simplicity or power is lost. Instead of viewing the language design problem as one of providing the programmer with high level handles, we view the problem as one of designing an interface. On one side of this interface is the programmer (domain expert) who knows the application but needs no knowledge of any aspects of the platform. On the other side of the interface is the performance expert (programmer or program) who demands maximal flexibility for optimizing the mapping to a wide range of target platforms (parallel / serial, shared / distributed, homogeneous / heterogeneous, etc.) but needs no knowledge of the domain. Concurrent Collections (CnC) is based on this separation of concerns. The talk will present CnC and its benefits. About the speaker. Kathleen Knobe has focused throughout her career on parallelism especially compiler technology, runtime system design and language design. She worked at Compass (aka Massachusetts Computer Associates) from 1980 to 1991 designing compilers for a wide range of parallel platforms for Thinking Machines, MasPar, Alliant, Numerix, and several government projects. In 1991 she decided to finish her education. After graduating from MIT in 1997, she joined Digital Equipment’s Cambridge Research Lab (CRL). She stayed through the DEC/Compaq/HP mergers and when CRL was acquired and absorbed by Intel. She currently works in the Software and Services Group / Technology Pathfinding and Innovation.

97 MATHEMATICS AND COMPUTING↗

Full-wave simulations of ICRF heating regimes in toroidal plasma with non-Maxwellian distribution functions

At the power levels required for significant heating and current drive in magnetically-confined toroidal plasma, modification of the particle distribution function from a Maxwellian shape is likely (Stix 1975 Nucl. Fusion 15 737), with consequent changes in wave propagation and in the location and amount of absorption. In order to study these effects computationally, both the finite-Larmor-radius and the high-harmonic fast wave (HHFW), versions of the full-wave, hot-plasma toroidal simulation code TORIC (Brambilla 1999 Plasma Phys. Control. Fusion 41 1 and Brambilla 2002 Plasma Phys. Control. Fusion 44 2423), have been extended to allow the prescription of arbitrary velocity distributions of the form f(v(parallel to), v(perpendicular to) , psi, theta). For hydrogen (H) minority heating of a deuterium (D) plasma with anisotropic Maxwellian H distributions, the fractional H absorption varies significantly with changes in parallel temperature but is essentially independent of perpendicular temperature. On the other hand, for HHFW regime with anisotropic Maxwellian fast ion distribution, the fractional beam ion absorption varies mainly with changes in the perpendicular temperature. The evaluation of the wave-field and power absorption, through the full wave solver, with the ion distribution function provided by either a Monte-Carlo particle and Fokker-Planck codes is also examined for Alcator C-Mod and NSTX plasmas. Non-Maxwellian effects generally tend to increase the absorption with respect to the equivalent Maxwellian distribution.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Scalable training of trustworthy and energy-efficient predictive graph foundation models for atomistic materials modeling: a case study with HydraGNN

We present our work on developing and training scalable, trustworthy, and energy-efficient predictive graph foundation models (GFMs) using HydraGNN, a multi-headed graph convolutional neural network architecture. HydraGNN expands the boundaries of graph neural network (GNN) computations in both training scale and data diversity. It abstracts over message passing algorithms, allowing both reproduction of and comparison across algorithmic innovations that define nearest-neighbor convolution in GNNs. This work discusses a series of optimizations that have allowed scaling up the GFMs training to tens of thousands of GPUs on datasets consisting of hundreds of millions of graphs. Our GFMs use multitask learning (MTL) to simultaneously learn graph-level and node-level properties of atomistic structures, such as energy and atomic forces. Using over 154 million atomistic structures for training, we illustrate the performance of our approach along with the lessons learned on two state-of-the-art US Department of Energy (US-DOE) supercomputers, namely the Perlmutter petascale system at the National Energy Research Scientific Computing Center and the Frontier exascale system at Oak Ridge Leadership Computing Facility. The HydraGNN architecture enables the GFM to achieve near-linear strong scaling performance using more than 2000 GPUs on Perlmutter and 16,000 GPUs on Frontier.

97 MATHEMATICS AND COMPUTING↗

Molecular mechanism of bundle formation by the bacterial actin ParM

The actin homolog ParM plays a microtubule-like role in segregating DNA prior to bacterial cell division. Fluorescence and cryo-electron microscopy have shown that ParM forms filament bundles between separating DNA plasmids in vivo. Given the lack of ParM bundling proteins it remains unknown how ParM bundles form at the molecular level. Here we show using time-lapse TIRF microscopy, under in vitro molecular crowding conditions, that ParM-bundle formation consists of two distinct phases. At the onset of polymerization bundle thickness and shape are determined in the form of nuclei of short helically disordered filaments arranged in a liquid-like lattice. These nuclei then undergo an elongation phase whereby they rapidly increase in length. At steady state, ParM bundles fuse into one single large aggregate. This behavior had been predicted by theory but has not been observed for any other cytomotive biopolymer, including F-actin. We employed electron micrographs of ParM rafts, which are 2-D analogs of 3-D bundles, to identify the main molecular interfilament contacts within these suprastructures. The interface between filaments is similar for both parallel and anti-parallel orientations and the distribution of filament polarity is random within a bundle. We suggest that the interfilament interactions are not due to the interactions of specific residues but rather to long-range, counter ion mediated, electrostatic attractive forces. A randomly oriented bundle ensures that the assembly is rigid and that DNA may be captured with equal efficiency at both ends of the bundle via the ParR binding protein.

60 APPLIED LIFE SCIENCES↗

Axially-deformed solution of the Skyrme-Hartree-Fock-Bogoliubov equations using the transformed harmonic oscillator basis (IV) HFBTHO (v4.0): A new version of the program

We describe the new version 4.0 of the code HFBTHO that solves the nuclear Hartree-Fock-Bogoliubov problem by using the deformed harmonic oscillator basis in cylindrical coordinates. In the new version, we have implemented the restoration of rotational, particle number, and reflection symmetry for even-even nuclei. The restoration of rotational symmetry does not require using bases closed under rotation. Furthermore, we added the SeaLL1 functional and improved the calculation of the Coulomb potential. Finally, we refactored the code to facilitate maintenance and future developments.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Performance upgrades to the MCNP6 burnup capability for large scale depletion calculations

The first MCNP based inline Monte Carlo depletion capability was officially released from the Radiation Safety Information and Computational Center as MCNPX 2.6.0. With the merger of MCNPX and MCNP5, MCNP6 combined the capability of both simulation tools, as well as providing new advanced technology, in a single radiation transport code. The new MCNP6 depletion capability was first showcased at the International Congress for Advancements in Nuclear Power Plants (ICAPP) meeting in 2012. At that conference the new capabilities addressed included the combined distributive and shared memory parallel architecture for the burnup capability, improved memory management, physics enhancements, and new predictability as compared to the H.B Robinson Benchmark. At Los Alamos National Laboratory, a special purpose cluster named “tebow,” was constructed such to maximize available RAM per CPU, as well as leveraging swap space with solid state hard drives, to allow larger scale depletion calculations (allowing for significantly more burnable regions than previously examined). As the MCNP6 burnup capability was scaled to larger numbers of burnable regions, a noticeable slowdown was realized.This paper details two specific computational performance strategies for improving calculation speedup: (1) retrieving cross sections during transport; and (2) tallying mechanisms specific to burnup in MCNP. To combat this slowdown new performance upgrades were developed and integrated into MCNP6 1.2.

42 ENGINEERING↗

Self-consistent nonlinear kinetic simulations of the anomalous Doppler instability of suprathermal electrons in plasmas

Suprathermal tails in the distributions of electron velocities parallel to the magnetic field are found in many areas of plasma physics, from magnetic confinement fusion to solar system plasmas. Parallel electron kinetic energy can be transferred into plasma waves and perpendicular gyration energy of particles through the anomalous Doppler instability (ADI), provided that energetic electrons with parallel velocities v{sub ||}≥(ω+Ω{sub ce})/k{sub ||} are present; here Ω{sub ce} denotes electron cyclotron frequency, ω the wave angular frequency, and k{sub ||} the component of wavenumber parallel to the magnetic field. This phenomenon is widely observed in tokamak plasmas. Here, we present the first fully self-consistent relativistic particle-in-cell simulations of the ADI, spanning the linear and nonlinear regimes of the ADI. We test the robustness of the analytical theory in the linear regime and follow the ADI through to the steady state. By directly evaluating the parallel and perpendicular dynamical contributions to j·E in the simulations, we follow the energy transfer between the excited waves and the bulk and tail electron populations for the first time. We find that the ratio Ω{sub ce}/(ω{sub pe}+Ω{sub ce}) of energy transfer between parallel and perpendicular, obtained from linear analysis, does not apply when damping is fully included, when we find it to be ω{sub pe}/(ω{sub pe}+Ω{sub ce}); here ω{sub pe} denotes the electron plasma frequency. We also find that the ADI can arise beyond the previously expected range of plasma parameters, in particular when Ω{sub ce}>ω{sub pe}. The simulations also exhibit a spectral feature which may correspond to the observations of suprathermal narrowband emission at ω{sub pe} detected from low density tokamak plasmas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Linear dispersion properties of ring velocity distribution functions

Linear properties of ring velocity distribution functions are investigated. The dispersion tensor in a form similar to the case of a Maxwellian distribution function, but for a general distribution function separable in velocities, is presented. Analytical forms of the dispersion tensor are derived for two cases of ring velocity distribution functions: one obtained from physical arguments and one for the usual, ad hoc ring distribution. The analytical expressions involve generalized hypergeometric, Kampé de Fériet functions of two arguments. For a set of plasma parameters, the two ring distribution functions are compared. At the parallel propagation with respect to the ambient magnetic field, the two ring distributions give the same results identical to the corresponding bi-Maxwellian distribution. At oblique propagation, the two ring distributions give similar results only for strong instabilities, whereas for weak growth rates their predictions are significantly different; the two ring distributions have different marginal stability conditions.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗