Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

Scalable Data Management of the Uintah Simulation Framework for Next-Generation Engineering Problems with Radiation

The need to scale next-generation industrial engineering problems to the largest computational platforms presents unique challenges. This paper focuses on data management related problems faced by the Uintah simulation framework at a production scale of 260K processes. Uintah provides a highly scalable asynchronous many-task runtime system, which in this work is used for the modeling of a 1000 megawatt electric (MWe) ultra-supercritical (USC) coal boiler. At 260K processes, we faced both parallel I/O and visualization related challenges, e.g., the default file-per-process I/O approach of Uintah did not scale on Mira. In this paper we present a simple to implement, restructuring based parallel I/O technique. We impose a restructuring step that alters the distribution of data among processes. The goal is to distribute the dataset such that each process holds a larger chunk of data, which is then written to a file independently. This approach finds a middle ground between two of the most common parallel I/O schemes–file per process I/O and shared file I/O–in terms of both the total number of generated files, and the extent of communication involved during the data aggregation phase. To address scalability issues when visualizing the simulation data, we developed a lightweight renderer using OSPRay, which allows scientists to visualize the data interactively at high quality and make production movies. Finally, this work presents a highly efficient and scalable radiation model based on the sweeping method, which significantly outperforms previous approaches in Uintah, like discrete ordinates. The integrated approach allowed the USC boiler problem to run on 260K CPU cores on Mira.

97 MATHEMATICS AND COMPUTING↗

Electromagnetic fluctuations and normal modes of a drifting relativistic plasma

We present an exact calculation of the power spectrum of the electromagnetic fluctuations in a relativistic equilibrium plasma described by Maxwell-Jüttner distribution functions. We consider the cases of wave vectors parallel or normal to the plasma mean velocity. The relative contributions of the subluminal and supraluminal fluctuations are evaluated. Analytical expressions of the spatial fluctuation spectra are derived in each case. These theoretical results are compared to particle-in-cell simulations, showing a good reproduction of the subluminal fluctuation spectra.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Analyzing and visualizing next generation climate data (Final Report)

The project "Analyzing and visualizing next generation climate data" adds block-structured (mosaic) grid support, parallel processing, and 2D/3D curvilinear interpolation to the open-source UV-CDAT climate data analysis tool. Block structured grid support complies to the Gridspec extension submitted to the Climate and Forecast metadata conventions. It contains two parts: aggregation of data spread over multiple mosaic tiles (M-SPEC) and aggregation of temporal data stored in different files (F-SPEC). Together, M-SPEC and F-SPEC allow users to interact with data stored in multiple files as if the data were in a single file. For computational expensive tasks, a flexible, multi-dimensional, multi-type distributed array class allows users to process data in parallel using remote memory access. Both nodal and cell based interpolation is supported; users can choose between different interpolation libraries including ESMF and LibCF depending on the their particular needs.

54 ENVIRONMENTAL SCIENCES↗

Distributed memory, GPU accelerated Fock construction for hybrid, Gaussian basis density functional theory

With the growing reliance of modern supercomputers on accelerator-based architecture such a graphics processing units (GPUs), the development and optimization of electronic structure methods to exploit these massively parallel resources has become a recent priority. While significant strides have been made in the development GPU accelerated, distributed memory algorithms for many modern electronic structure methods, the primary focus of GPU development for Gaussian basis atomic orbital methods has been for shared memory systems with only a handful of examples pursing massive parallelism. Here in this work, we present a set of distributed memory algorithms for the evaluation of the Coulomb and exact exchange matrices for hybrid Kohn–Sham DFT with Gaussian basis sets via direct density-fitted (DF-J-Engine) and seminumerical (sn-K) methods, respectively. The absolute performance and strong scalability of the developed methods are demonstrated on systems ranging from a few hundred to over one thousand atoms using up to 128 NVIDIA A100 GPUs on the Perlmutter supercomputer.

97 MATHEMATICS AND COMPUTING↗

Development of Large-Dimension Configuration-Interaction Shell-Model Code

The project period was devoted to several developments in the technical capabilities of the BIGSTIC large-dimension configuration-interaction shell-model code, written in Fortran 90. The specific computational goals for the project period were: (1) store Lanczos vectors on core in RAM to minimize I/O; (2) rewrite reorthogonalization with Lanczos vectors stored in core, consult with personnel at LLNL, LBL, ORNL, Iowa State University to maximize performance; (3) restrict creation of N-body jumps to those needed by an individual node; and (4) distribute 3-body interaction over many cores. Significant progress was made towards these goals, especially (1) and (2), although in the process they discovered intermediate tasks that had to be accomplished first. The achievements were as follows - I put into place structures and algorithms to facility fragmenting very large-dimension Lanczos intermediate vectors. Only by fragmenting the vectors can we carry out (1) and (2). In addition, I reorganized the action of the Hamiltonian matrix and created a new division of operations for MPI. Based upon earlier work, I made plans of a revised algorithm for distribution of work with MPI, with a particular eye towards breaking up the Lanczos vectors. I introduce a new derived type (opbundles) which collects the parameters for the Hamiltonian, and rewrote the application routines to use it. It has been validated and verified. I made progress towards revised MPI parallelization. Using the opbundles, I was able to compute a distribution of work over compute nodes, which should be very efficient. This new distribution is easier to derive and more efficient, in principle, than the old distribution. Furthermore, it should make applications with fragmented Lanczos vectors easier. Implementation is still in progress.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Study of up–down poloidal density asymmetry of high- $\mathrm{Z}$ impurities with the new impurity version of XGCa

Addition of multispecies impurity ions to the total-f gyrokinetic particle-in-cell code XGCa is reported, including a cross-verification of neoclassical physics against the NEO code. This new version of the neoclassical gyrokinetic code XGCa is used to benchmark and confirm the previous reduced-equation-based prediction that high- $\mathrm{Z}$ impurity particles in the Pfirsch–Schlüter regime can exhibit a significant level of up–down poloidal asymmetry, through the large parallel friction force, and thus influence the radial plasma transport significantly. The study is performed in a plasma with weak toroidal rotation. In comparison, when the impurity particles are in the plateau regime, the up–down poloidal asymmetry becomes weak, with the parallel friction force becoming weaker than the parallel viscous force. It is also found that the linearization of the perturbed distribution function, based on the small poloidal asymmetry assumption, can become invalid. Finally, using the numerical data from XGCa, each term in the parallel fluid force-balance equation have been analysed to find that both the main ions and the electrons respond to the poloidal potential variation adiabatically when the high- $\mathrm{Z}$ tungsten possesses large poloidal variation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Quantitative Fluorescence Microscopy To Determine Molecular Occupancy of Phospholipid Vesicles

Encapsulation of molecules in phospholipid vesicles provides unique opportunities to study chemical reactions in small volumes as well as the behavior of individual proteins, enzymes, and ribozymes in a confined region without requiring a tether to immobilize the molecule to a surface. These experiments generally depend on generating a predictable loading of vesicles with small numbers of target molecules and thus raise a significant measurement challenge, namely, to quantify molecular occupancy of vesicles at the single-molecule level. In this work, we describe an imaging experiment to measure the time-dependent fluorescence from individual dye molecules encapsulated in ~130 nm vesicles that are adhered to a glass surface. For determining a fit of the molecular occupancy data to a Poisson model, it is critical to count empty vesicles in the population since these dominate the sample when the mean occupancy is small, λ ≤ ~1. Counting empty vesicles was accomplished by subsequently labeling all the vesicles with a lipophilic dye and reimaging the sample. By counting both the empty vesicles and those containing fluors, and quantifying the number of fluors present, we demonstrate a self-consistent Poisson distribution of molecular occupancy for well-solvated molecules, as well as anomalies due to aggregation of dye, which can arise even at very low solution concentrations. Here, by observation of many vesicles in parallel in an image, this approach provides quantitative information about the distribution of molecular occupancy in a population of vesicles.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

X-composer: enabling cross-environments in-situ workflows between HPC and cloud

As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.

Wang, Dali↗

Accelerating the Lagrangian Particle Tracking in Hydrologic Modeling to Continental-Scale

Unprecedented climate change and anthropogenic activities have induced increasing ecohydrological problems, which have motivated the development of large-scale modeling for solutions. Water age/quality is as important as water quantity for understanding the water cycle. However, current scientific progress in tracking water parcels at large-scale with high spatiotemporal resolutions is far behind that in water balance/quantity owing to the lack of powerful tools. EcoSLIM is a particle tracking model that works with the hydrologic model ParFlow-CLM, which couples surface-subsurface hydrology with land surface processes. Here, we demonstrate a parallel framework to accelerate EcoSLIM to continental-scale on a distributed, multi-GPU platform with CUDA-Aware MPI. In tests from catchment-, to regional-, and then to continental-scale using 25-million to 1.6-billion particles, EcoSLIM shows significant speedup and excellent parallel performance. The parallel framework is portable to atmospheric and oceanic particle tracking models, where parallelization is inadequate and a standard parallel framework is absent. Parallelized EcoSLIM is a promising tool to accelerate our understanding of the terrestrial water cycle and the upscaling of subsurface hydrology to Earth system models.

54 ENVIRONMENTAL SCIENCES↗

DIMPLES: Distributed Influence Maximization for Pandemic pLanning on Exascale Systems

We study exascale parallel algorithms for the selection of intervention or monitoring strategies in massive realistic socio-technical networks through scalable Influence Maximization (InfMax) algorithms. We employ novel techniques to enable efficient scaling on up to 8k nodes of OLCF Frontier, with 65k AMD GPUs and 458k AMD CPU cores. Current state-of-the-art InfMax tools are limited to networks with only a few million actors (vertices) and a few hundred million interactions (edges). By overcoming these limitations, we show that our approach is capable of processing a realistic social contact network of the United States with 285 million nodes and about 8 billion edges. This two orders-of-magnitude improvement over the previous state-of-the-art is obtained by leveraging algorithmic advancements for the InfMax problem and designing several problem-specific approaches to overlap communication with computation, improve GPU efficiency, and lower the application’s memory requirements. We evaluate strong scaling for computing 10k most influential seeds using up to 8k nodes of an exascale system, and weak scaling from 128 to 8k system nodes for seed sets ranging from 625 to 40k seeds. We achieve the fastest-known runtime of 25 minutes while performing 48 million diffusion simulations totaling 2.31 petabytes to identify 40k influential seeds using 8k nodes, and take 5.75 minutes to identify 10k seeds while using 4k nodes.

Minutoli, Marco [Pacific Northwest National Labora↗

New Results on Communication- and Memory-Aware Load Balancing Model and Algorithms

While load balancing in distributed-memory computing has been well-studied, we present an innovative approach to this problem: a unified, reduced-order model that combines three key components to describe “work” in a distributed system: computation, communication, and memory. Our model enables an optimizer to explore complex tradeoffs in task placement, such as augmented parallelism, at the expense of data replication increasing memory usage. We propose a fully distributed, heuristic-based load balancing optimization algorithm, and demonstrate that it quickly finds close-to-optimal solutions. We formalize the complex optimization problem as a mixed-integer linear program, and compare it to our strategy. Finally, we show that when applied to an electromagnetics code, our approach obtains up to 2.3x speedups for the imbalanced execution.

97 MATHEMATICS AND COMPUTING↗

A rigorous sequential update strategy for parallel kinetic Monte Carlo simulation

The kinetic Monte Carlo (kMC) method is used in many scientific fields in applications involving rare-event transitions. Due to its discrete stochastic nature, efforts to parallelize kMC approaches often produce unbalanced time evolutions requiring complex implementations to ensure correct statistics. In the context of parallel kMC, the sequential update technique has shown promise by generating high quality distributions with high relative efficiencies for short-range systems. In this work, we provide an extension of the sequential update method in a parallel context that rigorously obeys detailed balance, which guarantees exact equilibrium statistics for all parallelization settings. Furthermore, our approach also preserves nonequilibrium dynamics with minimal error for many parallelization settings, and can be used to achieve highly precise sampling.

36 MATERIALS SCIENCE↗

SOLAR WIND PROTONS AT 1 AU: TRENDS AND BOUNDS, CONSTRAINTS AND CORRELATIONS

The proton temperature anisotropy in the solar wind exhibits apparent bounds which are compatible with the theoretical constraints imposed by temperature-anisotropy-driven kinetic instabilities. Recent statistical analyses based on conditional averaging indicate that near these theoretical constraints the solar wind protons typically have enhanced temperatures and a weaker collisionality. Here we carefully analyze the solar wind data and show that these results are a consequence of superposition of multiple correlations in the solar wind, namely, they mostly result from the correlation between the proton temperature and the solar wind velocity and from the superimposed anti-correlation between the proton temperature anisotropy and the proton parallel beta in the fast solar wind. Colder and more collisional data are distributed around temperature isotropy whereas hotter and less collisional data have a wider range of the temperature anisotropy anti-correlated with the proton parallel beta with signatures of constraints owing to the temperature-anisotropy-driven instabilities. However, most of the hot and weakly collisional data, including the hottest and least collisional ones, lies far from the marginal stability regions. Consequently, we conclude that there is no clear relation between the enhanced temperatures and instability constraints and that the conditional averaging used for these analyses must be used carefully and need to be well tested.

79 ASTRONOMY AND ASTROPHYSICS↗

Method of making a distributed optical fiber sensor having enhanced Rayleigh scattering and enhanced temperature stability, and monitoring systems employing same

A method of making an optical fiber sensor device for distributed sensing includes generating a laser beam comprising a plurality of ultrafast pulses, and focusing the laser beam into a core of an optical fiber to form a nanograting structure within the core, wherein the nanograting structure includes a plurality of spaced nanograting elements each extending substantially parallel to a longitudinal axis of optical fiber. Also, an optical fiber sensor device for distributed sensing includes an optical fiber having a longitudinal axis, a core, and a nanograting structure within the core, wherein the nanograting structure includes a plurality of spaced nanograting elements each extending substantially parallel to the longitudinal axis of the optical fiber. Also, a distributed sensing method and system and an energy production system that employs such an optical fiber sensor device.

42 ENGINEERING↗

Method of making a distributed optical fiber sensor having enhanced Rayleigh scattering and enhanced temperature stability, and monitoring systems employing same

A method of making an optical fiber sensor device for distributed sensing includes generating a laser beam comprising a plurality of ultrafast pulses, and focusing the laser beam into a core of an optical fiber to form a nanograting structure within the core, wherein the nanograting structure includes a plurality of spaced nanograting elements each extending substantially parallel to a longitudinal axis of optical fiber. Also, an optical fiber sensor device for distributed sensing includes an optical fiber having a longitudinal axis, a core, and a nanograting structure within the core, wherein the nanograting structure includes a plurality of spaced nanograting elements each extending substantially parallel to the longitudinal axis of the optical fiber. Also, a distributed sensing method and system and an energy production system that employs such an optical fiber sensor device.

Chen, Peng Kevin↗

Implementing High Performance Remote Method Invocation in CCA

We report our effort in engineering a high performance remote method invocation (RMI) mechanism for the Common Component Architecture (CCA). This mechanism provides a highly efficient and easy-to-use mechanism for distributed computing in CCA, enabling CCA applications to effectively leverage parallel systems to accelerate computations. This work is built on the previous work of Babel RMI. Babel is a high performance language interoperability tool that is used in CCA for scientific application writers to share, reuse, and compose applications from software components written in different programming languages. Babel provides a transparent and flexible RMI framework for distributed computing. However, the existing Babel RMI implementation is built on top of TCP and does not provide the level of performance required to distribute fine-grained tasks. We observed that the main reason the TCP based RMI does not perform well is because it does not utilize the high performance interconnect hardware on a cluster efficiently. We have implemented a high performance RMI protocol, HPCRMI. HPCRMI achieves low latency by building on top of a low-level portable communication library, Aggregated Remote Message Copy Interface (ARMCI), and minimizing communication for each RMI call. Our design allows a RMI operation to be completed by only two RDMA operations. We also aggressively optimize our system to reduce copying. In this paper, we discuss the design and our experimental evaluation of this protocol. Our experimental results show that our protocol can improve RMI performance by an order of magnitude.

99 GENERAL AND MISCELLANEOUS↗

Parallel ICA of FDG-PET and PiB-PET in three conditions with underlying Alzheimer's pathology

The relationships between clinical phenotype, β-amyloid (Aβ) deposition and neurodegeneration in Alzheimer's disease (AD) are incompletely understood yet have important ramifications for future therapy. The goal of this study was to utilize multimodality positron emission tomography (PET) data from a clinically heterogeneous population of patients with probable AD in order to: (1) identify spatial patterns of Aβ deposition measured by ( 11 C)-labeled Pittsburgh Compound B (PiB-PET) and glucose metabolism measured by FDG-PET that correlate with specific clinical presentation and (2) explore associations between spatial patterns of Aβ deposition and glucose metabolism across the AD population. We included all patients meeting the criteria for probable AD (NIA–AA) who had undergone MRI, PiB and FDG-PET at our center (N = 46, mean age 63.0 ± 7.7, Mini-Mental State Examination 22.0 ± 4.8). Patients were subclassified based on their cognitive profiles into an amnestic/dysexecutive group (AD-memory; n = 27), a language-predominant group (AD-language; n = 10) and a visuospatial-predominant group (AD-visuospatial; n = 9). All patients were required to have evidence of amyloid deposition on PiB-PET. To capture the spatial distribution of Aβ deposition and glucose metabolism, we employed parallel independent component analysis (pICA), a method that enables joint analyses of multimodal imaging data.

60 APPLIED LIFE SCIENCES↗

Short-Term Load Forecasting Based Automatic Distribution Network Reconfiguration

In a traditional dynamic network reconfiguration study, the optimal topology is determined at every scheduled time point by using the real load data measured at that time. The development of the load forecasting technique can provide an accurate prediction of the load power that will happen in a future time and provide more information about load changes. With the inclusion of load forecasting, the optimal topology can be determined based on the predicted load conditions during a longer time period instead of using a snapshot of the load at the time when the reconfiguration happens; thus, the distribution system operator can use this information to better operate the system reconfiguration and achieve optimal solutions. This paper proposes a short-term load forecasting approach to automatically reconfigure distribution systems in a dynamic and pre-event manner. Specifically, a short-term and high-resolution distribution system load forecasting approach is proposed with a forecaster based on support vector regression and parallel parameters optimization. The network reconfiguration problem is solved by using the forecasted load continuously to determine the optimal network topology with the minimum amount of loss at the future time. The simulation results validate and evaluate the proposed approach.

24 POWER TRANSMISSION AND DISTRIBUTION↗