Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel, scaling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

PUMIPic: A mesh-based approach to unstructured mesh Particle-In-Cell on GPUs

Unstructured mesh particle-in-cell, PIC, simulations executing on the current and next generation of massively parallel systems require new methods for both the mesh and particles to achieve performance and scalability on GPUs. The traditional approach to implementing PIC simulations defines data structures and algorithms in terms of particles with a full copy of the unstructured mesh on every process. To effectively scale the unstructured mesh and particles, mesh-based PIC uses the unstructured mesh as the predominant data structure with the particles stored in terms of the mesh entities. Here, this paper details the PUMIPic library, a framework for developing efficient and performance-portable mesh-based PIC simulations on GPU systems. A pseudo physics simulation based on a five-dimensional gyro-kinetic code for modeling plasma physics is used to examine the performance of PUMIPic. Scaling studies of the unstructured mesh partition and number of particles are performed up to 4096 nodes of the Summit system at Oak Ridge National Laboratory. The studies show that mesh-based PIC can utilize a partitioned mesh and maintain scaling up to system limitations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Electric field directed formation of aligned conjugated polymer fibers

Alternating current (AC) electric fields effectively align poly (3-alkylthiophene) (P3AT) fibers during a one-dimensional crystallization process. Structural, mechanical and electrical properties have been probed using microscopy, small angle neutron scattering (SANS), atomic force microscopy (AFM), x-ray diffraction (XRD), rheology, UV-vis spectroscopy, and dielectric spectroscopy. Optimum frequency and amplitudes were identified for specific P3AT concentrations and for variable solvent quality. Optical microscopy along with SANS and XRD demonstrate alignment persists over both micrometer and nanometer length scales. Small amplitude oscillatory shear rheology also showed that the shear modulus increased for aligned fibers. The structural changes were correlated to improvements in electric conductivity as probed by dielectric spectroscopy. XRD experiments performed with variable sample orientations also showed that the P3HT stacking direction was aligned parallel to the direction of the fiber axis. Electric field alignment was also possible for other alkyl-thiophene polymers and for polymers with simple backbone repeat units.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2024

Pretraining Billion-Scale Geospatial Foundational Models on Frontier

As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.

Tsaris, Aristeidis (aris)↗
DOE OSTI2022

Image Gradient Decomposition for Parallel and Memory-Efficient Ptychographic Reconstruction

Ptychography is a popular microscopic imaging modality for many scientific discoveries and sets the record for highest image resolution. Unfortunately, the high image resolution for ptychographic reconstruction requires significant amount of memory and computations, forcing many applications to compromise their image resolution in exchange for a smaller memory footprint and a shorter reconstruction time. In this paper, we propose a novel image gradient decomposition method that significantly reduces the memory footprint for ptychographic reconstruction by tessellating image gradients and diffraction measurements into tiles. In addition, we propose a parallel image gradient decomposition method that enables asynchronous point-to-point communications and parallel pipelining with minimal overhead on a large number of GPUs. Our experiments on a Titanate material dataset (PbTiO3) with 16632 probe locations show that our Gradient Decomposition algorithm reduces memory footprint by 51 times. In addition, it achieves time-to-solution within 2.2 minutes by scaling to 4158 GPUs with a super-linear strong scaling efficiency at 364% compared to runtimes at 6 GPUs. This performance is 2.7 times more memory efficient, 9 times more scalable and 86 times faster than the state-of-the-art algorithm.

Wang, Xiao↗
DOE OSTI2017

Multistep inversion workflow for 3D long-offset damped elastic waves in the Fourier domain

We present a new workflow for imaging damped three-dimensional elastic wavefields in the Fourier domain. The workflow employs a multiscale imaging approach, in which offset lengths are laddered, where frequency content and damping of the data are changed cyclically. Thus, the inversion process is launched using short-offset and low-frequency data to recover the long spatial wavelength of the image at a shallow depth. Increasing frequency and offset length leads to the recovery of the fine-scale features of the model at greater depths. For the fixed offset, we employ (in the imaging process) a few discrete frequencies with a set of Laplace damping parameters. The forward problem is solved with a finite-difference frequency-domain method based on a massively parallel iterative solver. The inversion code is based upon the solution of a least squares optimisation problem and is solved using a nonlinear gradient method. It is fully parallelised for distributed memory computational platforms. Our full-waveform inversion workflow is applied to the 3D Marmousi-2 and SEG/EAGE Salt models with long-offset data. The maximum inverted frequencies are 6 Hz for the Marmousi model and 2 Hz for the SEG/EAGE Salt model. The detailed structures are imaged successfully up to the depth approximately equal to one-third of the maximum offset length at a resolution consistent with the inverted frequencies.

3D full waveform inversion↗
DOE OSTI2018

Scalable Preconditioners for Structure Preserving Discretizations of Maxwell Equations in First Order Form

Here, we report multiple physical time-scales can arise in electromagnetic simulations when dissipative effects are introduced through boundary conditions, when currents follow external time-scales, and when material parameters vary spatially. In such scenarios, the time-scales of interest may be much slower than the fastest time-scales supported by the Maxwell equations, therefore making implicit time integration an efficient approach. The use of implicit temporal discretizations results in linear systems in which fast time-scales, which severely constrain the stability of an explicit method, can manifest as so-called stiff modes. This study proposes a new block preconditioner for structure preserving (also termed physics compatible) discretizations of the Maxwell equations in first order form. The intent of the preconditioner is to enable the efficient solution of multiple-time-scale Maxwell type systems. An additional benefit of the developed preconditioner is that it requires only a traditional multigrid method for its subsolves and compares well against alternative approaches that rely on specialized edge-based multigrid routines that may not be readily available. Lastly, results demonstrate parallel scalability at large electromagnetic wave CFL numbers on a variety of test problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Performance Evaluation of Heterogeneous GPU Programming Frameworks for Hemodynamic Simulations

Preparing for the deployment of large scientific and engineering codes on upcoming exascale systems with GPU-dense nodes is made challenging by the unprecedented diversity of device architectures and heterogeneous programming models. In this work, we evaluate the process of porting a massively parallel, fluid dynamics code written in CUDA to SYCL, HIP, and Kokkos with a range of backends, using a combination of automated tools and manual tuning. We use a proxy application along with a custom performance model to inform the results and identify additional optimization strategies. At scale performance of the programming model implementations are evaluated on pre-production GPU node architectures for Frontier and Aurora, as well as on current NVIDIA device-based systems Summit and Polaris. Real-world workloads representing 3D blood flow calculations in complex vasculature are assessed. Our analysis highlights critical trade-offs between code performance, portability, and development time.

Martin, Aristotle↗
DOE OSTI2016

Next Generation Protein Interactomes for Plant Systems Biology and Biomass Feedstock Research

Biofuel crop cultivation is a necessary step in heading towards a sustainable future, making their genomic studies a priority. While technology platforms that currently exist for studying non-model crop species, like switch-grass or sorghum, have yielded large quantities of genomic and expression data, still a large gap exists between molecular mechanism and phenotype. The aspect of molecular activity at the level of protein-protein interactions has recently begun to bridge this gap, providing a more global perspective. Interactome analysis has defined more specific functional roles of proteins based on their interaction partners, neighborhoods, and other network features, making it possible to distinguish unique modules of immune response to different plant pathogens(Jiang, Dong, and Zhang 2016). As we work towards cultivating heartier biofuel crops, interactome data will lead to uncovering crop-specific defense and development networks. However, the collection of protein interaction data has been limited to expensive, time-consuming, hard-to-scale assays that mostly require cloned ORF collections. For these reasons, we have successfully developed a highly scalable, economical, and sensitive yeast two-hybrid assay, ProCREate, that can be universally applied to generate proteome-wide primary interactome data. ProCREate enables en masse pooling and massively paralleled sequencing for the identification of interacting proteins by exploiting Cre-lox recombination. ProCREate can be used to screen ORF/cDNA libraries from feedstock plant tissues. The interactome data generated will yield deeper insight into many molecular processes and pathways that can be used to guide improvement of feedstock productivity and sustainability.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Productive Programming of Distributed Systems with the SHAD C++ Library

High-performance computing (HPC) is often perceived as a matter of making large-scale systems (e.g., clusters) run as fast as possible, regardless the required programming effort. However, the idea of "bringing HPC to the masses" has recently emerged. Inspired by this vision, we have designed SHAD, the Scalable High-performance Algorithms and Data-structures library. SHAD is open source software, written in C++, for C++ developers. Unlike other HPC libraries for distributed systems, which rely on SPMD models, SHAD adopts a shared-memory programming abstraction, to make C++ programmers feel at home. Underneath, SHAD manages tasking and data-movements, moving the computation where data resides and taking advantage of asynchrony to tolerate network latency. At the bottom of his stack, SHAD can interface with multiple runtime systems: this not only improves developer’s productivity, by hiding the complexity of such software and of the underlying hardware, but also greatly enhance code portability. Thanks to its abstraction layers, SHAD can indeed target different systems, ranging from laptops to HPC clusters, without any need for modifying the user-level code. We have prototyped and open-sourced the implementation of (a subset of) the C++ standard library (STL) targeting multi-node HPC clusters. Our work allows plain STL-based C++ code to scale on HPC systems, with no need for rewriting the code to exploit the complex hardware. SHAD is available under Apache v2 License at https://github.com/pnnl/SHAD. In this paper we overview the design of the SHAD library, depicting its main components: runtime systems abstractions for tasking; parallel and distributed data-structures; STL-compliant interfaces and algorithms.

Castellana, Vito G.↗
DOE OSTI2019

A massively-parallel, unstructured overset method to simulate moving bodies in turbulent flows

An unstructured overset method capable of performing direct numerical simulation (DNS) and large eddy simulation (LES) of many (O(10 5 )) moving bodies, utilizing many computational cores (O(10 5 )), in turbulent, incompressible fluid flow is presented. Unstructured meshes are attached to bodies and placed within a fixed background domain. Body meshes are allowed to arbitrarily overlap and move throughout the domain. Within each mesh a high resolution, unstructured, non-dissipative finite volume method is used to solve for the flow field. Boundary conditions for each mesh are provided by interpolation from flow solutions on overlapping meshes. When many unstructured meshes of different resolution overlap, care is required in the connection between the different flow solutions. An interpolant is created which seeks to preserve volume conservation of flow quantities between meshes regardless of overlapping mesh differences. An implicit fractional step method is used for time advancement, requiring the calculation of a predicted fluid velocity and corrector pressure field. For the predictor step, the resulting interpolation is directly introduced into the implicit equations for the predicted flow field. For the corrected pressure field, the continuity between meshes is weakly enforced using a penalty formulation. The pressure formulation is symmetric, positive-definite and non-singular resulting in a formulation which is readily solvable using traditional iterative matrix inversion techniques. An Arbitrary Euler-Lagrangian (ALE) method coupled to a 6 degrees of freedom rigid body equation system (6-DOF) is used for body motion. For rotation, a quaternion representation is used to solve Euler's equations of rigid body motion. A linear spring damper model, which uses geometry information readily available from the overset assembly process, is used for collisions. Validation of the method for canonical flow fields is presented including assessment of order of accuracy and kinetic energy conservation properties. Particle-resolved direct numerical simulation (PR-DNS) of single particles in various flow fields are presented for validation. PR-DNS results of 50,000 spherical particles freely moving within turbulent channel flow are shown as a demonstration of the method at full scale. LES results of a marine propeller under crashback conditions are shown to demonstrate the ability to simulate highly unsteady turbulent flows over complex, moving geometries.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Status of Shift for NCS Applications in SCALE 6.3 [Slides]

Shift can run most KENO V.a and KENO-VI inputs, but beware of small geometry fudge factors. Shift is faster than KENO for fast spectrum systems, but has a higher uncertainty per particle. KENO FoM is higher for thermal systems. KENO V.a in MG is significantly faster than KENO V.a in CE, so this may exacerbate Shift difference. Shift generally has greater speedup for parallel calculations. Shift has only IFP available for TSUNAMI-3D, but parallel calculations are enabled.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Critical role of isopotential surfaces for magnetostatic ponderomotive forces

By producing localized wave regions at the ends of an open-field-line magnetic confinement system, ponderomotive walls can be used to differentially confine different species in the plasma. Furthermore, if the plasma is rotating, this wall can be magnetostatic in the laboratory frame, resulting in simpler engineering and better power flow. However, recent work on such magnetostatic walls has shown qualitatively different potentials than those found in the earlier, nonrotating theory. Here, using a simple slab model of a ponderomotive wall, we resolve this discrepancy. We show that the form of the ponderomotive potential in the comoving plasma frame depends on the assumption made about the electrostatic potential in the laboratory frame. If the laboratory-frame potential is unperturbed by the magnetic oscillation, one finds a parallel-polarized wave in the comoving frame, while if each field line remains equipotential throughout the perturbation region, one finds a perpendicularly polarized wave. As a result, this in turn dramatically changes the averaged ponderomotive force experienced by a charged particle along the field line, not only its scaling, but also its direction.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

The Persistent Challenge of Data Locality in the Post-Exascale Era

The era of exascale computing, exemplified by systems like Frontier achieving exaflop-level performance, marks a milestone. However, the quest for sheer compute power leads to strong imbalance in system design. Hence, scaling advancements in memory, network bandwidth, and storage are also necessary and pose challenges, with a crucial need to address data locality issues. This article underscores the fundamental importance of data locality as a key abstraction for optimizing application performance. Despite notable software solutions, the growing complexity of parallelism and memory hierarchy demands performance-portable data locality solutions across diverse computing platforms. Additionally, the article revisits data locality aspects, covering hardware considerations, application perspectives, software stack abstractions, and tool support. It concludes with insights into data locality challenges and opportunities, emphasizing the ongoing significance of collaborative research for progress in this critical issue.

Unat, Didem [Koc University, Istanbul (Turkey)] (O↗
DOE OSTI2017

On the Evolution From Micrometer-Scale Inhomogeneity to Global Overheated Structure During the Intense Joule Heating of a z-Pinch Rod

Ultrafast optical microscopy of metal z-pinch rods pulsed with megaampere current is contributing new data and critical insight into what provides the fundamental seed for the magneto-Rayleigh-Taylor (MRT) instability. A two-frame near infrared/visible intensified-charge-coupled device gated imager with 2-ns temporal resolution and 3-μm spatial resolution captured emissions from the nonuniformly Joule heated surfaces of ultrasmooth aluminum (Al) rods. Nonuniform surface emissions are consistently first observed from discrete, 10-μm scale, subelectronvolt spots. Aluminum 6061 alloy, with micrometer-scale nonmetallic resistive inclusions, forms several times more spots than 99.999% pure Al 5N; 5-10 ns later, azimuthally stretched elliptical spots and distinct strata (40-100 μm wide by 10 μm tall) are observed on Al 6061, but not on Al 5N. In such overheat strata, aligned parallel to the magnetic field, we find that they are highly effective seeds for MRT instability growth. Our data give credence to the hypothesis that early nonuniform Joule heating, such as the electrothermal instability, may provide the dominant seed for MRT.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

PoliMOR

PoliMOR is a scalable, automated, and customizable policy engine framework for multi-tiered parallel file systems. It is composed of single-purpose agents that handle tasks such as gathering file metadata, making policy decisions, and then executing actions based on those policies. These agents are designed to communicate using distributed message queues, allowing the number of individual agents to be scaled up as needed. PoliMOR automates the data management tasks by precluding the need for admin intervention. The agents in PoliMOR can be customized to integrate any utilities/tools that perform tasks like metadata scanning and data placement management.

Brumgard, Christopher↗
DOE OSTI2017

A Polarimetric Approach for Constraining the Dynamic Foreground Spectrum for Cosmological Global 21 cm Measurements

The cosmological global (sky-averaged) 21 cm signal is a powerful tool to probe the evolution of the intergalactic medium in high-redshift universe (z⩽6). One of the biggest observational challenges is to remove the foreground spectrum which is at least four orders of magnitude brighter than the cosmological 21 cm emission. Conventional global 21 cm experiments rely on the spectral smoothness of the foreground synchrotron emission to separate it from the unique 21 cm spectral structures in a single total-power spectrum. However, frequency-dependent instrumental and observational effects are known to corrupt such smoothness and complicate the foreground subtraction. We introduce a polarimetric approach to measure the projection-induced polarization of the anisotropic foreground onto a stationary dual-polarized antenna. Due to Earth rotation, when pointing the antenna at a celestial pole, the revolving foreground will modulate this polarization with a unique frequency-dependent sinusoidal signature as a function of time. In our simulations, by harmonic decomposing this dynamic polarization, our technique produces two separate spectra in parallel from the same observation: (i) a total sky power consisting both the foreground and the 21 cm background and (ii) a model-independent measurement of the foreground spectrum at a harmonic consistent to twice the sky rotation rate. In the absence of any instrumental effects, by scaling and subtracting the latter from the former, we recover the injected global 21 cm model within the assumed uncertainty. We further discuss several limiting factors and potential remedies for future implementation.

79 ASTRONOMY AND ASTROPHYSICS↗