Search NASA⌕ Search

SEARCH · Search NASA

Results for “data parallel”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Experimental Analysis of File Transfer Rates over Wide-Area Dedicated Connections

File transfers over dedicated connections, supported by large parallel file systems, have become increasingly important in high-performance computing and big data workflows. It remains a challenge to achieve peak rates for such transfers due to the complexities of file I/O, host, and network transport subsystems, and equally importantly, their interactions. We present extensive measurements of disk-to-disk file transfers using Lustre and XFS file systems mounted on multi-core servers over a suite of 10 Gbps emulated connections with 0-366 ms round trip times. Our results indicate that large buffer sizes and many parallel flows do not always guarantee high transfer rates. Furthermore, large variations in the measured rates necessitate repeated measurements to ensure confidence in inferences based on them. We propose a new method to efficiently identify the optimal joint file I/O and network transport parameters using a small number of measurements. We show that for XFS and Lustre with direct I/O, this method identifies configurations achieving 97% of the peak transfer rate while probing only 12% of the parameter space.

Rao, Nageswara S.↗
DOE OSTI2015

MapReduceXMT v. Beta 0.1

The MapReduceXMT library ports the MapReduce framework onto the Cray XMT. MapReduce is a programming paradigm and an approach to data management for unstructured problems. It has gained relevance due to its ability to map serial operations onto parallel distributed architectures, significantly improving developer/analyst productivity. The MapReduceXMT implements the key aspects of MapReduce for the Cray XMT, a massively threaded system that is inherently difficult to program. MapReduceXMT allows users to utilize the machine effectively and efficiently without extensive training in multi-threaded programming. The MapReduceXMT library ports the MapReduce framework onto the Cray XMT. MapReduce is a programming paradigm and an approach to data management for unstructured problems. It has gained relevance due to its ability to map serial operations onto parallel distributed architectures, significantly improving developer/analyst productivity. The MapReduceXMT implements the key aspects of MapReduce for the Cray XMT, a massively threaded system that is inherently difficult to program. MapReduceXMT allows users to utilize the machine effectively and efficiently without extensive training in multi-threaded programming.

Goodman, Eric↗
DOE OSTI2022

Evaluation of intracoronary hemodynamics identifies perturbations in vorticity

Coronary artery disease (CAD) is highly prevalent and associated with adverse events. Challenges have emerged in the treatment of intermediate coronary artery stenoses. These lesions are often interrogated with fractional flow reserve (FFR) testing to determine if a stenosis is likely to be causative for ischemia in a cardiac territory. This invasive test requires insertion of a pressure wire into a coronary vessel. Recently computational fluid dynamics (CFD) has been used to noninvasively assess fractional flow reserve in vessels reconstructed from medical imaging data. However, many of these simulations are unable to provide additional information about intravascular hemodynamics, including velocity, endothelial shear stress (ESS), and vorticity. We hypothesized that vorticity, which has demonstrated utility in the assessment of ventricular and aortic diseases, would also be an important hemodynamic factor in CAD. Three-dimensional (3D), patient-specific coronary artery geometries that included all vessels >1 mm in diameter were created from angiography data obtained from 10 patients who underwent diagnostic angiography and FFR testing (n = 9). A massively parallel CFD solver (HARVEY) was used to calculate coronary hemodynamic parameters including pressure, velocity, ESS, and vorticity. These simulations were validated by comparing velocity flow fields from simulation to both velocities derived from in vitro particle image velocimetry and to invasively acquired pressure wire-based data from clinical testing. There was strong agreement between findings from CFD simulations and particle image velocimetry experimental testing (p < 0.01). CFD-FFR was also highly correlated with invasively measured FFR (ρ = 0.77, p = 0.01) with an average error of 5.9 ± 0.1%. CFD-FFR also had a strong inverse correlation with the vorticity (ρ = -0.86, p = 0.001). Simulations to determine the effect of the coronary stenosis on intravascular hemodynamics demonstrated significant differences in velocity and vorticity (both p < 0.05). Further evaluation of an angiographically normal appearing non-FFR coronary vessel in patients with CAD also demonstrated differences in vorticity when compared with FFR vessels (p < 0.05). The use of highly accurate 3D CFD-derived intravascular hemodynamics provides additional information beyond pressure measurements that can be used to calculate FFR. Vorticity is one parameter that is modified by a coronary stenosis and appears to be abnormal in angiographically normal vessels in patients with CAD, highlighting a possible use-case in preventative screening for early coronary disease.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Parallel I/O Evaluation Techniques and Emerging HPC Workloads: A Perspective

Emerging workloads such as artificial intelligence, big data analytics and complex multi-step workflows alongside future exascale applications are anticipated future HPC workloads, which will result in a more diverse I/O system workload and even less predictable I/O behavior and access patterns. Along with the ever increasing gap between the compute and storage performance capabilities, the in-depth understanding of extreme-scale I/O behavior and the I/O performance modeling and prediction are essential tools of the large-scale I/O evaluation process for addressing the needs of extreme-scale hybrid workloads. In this survey article, we focus on the state-of-the-art of the I/O behavior and performance analysis process for HPC systems in a 5-year time window and identify future research challenges.

Neuwirth, Sarah↗
DOE OSTI2016

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The.LHC, operating at CERN, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than grid can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility. Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full pro duction for the ATLAS since September 2015. We will present our current accomplishments with running PanDA at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility (OLCF), MIRA supercomputer at Argonne Leadership Computing Facilities (ALCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava and others). Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full production for the ATLAS experiment since September 2015. We will present our current accomplishments with running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

Data-intensive science↗
DOE OSTI2016

Application of morphological synthesis for understanding electrode microstructure evolution as a function of applied charge/discharge cycles

Morphological analysis and synthesis operations were employed for analysis of electrode microstructure transformations and evolution accompanying the application of charge/discharge cycles to electrochemical storage systems (batteries). Using state-of-the-art morphological algorithms, it was possible to predict microstructure evolution in porous Si electrodes for Li-ion batteries with sufficient accuracy. Algorithms for image analyses (segmentation, feature extraction, and 3D-reconstructions using 2D-images) were also developed. Altogether, these techniques could be considered supplementary to phase-field mesoscopic approach to microstructure evolution that is based upon clear and definitive changes in the appearance of microstructure. However, unlike in phase-field, the governing equations for morphological approach are geometry-, not physics-based. Similar non-physics based approach to understanding different phenomena was attempted with the introduction of cellular automata. It is anticipated that morphological synthesis and analysis will represent a useful supplementary tool to phase-field and will render assistance to unraveling the underlying microstructure-property relationships. The paper contains data on electrochemical characterization of different electrode materials that was conducted in parallel to morphological study.

25 ENERGY STORAGE↗
DOE OSTI2020

PPINN: Parareal physics-informed neural network for time-dependent PDEs

Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree. While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time–space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposing a long-time problem into many independent short-time problems supervised by an inexpensive/fast coarse-grained (CG) solver. In particular, the serial CG solver is designed to provide approximate predictions of the solution at discrete times, while initiate many fine PINNs simultaneously to correct the solution iteratively. There is a two-fold benefit from training PINNs with small-data sets rather than working on a large-data set directly, i.e., training of individual PINNs with small-data is much faster, while training the fine PINNs can be readily parallelized. Consequently, compared to the original PINN approach, the proposed PPINN approach may achieve a significant speed-up for long-time integration of PDEs, assuming that the CG solver is fast and can provide reasonable predictions of the solution, hence aiding the PPINN solution to converge in just a few iterations. To investigate the PPINN performance on solving time-dependent PDEs, we first apply the PPINN to solve the Burgers equation, and subsequently we apply the PPINN to solve a two-dimensional nonlinear diffusion–reaction equation. Furthermore, our results demonstrate that PPINNs converge in a few iterations with significant speed-ups proportional to the number of time-subdomains employed.

42 ENGINEERING↗
DOE OSTI2018

Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley

Here, burst buffer is an effective solution for reducing the data transfer time and the I/O interference in HPC systems. Extending Burst Buffers (BBs) to handle Big Data applications is challenging because BBs must account for the large data inputs of Big Data applications and the Quality-of-Service (QoS) of HPC applications which are considered as first-class citizens in HPC systems. Existing BBs focus on only intermediate data of Big Data applications and incur a high performance degradation of both Big Data and HPC applications. We present Eley , a burst buffer solution that helps to accelerate the performance of Big Data applications while guaranteeing the QoS of HPC applications. To achieve this goal, Eley embraces interference-aware prefetching technique that makes reading data input faster while introducing low interference for HPC applications. Evaluations using a wide range of Big Data and HPC applications demonstrate that Eley improves the performance of Big Data applications by up to 30% compared to existing BBs while maintaining the QoS of HPC applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. Here, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for $\mathscr{O}$(N) task execution and $\mathscr{O}$(N ln N) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Interface capturing simulations of bubble population effects in PWR subchannels

As the computational power of high-performance computing (HPC) facilities grows, so too does the feasibility of using first principle based simulation to study turbulent two-phase flows within complex pressurized water reactor (PWR) geometries. Direct numerical simulation (DNS), integrated with an interface capturing method, allows for the collection of high-fidelity numerical data using advanced analysis techniques. The research presented here employs the massively parallel, finite-element based, unstructured mesh code, PHASTA, to simulate a set of two-phase bubbly flows through PWR subchannel geometries including auxiliary structures (spacer grids and mixing vanes). The main objective of the presented work is to analyze bubble dynamics and turbulence interactions at varying bubble concentrations to support the development of advanced two-phase flow closure models. Turbulent two-phase flows in PWR subchannels were simulated at hydraulic Reynolds numbers of 81,000 with bubble concentrations of 3%–15% by gas volume fraction (768–3928 resolved bubbles, respectively) and compared against a 1% void fraction case (262 bubbles) that had been previously simulated. The finite element mesh utilized for the study at higher bubble concentrations was composed of 1.55 billion elements, compared to the previous study which employed 1.11 billion elements, ensuring all turbulence scales and individual bubbles within the flow are fully resolved. For each case, the resolved initial bubble size was 0.65 mm in diameter (resolved with 25 grid points across the diameter). The simulations were analyzed to find flow features such as the mean velocity profile, bubble relative velocity and the effect of the bubbles on the turbulent conditions.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Porting ATLAS Fast Calorimeter Simulation to GPUs with Performance Portable Programming Models

FastCaloSim is a parameterized simulation of the particle energy response and of the energy distribution in the ATLAS calorimeter. It is a relatively small and self-contained package with massive inherent parallelism and captures the essence of GPU offloading via important operations like data transfer, memory initialization, floating point operations, and reduction. It was identified by the High Energy Physics Center for Computational Excellence project as a good testbed for evaluating the performance and ease of portability of programming models. In this paper, we will discuss the results of our evaluation of the porting process to Kokkos, SYCL, Alpaka, OpenMP and std::par (nvc++), and compare performance on NVIDIA, AMD and Intel GPUs, as well as multicore CPUs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Protein folding from heterogeneous unfolded state revealed by time-resolved X-ray solution scattering

One of the most challenging tasks in biological science is to understand how a protein folds. In theoretical studies, the hypothesis adopting a funnel-like free-energy landscape has been recognized as a prominent scheme for explaining protein folding in views of both internal energy and conformational heterogeneity of a protein. Despite numerous experimental efforts, however, comprehensively studying protein folding with respect to its global conformational changes in conjunction with the heterogeneity has been elusive. Here we investigate the redox-coupled folding dynamics of equine heart cytochrome c (cyt-c) induced by external electron injection by using time-resolved X-ray solution scattering. A systematic kinetic analysis unveils a kinetic model for its folding with a stretched exponential behavior during the transition toward the folded state. With the aid of the ensemble optimization method combined with molecular dynamics simulations, we found that during the folding the heterogeneously populated ensemble of the unfolded state is converted to a narrowly populated ensemble of folded conformations. These observations obtained from the kinetic and the structural analyses of X-ray scattering data reveal that the folding dynamics of cyt-c accompanies many parallel pathways associated with the heterogeneously populated ensemble of unfolded conformations, resulting in the stretched exponential kinetics at room temperature. This finding provides direct evidence with a view to microscopic protein conformations that the cyt-c folding initiates from a highly heterogeneous unfolded state, passes through still diverse intermediate structures, and reaches structural homogeneity by arriving at the folded state.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2017

Integration of Titan supercomputer at OLCF with ATLAS Production System

The PanDA (Production and Distributed Analysis) workload management system was developed to meet the scale and complexity of distributed computing for the ATLAS experiment. PanDA managed resources are distributed worldwide, on hundreds of computing sites, with thousands of physicists accessing hundreds of Petabytes of data and the rate of data processing already exceeds Exabyte per year. While PanDA currently uses more than 200,000 cores at well over 100 Grid sites, future LHC data taking runs will require more resources than Grid computing can possibly provide. Additional computing and storage resources are required. Therefore ATLAS is engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. In this paper we will describe a project aimed at integration of ATLAS Production System with Titan supercomputer at Oak Ridge Leadership Computing Facility (OLCF). Current approach utilizes modified PanDA Pilot framework for job submission to Titan's batch queues and local data management, with lightweight MPI wrappers to run single node workloads in parallel on Titan's multi-core worker nodes. It provides for running of standard ATLAS production jobs on unused resources (backfill) on Titan. The system already allowed ATLAS to collect on Titan millions of core-hours per month, execute hundreds of thousands jobs, while simultaneously improving Titans utilization efficiency. We will discuss the details of the implementation, current experience with running the system, as well as future plans aimed at improvements in scalability and efficiency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

A Unit Testing Framework for Scientific Legacy Code

Large-scale scientific applications play important roles in supporting research. However, it is often very expensive and time-consuming to make changes to, maintain and evolve the scientific code due to its complexity and poor programming skills of researchers. Therefore, in order to visualize scientific code architecture to optimize software design, understand undocumented source code, and analyze software flow and functionality, we first introduce a unit testing framework (UTF). Then, because such infrastructure's performance is very crucial in practical use since the scientific legacy applications simulate instances in a long period of time, we improve the UTF by applying Message Passing based Parallelization and parallel I/O operations. Furthermore, due to the scientific code has enormous state data and the I/O capacity on the server is limited, we apply in situ data analysis method to encounter fewer resource limitations, and adopt signal processing to greatly reduce data transfer. Last, we demonstrated the correctness and high-efficiency of our framework for legacy Earth model on Titan supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Approximate Computing Techniques for Iterative Graph Algorithms

Approximate computing enables processing of large-scale graphs by trading off quality for performance. Approximate computing techniques have become critical not only due to the emergence of parallel architectures but also the availability of large scale datasets enabling data-driven discovery. Using two prototypical graph algorithms, PageRank and community detection, we present several approximate computing heuristics to scale the performance with minimal loss of accuracy. We present several heuristics including loop perforation, data caching, incomplete graph coloring and synchronization, and evaluate their efficiency. We demonstrate performance improvements of up to 83% for PageRank and up to 450x for community detection, with low impact of accuracy for both the algorithms. We expect the proposed approximate techniques will enable scalable graph analytics on data of importance to several applications in science and their subsequent adoption to scale similar graph algorithms.

Panyala, Ajay R.↗
DOE OSTI2015

High Level Synthesis of RDF Queries for Graph Analytics

In this paper we present a set of techniques that enable the synthesis of efficient custom accelerators for memory intensive, irregular applications. To address irregular applications challenges (large memory footprints, unpredictable fine- grained data accesses, and high synchronization intensity), and exploit their opportunities (thread level parallelism, memory level parallelism), we propose a novel accelerator design which take advantage of an adaptive and Distributed Controller (DC) architecture, and a Memory Interface (MI) that supports parallel memory subsystems. Among the multitude of algorithms that may benefit from our solution, we focus on the acceleration of graph analytics applications, and in particular, on the syn- thesis of SPARQL queries on Resource Description Framework (RDF) databases. We achieve this objective by incorporating the synthesis techniques into Bambu, an Open Source high- level synthesis tools, and interfacing the system with GEMS, the Graph database Engine for Multithreaded Systems. The front- end of GEMS generates optimized C implementations of the input queries, modeled as pattern matching routines, which are then automatically synthesized by Bambu. We validate our approach synthesizing several SPARQL queries from the Lehigh University Benchmark (LUBM).

HLS, FPGA, Convey, RDF, GEMS, Memory-intensive,↗
DOE OSTI2015

Evaluating the Efficacy of Wavelet Configurations on Turbulent-Flow Data

I/O is increasingly becoming a significant constraint for simulation codes and visualization tools on modern supercomputers. Data compression is an attractive workaround, and, in particular, wavelets provide a promising solution. However, wavelets can be applied in multiple configurations, and the variations in configuration impact accuracy, storage cost, and execution time. While the variation in these factors over wavelet configurations have been explored in image processing, they are not well understood for visualization and analysis of scientific data. To illuminate this issue, we evaluate multiple wavelet configurations on turbulent-flow data. Our approach is to repeat established analysis routines on uncompressed and lossy-compressed versions of a data set, and then quantitatively compare their outcomes. Our findings show that accuracy varies greatly based on wavelet configuration, while storage cost and execution time vary less. Overall, our study provides new insights for simulation analysts and visualization experts, who need to make tradeoffs between accuracy, storage cost, and execution time.

97 MATHEMATICS AND COMPUTING↗