Search NASA⌕ Search

SEARCH · Search NASA

Results for “data parallel”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Big Data Mining for Assessing Calibration of Building Energy Models

Residential and commercial buildings in China, India, the United States (US), United Kingdom (UK), and Italy consume 39-45% of each nation's primary energy. Building energy models can be used to automatically optimize the return-on-investment for retrofits to improve a building’s energy efficiency. However, with an average of 3,000 building descriptors necessary to accurately simulate a single building, there is a market need to reduce the transaction cost for creating a more robust model for simulating every building in a city and accurately estimate savings prior to capital expenditures.We used two of the world’s fastest supercomputers, assembled unique datasets, and developed innovative algorithms for big data mining to assess different methods to create an accurate building energy model. The team has leveraged a total of eight high performance computing resources and 218 servers to analyze the best methods, metrics, and algorithms for creating an accurate building energy model beyond current industry standards necessary for private-sector financing. The project developed the world’s fastest buildings simulator, has completed over 8 million simulations totaling over 300TB, and mined this data with over 130,000 parallel artificial intelligence algorithms. This was used to quantify accuracy of AI-developed calibration algorithms with results that surpass industry standard guidelines and can identify individual building parameters to between 15% and 32% of their actual value.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2017

Parallel Computation of the Jacobian Matrix for Nonlinear Equation Solvers Using MATLAB

Demonstrating speedup for parallel code on a multicore shared memory PC can be challenging in MATLAB due to underlying parallel operations that are often opaque to the user. This can limit potential for improvement of serial code even for the so-called embarrassingly parallel applications. One such application is the computation of the Jacobian matrix inherent to most nonlinear equation solvers. Computation of this matrix represents the primary bottleneck in nonlinear solver speed such that commercial finite element (FE) and multi-body-dynamic (MBD) codes attempt to minimize computations. A timing study using MATLAB's Parallel Computing Toolbox was performed for numerical computation of the Jacobian. Several approaches for implementing parallel code were investigated while only the single program multiple data (spmd) method using composite objects provided positive results. Parallel code speedup is demonstrated but the goal of linear speedup through the addition of processors was not achieved due to PC architecture.

Rose, Geoffrey K.↗
DOE OSTI2024

The PARADIGM Project: Case Study in Balancing Experiment Uncertainty with Design simplicity

Accurate nuclear data are required for simulations of many applications including nuclear criticality safety. Actinide nuclear data at intermediate energies (from 1 to 100s of keV) are imprecise and inaccurate, because of scarce differential data, and an insufficient theory approach to capture the structures expected in the data to yield evaluated nuclear data, and lack of integral data for proper validation. This is a known deficiency but has proved challenging to address. More specifically, only 5% of integral experiments in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) benchmark suite address intermediate energies (Fig. 1). Associated calculated effective multiplication factor, k eff , values for these experiments are far outside the experimental uncertainties and are 25× further from experiment than for fast energies. These differences could either stem from systematic biases in nuclear data, experiments or both. The goal of the PARADIGM (PARallel Approach of Differential and InteGral Measurements) project is to significantly reduce (by more than tens of percent) the uncertainties of intermediate energy actinide nuclear data. The PARADIGM project designed and intends to execute LANSCE (Los Alamos Neutron Science CEnter) and NCERC (National Criticality Experiments Research Center) intermediate experiments in parallel. They will specifically address a high priority nuclear data need—reducing bias and uncertainty in intermediate plutonium nuclear data. The two experiment will achieve that by informing each other and nuclear theory. By doing all these steps in parallel, the timeline to deliver improved nuclear data to users will significantly be reduced. This work will focus on the integral experiment final design and the balance of design and modeling simplicity while minimizing experiment uncertainty.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

Enabling Low-Overhead HT-HPC Workflows at Extreme Scale using GNU Parallel

GNU Parallel is a versatile and powerful tool for process parallelization widely used in scientific computing. This paper demonstrates its effective application in high-performance computing (HPC) environments, particularly focusing on its scalability and efficiency in executing large-scale high-throughput high-performance computing (HT-HPC) workflows. Through real-world examples, we highlight GNU Parallel’s performance across various HPC workloads, including GPU computing, container-based workloads, and node-local NVMe storage. Our results on two leading supercomputers, OLCF’s Frontier and NERSC’s Perlmutter, showcase GNU Parallel’s rapid process dispatching ability and its capacity to maintain low overhead even at extreme scales. We explore GNU Parallel’s application in massive parallel file transfers using a scheduled Data Transfer Node (DTN) cluster, emphasizing its broad utility in diverse scientific workflows. Beyond its direct application as a viable workflow manager, GNU Parallel can be employed in conjunction with other workflow systems as a "last-mile" parallelizing driver and as a quick prototyping tool to design and extract parallel profiles from application executions. We then argue that the potential for GNU Parallel to transform workflow management at extreme scales is substantial, paving the way for more efficient and effective scientific discoveries.

Maheshwari, Ketan↗
DOE OSTI2017

PSelInv—A Distributed Memory Parallel Algorithm for Selected Inversion: The Symmetric Case

We describe an efficient parallel implementation of the selected inversion algorithm for distributed memory computer systems, which we call PSelInv. The PSelInv method computes selected elements of a general sparse matrix Athat can be decomposed as A= LU, where L is lower triangular and U is upper triangular. The implementation described in this article focuses on the case of sparse symmetric matrices. It contains an interface that is compatible with the distributed memory parallel sparse direct factorization SuperLU-DIST. Yet, the underlying data structure and design of PSelInv allows it to be easily combined with other factorization routines, such as PARDISO. We discuss general parallelization strategies such as data and task distribution schemes. Specifically, we describe how to exploit the concurrency exposed by the elimination tree associated with the LU factorization of A. We demonstrate the efficiency and accuracy of PSelInv by presenting several numerical experiments. In particular, we show that PSelInv can run efficiently on more than 4,000 cores for a modestly sized matrix. We also show how PSelInv can be used to accelerate large-scale electronic structure calculations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1993

Efficiently modeling neural networks on massively parallel computers

Neural networks are a very useful tool for analyzing and modeling complex real world systems. Applying neural network simulations to real world problems generally involves large amounts of data and massive amounts of computation. To efficiently handle the computational requirements of large problems, we have implemented at Los Alamos a highly efficient neural network compiler for serial computers, vector computers, vector parallel computers, and fine grain SIMD computers such as the CM-2 connection machine. This paper describes the mapping used by the compiler to implement feed-forward backpropagation neural networks for a SIMD (Single Instruction Multiple Data) architecture parallel computer. Thinking Machines Corporation has benchmarked our code at 1.3 billion interconnects per second (approximately 3 gigaflops) on a 64,000 processor CM-2 connection machine (Singer 1990). This mapping is applicable to other SIMD computers and can be implemented on MIMD computers such as the CM-5 connection machine. Our mapping has virtually no communications overhead with the exception of the communications required for a global summation across the processors (which has a sub-linear runtime growth on the order of O(log(number of processors)). We can efficiently model very large neural networks which have many neurons and interconnects and our mapping can extend to arbitrarily large networks (within memory limitations) by merging the memory space of separate processors with fast adjacent processor interprocessor communications. This paper will consider the simulation of only feed forward neural network although this method is extendable to recurrent networks.

Farber, Robert M.↗
NASA NTRS1995

Spectral identification of minerals using imaging spectrometry data: Evaluating the effects of signal to noise and spectral resolution using the tricorder algorithm

The rapid development of sophisticated imaging spectrometers and resulting flood of imaging spectrometry data has prompted a rapid parallel development of spectral-information extraction technology. Even though these extraction techniques have evolved along different lines (band-shape fitting, endmember unmixing, near-infrared analysis, neural-network fitting, and expert systems to name a few), all are limited by the spectrometer's signal to noise (S/N) and spectral resolution in producing useful information. This study grew from a need to quantitatively determine what effects these parameters have on our ability to differentiate between mineral absorption features using a band-shape fitting algorithm. We chose to evaluate the AVIRIS, HYDICE, MIVIS, GERIS, VIMS, NIMS, and ASTER instruments because they collect data over wide S/N and spectral-resolution ranges. The study evaluates the performance of the Tricorder algorithm, in differentiating between mineral spectra in the 0.4-2.5 micrometer spectral region. The strength of the Tricorder algorithm is in its ability to produce an easily understood comparison of band shape that can concentrate on small relevant portions of the spectra, giving it an advantage over most unmixing schemes, and in that it need not spend large amounts of time reoptimizing each time a new mineral component is added to its reference library, as is the case with neural-network schemes. We believe the flexibility of the Tricorder algorithm is unparalleled among spectral-extraction techniques and that the results from this study, although dealing with minerals, will have direct applications to spectral identification in other disciplines.

Swayze, Gregg A.↗
DOE OSTI2018

Dynamic Load Balancing Based on Constrained K-D Tree Decomposition for Parallel Particle Tracing

Particle tracing is a fundamental technique in flow field data visualization. In this work, we present a novel dynamic load balancing method for parallel particle tracing. Specifically, we employ a constrained k-d tree decomposition approach to dynamically redistribute tasks among processes. Each process is initially assigned a regularly partitioned block along with duplicated ghost layer under the memory limit. During particle tracing, the k-d tree decomposition is dynamically performed by constraining the cutting planes in the overlap range of duplicated data. This ensures that each process is reassigned particles as even as possible, and on the other hand the new assigned particles for a process always locate in its block. Result shows good load balance and high efficiency of our method.

Zhang, Jiang↗
DOE OSTI2020

A Visual Analytics Framework for Reviewing Streaming Performance Data

Understanding and tuning the performance of extreme-scale parallel computing systems demands a streaming approach due to the computational cost of applying offline algorithms to vast amounts of performance log data. Analyzing large streaming data is challenging because the rate of receiving data and limited time to comprehend data make it difficult for the analysts to sufficiently examine the data without missing important changes or patterns. To support streaming data analysis, we introduce a visual analytic framework comprising of three modules: data management, analysis, and interactive visualization. The data management module collects various computing and communication performance metrics from the monitored system using streaming data processing techniques and feeds the data to the other two modules. The analysis module automatically identifies important changes and patterns at the required latency. In particular, we introduce a set of online and progressive analysis methods for not only controlling the computational costs but also helping analysts better follow the critical aspects of the analysis results. Finally, the interactive visualization module provides the analysts with a coherent view of the changes and patterns in the continuously captured performance data. Through a multi-faceted case study on performance analysis of parallel discrete-event simulation, we demonstrate the effectiveness of our framework for identifying bottlenecks and locating outliers.

Human-centered computing↗
DOE OSTI2018

NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure

The evergrowing trend of big data has led scientists to share and transfer the simulation and analytical data across the geodistributed research and computing facilities. However, the existing data transfer frameworks used for data sharing lack the capability to adopt the attributes of the underlying parallel file systems (PFS). LADS (Layout-Aware Data Scheduling) is an end-to-end data transfer tool optimized for terabit network using a layout-aware data scheduling via PFS. However, it does not consider the NUMA (Nonuniform Memory Access) architecture. In this paper, we propose a NUMA-aware thread and resource scheduling for optimized data transfer in terabit network. First, we propose distributed RMA buffers to reduce memory controller contention in CPU sockets and then schedule the threads based on CPU socket and NUMA nodes inside CPU socket to reduce memory access latency. We design and implement the proposed resource and thread scheduling in the existing LADS framework. Experimental results showed from 21.7% to 44% improvement with memory-level optimizations in the LADS framework as compared to the baseline without any optimization.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1976

Architecture and data processing alternatives for the TSE computer. Volume 3: Execution of a parallel counting algorithm using array logic (Tse) devices

A parallel algorithm for counting the number of logic-l elements in a binary array or image developed during preliminary investigation of the Tse concept is described. The counting algorithm is implemented using a basic combinational structure. Modifications which improve the efficiency of the basic structure are also presented. A programmable Tse computer structure is proposed, along with a hardware control unit, Tse instruction set, and software program for execution of the counting algorithm. Finally, a comparison is made between the different structures in terms of their more important characteristics.

Metcalfe, A. G.↗
DOE OSTI2021

ParChain: a framework for parallel hierarchical agglomerative clustering using nearest-neighbor chain

This paper studies the hierarchical clustering problem, where the goal is to produce a dendrogram that represents clusters at varying scales of a data set. We propose the ParChain framework for designing parallel hierarchical agglomerative clustering (HAC) algorithms, and using the framework we obtain novel parallel algorithms for the complete linkage, average linkage, and Ward's linkage criteria. Compared to most previous parallel HAC algorithms, which require quadratic memory, our new algorithms require only linear memory, and are scalable to large data sets. ParChain is based on our parallelization of the nearest-neighbor chain algorithm, and enables multiple clusters to be merged on every round. We introduce two key optimizations that are critical for efficiency: a range query optimization that reduces the number of distance computations required when finding nearest neighbors of clusters, and a caching optimization that stores a subset of previously computed distances, which are likely to be reused. Experimentally, we show that our highly-optimized implementations using 48 cores with two-way hyper-threading achieve 5.8--110.1x speedup over state-of-the-art parallel HAC algorithms and achieve 13.75--54.23x self-relative speedup. Compared to state-of-the-art algorithms, our algorithms require up to 237.3x less space. Our algorithms are able to scale to data set sizes with tens of millions of points, which existing algorithms are not able to handle.

Computer Science↗
NASA NTRS1976

Linear dimensionality of Landsat agricultural data with implications for classification

A model for the Landsat multispectral scanner data, representing a generalization of the commonly used Gaussian model, has been formulated and analyzed. The model hypothesizes that the data for different crop types essentially lie on distinct hyperplanes in the feature space. Tests of this model reveal that: (1) the agricultural data from any single acquisition (i.e., four-channel) of Landsat are essentially two dimensional, regardless of the crop type; and (2) the data from different sites and different stages of crop development all lie on planes which are parallel. These findings have significant implications for data display, classification, feature extraction, and signature extension.

Wheeler, S. G.↗
NASA NTRS2021

The role of analysis error in the convergence of reanalysis production streams in MERRA-2

Due to production time constraints, most reanalyses are produced in multiple parallel streams instead of a single continuous one. These streams cover separate segments of the reanalysis time period with short overlaps to allow reconstruction of the official record. A fundamental assumption justifying this approach is that the streams will be assimilating the same observations during the periods where they overlap, and so will eventually converge to a similar atmospheric state, making discontinuities at stream junctions negligible. This assumption is revisited in this work by examining the impact of analysis error on the differences between MERRA-2 overlapping streams in three historical periods. Comparison results are shown in terms of standard deviations of stream differences as well as the spectral decomposition of the variance of their differences. Residual differences were found at the end of each year of overlap, with larger values observed in the earlier segments of the presatellite era. By drawing parallels with analysis error statistics estimated from the GMAO OSSE system, these differences are shown to reflect the varying constraint of data with the varying observing network, and to further carry the imprint of errors that the data assimilation process is not able to mitigate. As such, they are unlikely to be reduced by longer spinup periods. The ability of data assimilation to ensure continuity in the parallel streams is put into question when the observing system coverage is inadequate or simply when the data assimilation system as a whole is suboptimal.

Amal El Akkraoui↗
DOE OSTI2016

Thermally determining flow and/or heat load distribution in parallel paths

A method including obtaining calibration data for at least one sub-component in a heat transfer assembly, wherein the calibration data comprises at least one indication of coolant flow rate through the sub-component for a given surface temperature delta of the sub-component and a given heat load into said sub-component, determining a measured heat load into the sub-component, determining a measured surface temperature delta of the sub-component, and determining a coolant flow distribution in a first flow path comprising the sub-component from the calibration data according to the measured heat load and the measured surface temperature delta of the sub-component.

42 ENGINEERING↗
DOE OSTI2017

Systems and methods for rapid processing and storage of data

Systems and methods of building massively parallel computing systems using low power computing complexes in accordance with embodiments of the invention are disclosed. A massively parallel computing system in accordance with one embodiment of the invention includes at least one Solid State Blade configured to communicate via a high performance network fabric. In addition, each Solid State Blade includes a processor configured to communicate with a plurality of low power computing complexes interconnected by a router, and each low power computing complex includes at least one general processing core, an accelerator, an I/O interface, and cache memory and is configured to communicate with non-volatile solid state memory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Thermally determining flow and/or heat load distribution in parallel paths

A method including obtaining calibration data for at least one sub-component in a heat transfer assembly, wherein the calibration data comprises at least one indication of coolant flow rate through the sub-component for a given surface temperature delta of the sub-component and a given heat load into said sub-component, determining a measured heat load into the sub-component, determining a measured surface temperature delta of the sub-component, and determining a coolant flow distribution in a first flow path comprising the sub-component from the calibration data according to the measured heat load and the measured surface temperature delta of the sub-component.

42 ENGINEERING↗