Search NASA⌕ Search

SEARCH · Search NASA

Results for “data parallel”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,387 records · Page 77
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

Optical distributed sensors for feedback control: Characterization of photorefractive resonator

The aim of the project was to explore, define, and assess the possibilities of optical distributed sensing for feedback control. This type of sensor, which may have some impacts in the dynamic control of deformable structures and the monitoring of small displacements, can be divided into data acquisition, data processing, and control design. Analogue optical techniques, because they are noninvasive and afford massive parallelism may play a significant role in the acquisition and the preprocessing of the data for such a sensor. Assessing these possibilities was the aim of the first stage of this project. The scope of the proposed research was limited to: (1) the characterization of photorefractive resonators and the assessment of their possible use as a distributed optical processing element; and (2) the design of a control system utilizing signals from distributed sensors. The results include a numerical and experimental study of the resonator below threshold, an experimental study of the effect of the resonator's transverse confinement on its dynamics above threshold, a numerical study of the resonator above threshold using a modal expansion approach, and the experimental test of this model. A detailed account of each investigation, including methodology and analysis of the results are also included along with reprints of published and submitted papers.

Indebetouw, Guy↗
DOE OSTI2016

HPC-NMF: A High-Performance Parallel Algorithm for Nonnegative Matrix Factorization

NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient distributed algorithms to solve the problem for big data sets. We propose a high-performance distributed-memory parallel algorithm that computes the factorization by iteratively solving alternating non-negative least squares (NLS) subproblems for $\WW$ and $\HH$. It maintains the data and factor matrices in memory (distributed across processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild assumptions). As opposed to previous implementation, our algorithm is also flexible: It performs well for both dense and sparse matrices, and allows the user to choose any one of the multiple algorithms for solving the updates to low rank factors $\WW$ and $\HH$ within the alternating iterations.

Kannan, Ramakrishnan↗
NASA NTRS1994

Comparison of recalculated Dobson and TOMS total ozone at Hradec Kralove, Czechoslovakia, 1978-1990

The reevaluated Dobson total ozone data from Hradec Kralove, Czechoslovakia were compared with independent Total Ozone Mapping Spectrophotometer (TOMS) 'version 6' data set. The comparison was performed by means of the parallel daily averages of ground-based and satellite total ozone pairs of the period November 1978 to December 1990. The comparison showed slight differences between both data series. Their average relative difference is 0.48 percent. The similar results have been reached for subsets of direct sun and zenith types of measurements as well. Their relative differences are 0.61 percent and 0.11 percent respectively. These facts indicate not only good mutual relation of both data sources but also reliability and accuracy of the zenith charts of the spectrophotometer No. 74 used at Hradec Kralove. Preliminary assessment of seasonal MU-dependence of the differences between Dobson and TOMS data was made while using total ozones of winter and summer months representing values of MU=2.70-5.20 and MU = 1.12-1.30 respectively. The results did not show systematic underestimation or overestimation of total ozone due to MU-dependence of the instrument at Hradec Kralove in both seasons.

Stanek, Martin↗
NASA NTRS1977

Continuous data stream FIFO magnetic bubble domain shift register

There is described a simple first-in, first out (FIFO) magnetic bubble domain shift register which has continuous data storage capability. In a preferred embodiment, two parallel detector branches are associated with a main storage loop. The bubbles in the storage loop are replicated toward each detector branch by passive replicators in the storage loop. Annihilators associated with each of the replicators and each of the detectors are arranged an appropriate distance from the replicators so that selective energization of the annihilators permits certain bits to propagate to one detector and other bits to propagate to the other detector. Connection of the detectors in an appropriate bridge circuit permits full data rate reclamation.

Chen, Thomas T.↗
DOE OSTI2015

Evolution of CMS Workload Management Towards Multicore Job Support

The successful exploitation of multicore processor architectures is a key element of the LHC distributed computing system in the coming era of the LHC Run 2. High-pileup complex-collision events represent a challenge for the traditional sequential programming in terms of memory and processing time budget. The CMS data production and processing framework is introducing the parallel execution of the reconstruction and simulation algorithms to overcome these limitations. CMS plans to execute multicore jobs while still supporting singlecore processing for other tasks difficult to parallelize, such as user analysis. The CMS strategy for job management thus aims at integrating single and multicore job scheduling across the Grid. This is accomplished by employing multicore pilots with internal dynamic partitioning of the allocated resources, capable of running payloads of various core counts simultaneously. An extensive test programme has been conducted to enable multicore scheduling with the various local batch systems available at CMS sites, with the focus on the Tier-0 and Tier-1s, responsible during 2015 of the prompt data reconstruction. Scale tests have been run to analyse the performance of this scheduling strategy and ensure an efficient use of the distributed resources. This paper presents the evolution of the CMS job management and resource provisioning systems in order to support this hybrid scheduling model, as well as its deployment and performance tests, which will enable CMS to transition to a multicore production model for the second LHC run.

Perez-Calero Yzquierdo, A.↗
NASA NTRS2004

Benchmarking Memory Performance with the Data Cube Operator

Data movement across a computer memory hierarchy and across computational grids is known to be a limiting factor for applications processing large data sets. We use the Data Cube Operator on an Arithmetic Data Set, called ADC, to benchmark capabilities of computers and of computational grids to handle large distributed data sets. We present a prototype implementation of a parallel algorithm for computation of the operatol: The algorithm follows a known approach for computing views from the smallest parent. The ADC stresses all levels of grid memory and storage by producing some of 2d views of an Arithmetic Data Set of d-tuples described by a small number of integers. We control data intensity of the ADC by selecting the tuple parameters, the sizes of the views, and the number of realized views. Benchmarking results of memory performance of a number of computer architectures and of a small computational grid are presented.

Frumkin, Michael A.↗
NASA NTRS2012

Radiation-Hard SpaceWire/Gigabit Ethernet-Compatible Transponder

A radiation-hard transponder was developed utilizing submicron/nanotechnology from IBM. The device consumes low power and has a low fabrication cost. This device utilizes a Plug-and-Play concept, and can be integrated into intra-satellite networks, supporting SpaceWire and Gigabit Ethernet I/O. A space-qualified, 100-pin package also was developed, allowing space-qualified (class K) transponders to be delivered within a six-month time frame. The novel, optical, radiation-tolerant transponder was implemented as a standalone board, containing the transponder ASIC (application specific integrated circuit) and optical module, with an FPGA (field-programmable gate array) friendly parallel interface. It features improved radiation tolerance; high-data-rate, low-power consumption; and advanced functionality. The transponder utilizes a patented current mode logic library of radiation-hardened-by-architecture cells. The transponder was developed, fabricated, and radhard tested up to 1 MRad. It was fabricated using 90-nm CMOS (complementary metal oxide semiconductor) 9 SF process from IBM, and incorporates full BIT circuitry, allowing a loop back test. The low-speed parallel LVCMOS (lowvoltage complementary metal oxide semiconductor) bus is compatible with Actel FPGA. The output LVDS (low-voltage differential signaling) interface operates up to 1.5 Gb/s. Built-in CDR (clock-data recovery) circuitry provides robust synchronization and incorporates two alarm signals such as synch loss and signal loss. The ultra-linear peak detector scheme allows on-line control of the amplitude of the input signal. Power consumption is less than 300 mW. The developed transponder with a 1.25 Gb/s serial data rate incorporates a 10-to-1 serializer with an internal clock multiplication unit and a 10-1 deserializer with internal clock and data recovery block, which can operate with 8B10B encoded signals. Three loop-back test modes are provided to facilitate the built-in-test functionality. The design is based on a proprietary library of differential current switching logic cells implemented in the standard 90-nm CMOS 9SF technology from IBM. The proprietary low-power LVDS physical interface is fully compatible with the SpaceWire standard, and can be directly connected to the SFP MSA (small form factor pluggable Multiple Source Agreement) optical transponder. The low-speed parallel interfaces are fully compatible with the standard 1.8 V CMOS input/output devices. The utilized proprietary annular CMOS layout structures provide TID tolerance above 1.2 MRad. The complete chip consumes less than 150 mW of power from a single 1.8-V positive supply source.

Katzman, Vladimir↗
NASA NTRS1994

Parallel digital modem using multirate digital filter banks

A new class of architectures for an all-digital modem is presented in this report. This architecture, referred to as the parallel receiver (PRX), is based on employing multirate digital filter banks (DFB's) to demodulate, track, and detect the received symbol stream. The resulting architecture is derived, and specifications are outlined for designing the DFB for the PRX. The key feature of this approach is a lower processing rate then either the Nyquist rate or the symbol rate, without any degradation in the symbol error rate. Due to the freedom in choosing the processing rate, the designer is able to arbitrarily select and use digital components, independent of the speed of the integrated circuit technology. PRX architecture is particularly suited for high data rate applications, and due to the modular structure of the parallel signal path, expansion to even higher data rates is accommodated with each. Applications of the PRX would include gigabit satellite channels, multiple spacecraft, optical links, interactive cable-TV, telemedicine, code division multiple access (CDMA) communications, and others.

Sadr, Ramin↗
DOE OSTI2019

Detecting Data-Races in High-Performance Computing

High-performance computing is susceptible to data-race bugs. Data races occur when sequences of computed events do not happen in the intended order, a significant barrier to the integrity of parallel programs using shared-memory architectures. Many approaches to data-race detection are based on traditional static-analysis techniques that typically execute on a program’s source code or control-flow graph (the graphical representation of all possible orders of events in the execution of a program). Similar to classical data-flow analyses, such tools can produce false alarms that report spurious race conditions on error free programs. We have developed a software verification technique that requires as input only a program’s source code and can automatically detect a data-race bug which definitely exists, determine that a program is definitely data-race free, or report that it cannot determine either of the two cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Enabling Parallel Execution of System-level Simulations in SAM

This report summarizes the recent code updates related to “element ghosting” in SAM to enable the parallel execution of system-level simulations using multiple processors/cores. Unlike typical MOOSE-based applications, for system-level simulations, SAM mostly deals with a collection of discrete small pieces of meshes, and the connection of physics on these meshes are realized by using “connector” types of components/code structures, such as conjugate heat transfer and flow junctions. The required code implementation is to correctly mark the necessary ghost elements for each type of such components/code structures; thus, the lower-level libraries can correctly perform the necessary data transfer between processors (CPUs) when executed in parallel mode. After the code updates, SAM can now run system-level simulations in the parallel mode. The parallel execution capability was then tested with an ABTR input model with 23k DOFs. Significant speedup was demonstrated when the optimal number of CPUs were used in parallel mode. Future systematic studies on parallelization performance using additional test cases covering different physics/scenarios will be needed to provide additional insights into the scalability of SAM parallelization.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS2016

IN13B-1660: Analytics and Visualization Pipelines for Big Data on the NASA Earth Exchange (NEX) and OpenNEX

We are developing capabilities for an integrated petabyte-scale Earth science collaborative analysis and visualization environment. The ultimate goal is to deploy this environment within the NASA Earth Exchange (NEX) and OpenNEX in order to enhance existing science data production pipelines in both high-performance computing (HPC) and cloud environments. Bridging of HPC and cloud is a fairly new concept under active research and this system significantly enhances the ability of the scientific community to accelerate analysis and visualization of Earth science data from NASA missions, model outputs and other sources. We have developed a web-based system that seamlessly interfaces with both high-performance computing (HPC) and cloud environments, providing tools that enable science teams to develop and deploy large-scale analysis, visualization and QA pipelines of both the production process and the data products, and enable sharing results with the community. Our project is developed in several stages each addressing separate challenge - workflow integration, parallel execution in either cloud or HPC environments and big-data analytics or visualization. This work benefits a number of existing and upcoming projects supported by NEX, such as the Web Enabled Landsat Data (WELD), where we are developing a new QA pipeline for the 25PB system.

visualization↗
DOE OSTI2018

A Methodology for the Rapid Development of Scalable HPC Data Services

There is growing evidence in the scientifc computing community that parallel file systems are not sufficient for all HPC storage workloads. This realization has motivated extensive research in new storage system designs. The question of which alternative we should turn towards implies that there could be a single answer satisfying a wide range of very diverse applications. We argue that such a generic solution does not exist. Instead, custom data services should be designed and tailored to the needs of specific applications on specific hardware. Furthermore, custom data services should be designed in close collaboration with users. In this paper, we present a methodology for the rapid development of such data services. This methodology promotes the design of reusable building blocks that can be composed together efficiently through a runtime based on high-performance threading, tasking, and remote procedure calls. We illustrate the success of our methodology by showcasing three examples of data services designed from the same building blocks, yet targeting entirely different applications.

Dorier, Mattieu↗
NASA NTRS1986

Convection beneath young oceanic lithosphere - Implications for thermal structure and gravity

Small-scale convection under the oceanic lithosphere which begins in the first 5 m.y. of cooling can produce a gravity signal with the amplitude and wavelength observed for large areas of the central Pacific and southern Indian oceans using Seasat altimeter data. The trend of the observed anomalies is parallel to the direction of plate motion as might be expected if they were produced by small-scale convection. Models predict that the wavelengths of gravity anomalies increase more rapidly with age than is observed. The persistence of short relatively uniform wavelength anomalies (less than 200 km) to crustal ages of 50 Ma may indicate that they were produced when the lithosphere was very young and thin and were 'frozen in' as cooling thickened the elastic lithosphere. Small-scale convection which begins under very young lithosphere does not violate other geophysical data such as the rate of seafloor subsidence and variations of geoid height with age. After convection has begun, the subsidence due to thermal contraction within the lithosphere varies linearly with age, in the absence of mantle heat sources, although the rate of change of these quantities is affected by convection. Much of the variation of the geoid height across fracture zones can be fit by a model which includes small-scale convection.

Buck, W. R.↗
DOE OSTI2016

Tigres Workflow Library: Supporting Scientific Pipelines on HPC Systems

The growth in scientific data volumes has resulted in the need for new tools that enable users to operate on and analyze data on large-scale resources. In the last decade, a number of scientific workflow tools have emerged. These tools often target distributed environments, and often need expert help to compose and execute the workflows. Data-intensive workflows are often ad-hoc, they involve an iterative development process that includes users composing and testing their workflows on desktops, and scaling up to larger systems. In this paper, we present the design and implementation of Tigres, a workflow library that supports the iterative workflow development cycle of data-intensive workflows. Tigres provides an application programming interface to a set of programming templates i.e., sequence, parallel, split, merge, that can be used to compose and execute computational and data pipelines. We discuss the results of our evaluation of scientific and synthetic workflows showing Tigres performs with minimal template overheads (mean of 13 seconds over all experiments). We also discuss various factors (e.g., I/O performance, execution mechanisms) that affect the performance of scientific workflows on HPC systems.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2011

Parallel Wavefront Analysis for a 4D Interferometer

This software provides a programming interface for automating data collection with a PhaseCam interferometer from 4D Technology, and distributing the image-processing algorithm across a cluster of general-purpose computers. Multiple instances of 4Sight (4D Technology s proprietary software) run on a networked cluster of computers. Each connects to a single server (the controller) and waits for instructions. The controller directs the interferometer to several images, then assigns each image to a different computer for processing. When the image processing is finished, the server directs one of the computers to collate and combine the processed images, saving the resulting measurement in a file on a disk. The available software captures approximately 100 images and analyzes them immediately. This software separates the capture and analysis processes, so that analysis can be done at a different time and faster by running the algorithm in parallel across several processors. The PhaseCam family of interferometers can measure an optical system in milliseconds, but it takes many seconds to process the data so that it is usable. In characterizing an adaptive optics system, like the next generation of astronomical observatories, thousands of measurements are required, and the processing time quickly becomes excessive. A programming interface distributes data processing for a PhaseCam interferometer across a Windows computing cluster. A scriptable controller program coordinates data acquisition from the interferometer, storage on networked hard disks, and parallel processing. Idle time of the interferometer is minimized. This architecture is implemented in Python and JavaScript, and may be altered to fit a customer s needs.

Rao, Shanti R.↗
NASA NTRS2016

Benchmark Comparison of Cloud Analytics Methods Applied to Earth Observations

Cloud computing has the potential to bring high performance computing capabilities to the average science researcher. However, in order to take full advantage of cloud capabilities, the science data used in the analysis must often be reorganized. This typically involves sharding the data across multiple nodes to enable relatively fine-grained parallelism. This can be either via cloud-based file systems or cloud-enabled databases such as Cassandra, Rasdaman or SciDB. Since storing an extra copy of data leads to increased cost and data management complexity, NASA is interested in determining the benefits and costs of various cloud analytics methods for real Earth Observation cases. Accordingly, NASA's Earth Science Technology Office and Earth Science Data and Information Systems project have teamed with cloud analytics practitioners to run a benchmark comparison on cloud analytics methods using the same input data and analysis algorithms. We have particularly looked at analysis algorithms that work over long time series, because these are particularly intractable for many Earth Observation datasets which typically store data with one or just a few time steps per file. This post will present side-by-side cost and performance results for several common Earth observation analysis operations.

science data management↗
DOE OSTI2025

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗
NASA NTRS1989

Configuration space representation in parallel coordinates

By means of a system of parallel coordinates, a nonprojective mapping from R exp N to R squared is obtained for any positive integer N. In this way multivariate data and relations can be represented in the Euclidean plane (embedded in the projective plane). Basically, R squared with Cartesian coordinates is augmented by N parallel axes, one for each variable. The N joint variables of a robotic device can be represented graphically by using parallel coordinates. It is pointed out that some properties of the relation are better perceived visually from the parallel coordinate representation, and that new algorithms and data structures can be obtained from this representation. The main features of parallel coordinates are described, and an example is presented of their use for configuration space representation of a mechanical arm (where Cartesian coordinates cannot be used).

Fiorini, Paolo↗