Search NASA⌕ Search

SEARCH · Search NASA

Results for “Scientific data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

DIRAQ: scalable in situ data- and resource-aware indexing for optimized query performance

Scientific data analytics in high-performance computing environments has been evolving along with the advancement of computing capabilities. With the onset of exascale computing, the increasing gap between compute performance and I/O bandwidth has rendered the traditional post-simulation processing a tedious process. Despite the challenges due to increased data production, there exists an opportunity to benefit from "cheap" computing power to perform query-driven exploration and visualization during simulation time. To accelerate such analyses, applications traditionally augment, post-simulation, raw data with large indexes, which are then repeatedly utilized for data exploration. However, the generation of current state-of-the-art indexes involves a compute- and memory-intensive processing, thus rendering them inapplicable in an in situ context. In this paper we propose DIRAQ, a parallel in situ, in network data encoding and reorganization technique that enables the transformation of simulation output into a query-efficient form, with negligible runtime overhead to the simulation run. DIRAQ's effective core-local, precision-based encoding approach incorporates an embedded compressed index that is 3---6 $$\times $$ ? smaller than current state-of-the-art indexing schemes. Its data-aware index adjustmentation improves performance of group-level index layout creation by up to 35 % and reduces the size of the generated index by up to 27 %. Moreover, DIRAQ's in network index merging strategy enables the creation of aggregated indexes that speed up spatial-context query responses by up to $$10\times $$ 10 ? versus alternative techniques. DIRAQ's topology-, data-, and memory-aware aggregation strategy results in efficient I/O and yields overall end-to-end encoding and I/O time that is less than that required to write the raw data with MPI collective I/O.

Computer Science↗

A Case for Database Filesystems

Data intensive science is offering new challenges and opportunities for Information Technology and traditional relational databases in particular. Database filesystems offer the potential to store Level Zero data and analyze Level 1 and Level 3 data within the same database system [2]. Scientific data is typically composed of both unstructured files and scalar data. Oracle SecureFiles is a new database filesystem feature in Oracle Database 11g that is specifically engineered to deliver high performance and scalability for storing unstructured or file data inside the Oracle database. SecureFiles presents the best of both the filesystem and the database worlds for unstructured content. Data stored inside SecureFiles can be queried or written at performance levels comparable to that of traditional filesystems while retaining the advantages of the Oracle database.

99 GENERAL AND MISCELLANEOUS↗

Apollo experience report: Apollo lunar surface experiments package data processing system

Apollo Program experience in the processing of scientific data from the Apollo lunar surface experiments package, in which computers and associated hardware and software were used, is summarized. The facility developed for the preprocessing of the lunar science data is described, as are several computer facilities and programs used by the Principal Investigators. The handling, processing, and analyzing of lunar science data and the interface with the Principal Investigators are discussed. Pertinent problems that arose in the development of the data processing schemes are discussed so that future programs may benefit from the solutions to the problems. The evolution of the data processing techniques for lunar science data related to recommendations for future programs of this type.

Eason, R. L.↗

Automated storage and retrieval of data obtained in the Interkosmos project

The formation of a data bank and information retrieval system for scientific data is described. The stored data can be digital or documentation data. Data classification methods are discussed along with definition and compilation of the dictionary utilized, definition of the indexing scheme, and definition of the principles used in constructing a file for documents, data blocks, and tapes. Operating principles are also presented.

Ziolkovski, K.↗

Composition Measurements at the Magnetopause and in the Plasma Mantle

This final report describes activities under NASA grant NAGW-4049 to Lockheed Missiles and Space Company. The report covers the entire period of the grant from 15 August 1994 to 31 January 1998. The original grant was for 3 years ending in August 1997; however the grant was extended 6 months to accomodate additional data analysis that added significantly to the scientific results. This is a grant under the NASA Supporting Research and Technology Program for the analysis and interpretation of the combined scientific data from the ISEE-1 Plasma Composition Experiment and the AMPTE/CCE Hot Plasma Composition Experiment. These combined data sets were used in a study of the Earth's magnetopause to develop a fundamental understanding of plasma entry and dynamics at the boundary and formation and maintenance of the low latitude boundary layer under a variety of solar wind and magnetospheric conditions and at a wide range of local times.

Gary, S. P.↗

Training Under-Represented Students in Biological Research at Fisk University

The objectives of our training and research project in biology at Fisk are to motivate and train our African-American undergraduate and graduate students by (a) teaching the basic principles and applications of different biological, biochemical and biophysical research techniques; (b) providing a "hands on experience" with laboratory instrumentation (c) requiring the students to participate in the proposed research project entitled "Cyanobacterial Bioreactors for Oxygen and Ammonia Production under "CELSS" Conditions" to gain confidence in independently conducting experiments and (d) providing training in scientific data collection and presentation to peers in scientific conferences or meetings.

Gunasekaran, Muthukumaran↗

Defining the Core Archive Data Standards of the International Planetary Data Alliance (IPDA)

A goal of the International Planetary Data Alliance (lPDA) is to develop a set of archive data standards that enable the sharing of scientific data across international agencies and missions. To help achieve this goal, the IPDA steering committee initiated a six month proj ect to write requirements for and draft an information model based on the Planetary Data System (PDS) archive data standards. The project had a special emphasis on data formats. A set of use case scenarios were first developed from which a set of requirements were derived for the IPDA archive data standards. The special emphasis on data formats was addressed by identifying data formats that have been used by PDS nodes and other agencies in the creation of successful data sets for the Planetary Data System (PDS). The dependency of the IPDA information model on the PDS archive standards required the compilation of a formal specification of the archive standards currently in use by the PDS. An ontology modelling tool was chosen to capture the information model from various sources including the Planetary Science Data Dictionary [I] and the PDS Standards Reference [2]. Exports of the modelling information from the tool database were used to produce the information model document using an object-oriented notation for presenting the model. The tool exports can also be used for software development and are directly accessible by semantic web applications.

ontology↗

Miniature L-Band Radar Transceiver

A miniature L-band transceiver that operates at a carrier frequency of 1.25 GHz has been developed as part of a generic radar electronics module (REM) that would constitute one unit in an array of many identical units in a very-large-aperture phased-array antenna. NASA and the Department of Defense are considering the deployment of such antennas in outer space; the underlying principles of operation, and some of those of design, also are applicable on Earth. The large dimensions of the antennas make it advantageous to distribute radio-frequency electronic circuitry into elements of the arrays. The design of the REM is intended to implement the distribution. The design also reflects a requirement to minimize the size and weight of the circuitry in order to minimize the weight of any such antenna. Other requirements include making the transceiver robust and radiation-hard and minimizing power demand. Figure 1 depicts the functional blocks of the REM, including the L-band transceiver. The key functions of the REM include signal generation, frequency translation, amplification, detection, handling of data, and radar control and timing. An arbitrary-waveform generator that includes logic circuitry and a digital-to-analog converter (DAC) generates a linear-frequency-modulation chirp waveform. A frequency synthesizer produces local-oscillator signals used for frequency conversion and clock signals for the arbitrary-waveform generator, for a digitizer [that is, an analog-to-digital converter (ADC)], and for a control and timing unit. Digital functions include command, timing, telemetry, filtering, and high-rate framing and serialization of data for a high-speed scientific-data interface. The aforementioned digital implementation of filtering is a key feature of the REM architecture. Digital filters, in contradistinction to analog ones, provide consistent and temperature-independent performance, which is particularly important when REMs are distributed throughout a large array. Digital filtering also enables selection among multiple filter parameters as required for different radar operating modes. After digital filtering, data are decimated appropriately in order to minimize the data rate out of an antenna panel. The L-band transceiver (see Figure 2) includes a radio-frequency (RF)-to-baseband down-converter chain and an intermediate- frequency (IF)-to-RF up-converter chain. Transmit/receive (T/R) switches enable the use of a single feed to the antenna for both transmission and reception. The T/R switches also afford a built-in test capability by enabling injection of a calibration signal into the receiver chain. In order of decreasing priority, components of the transceiver were selected according to requirements of radiation hardness, then compactness, then low power. All of the RF components are radiation-hard. The noise figure (NF) was optimized to the extent that (1) a low-noise amplifier (LNA) (characterized by NF < 2 dB) was selected but (2) the receiver front-end T/R switches were selected for a high degree of isolation and acceptably low loss, regardless of the requirement to minimize noise.

McWatters, Dalia↗

EMPRESS

EMPRESS provides metadata management capabilities intended to address annotating data within a single variable in a scientific data set rather than at the variable or entire file granularity level

Lawson, Margaret↗

Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP

With ever-increasing volumes of scientific data produced by high-performance computing applications, significantly reducing data size is critical because of limited capacity of storage space and potential bottlenecks on I/O or networks in writing/reading or transferring data. SZ and ZFP are two leading BSD licensed open source C/C++ libraries for compressed floating-point arrays that support high throughput read and write random access. However, their performance is not consistent across different data sets and across different fields of some data sets, which raises the need for an automatic online (during compression) selection between SZ and ZFP, with minimal overhead. In this paper, the automatic selection optimizes the rate-distortion, an important statistical quality metric based on the signal-to-noise ratio. To optimize for rate-distortion, we investigate the principles of SZ and ZFP. We then propose an efficient online, low-overhead selection algorithm that predicts the compression quality accurately for two compressors in early processing stages and selects the best-fit compressor for each data field. Here, we implement the selection algorithm into an open-source library, and we evaluate the effectiveness of our proposed solution against plain SZ and ZFP in a parallel environment with 1,024 cores. Evaluation results on three data sets representing about 100 fields show that our selection algorithm improves the compression ratio up to 70 percent with the same level of data distortion because of very accurate selection (around 99 percent) of the bestfit compressor, with little overhead (less than 7 percent in the experiments).

97 MATHEMATICS AND COMPUTING↗

An Application of Multivariate Statistical Analysis for Query-Driven Visualization

Driven by the ability to generate ever-larger, increasingly complex data, there is an urgent need in the scientific community for scalable analysis methods that can rapidly identify salient trends in scientific data. Query-Driven Visualization (QDV) strategies are among the small subset of techniques that can address both large and highly complex datasets. This paper extends the utility of QDV strategies with a statistics-based framework that integrates non-parametric distribution estimation techniques with a new segmentation strategy to visually identify statistically significant trends and features within the solution space of a query. In this framework, query distribution estimates help users to interactively explore their query's solution and visually identify the regions where the combined behavior of constrained variables is most important, statistically, to their inquiry. Our new segmentation strategy extends the distribution estimation analysis by visually conveying the individual importance of each variable to these regions of high statistical significance. We demonstrate the analysis benefits these two strategies provide and show how they may be used to facilitate the refinement of constraints over variables expressed in a user's query. We apply our method to datasets from two different scientific domains to demonstrate its broad applicability.

97 MATHEMATICS AND COMPUTING↗

Lifting MGARD: Construction of (pre)wavelets on the interval using polynomial predictors of arbitrary order

MGARD (MultiGrid Adaptive Reduction of Data) is an algorithm for compressing and refactoring scientific data, based on the theory of multigrid methods. The core algorithm is built around stable multilevel decompositions of conforming piecewise linear $C^0$ finite element spaces, enabling accurate error control in various norms and derived quantities of interest. In this work, we extend this construction to arbitrary order Lagrange finite elements $\mathbb{Q}_p$, $p \geq 0$, and propose a reformulation of the algorithm as a lifting scheme with polynomial predictors of arbitrary order. Additionally, a new formulation using a compactly supported wavelet basis is discussed, and an explicit construction of the proposed wavelet transform for uniform dyadic grids is described.

Reshniak, Viktor [Oak Ridge National Laboratory (O↗

Pioneer Venus Unified Abstract Data Library and Quick Look Data Delivery System

Development of the Pioneer Venus (PV) Unified Abstract Data System (UADS) and Quick Look Data System (QLDS) was prompted by the need to provide PV investigators rapid and easy access to PV mission data. The UADS is intended to maximize the scientific benefits of the mission by facilitating the exchange of reduced scientific data. QLDS provides a method by which sampled daily mission data is rapidly transmitted to principal investigators providing them a quick look at that orbit's data.

Ferandin, J. A.↗

Database Management

Management of the data within a planetary data system (PDS) is addressed. Principles of modern data management are described and several large NASA scientific data base systems are examined. Data management in PDS is outlined and the major data management issues are introduced.

Source record↗

Post-Flight Estimation of Motion of Space Structures: Part 1

A computer program estimates the relative positions and orientations of two space structures from data on the angular positions and distances of fiducial objects on one structure as measured by a target tracking electronic camera and laser range finders on another structure. The program is written specifically for determining the relative alignments of two antennas, connected by a long truss, deployed in outer space from a space shuttle. The program is based partly on transformations among the various coordinate systems involved in the measurements and on a nonlinear mathematical model of vibrations of the truss. The program implements a Kalman filter that blends the measurement data with data from the model. Using time series of measurement data from the tracking camera and range finders, the program generates time series of data on the relative position and orientation of the antennas. A similar program described in a prior NASA Tech Briefs article was used onboard for monitoring the structures during flight. The present program is more precise and designed for use on Earth in post-flight processing of the measurement data to enable correction, for antenna motions, of scientific data acquired by use of the antennas.

Brugarolas, Paul↗

Extending the Publish/Subscribe Abstraction for High-Performance I/O and Data Management at Extreme Scale

The Adaptable I/O System (ADIOS) represents the culmination of substantial investment in Scientific Data Management, and it has demonstrated success for several important extreme-scale science cases. However, looking towards the exascale and beyond, we see the development of yet more stringent data management requirements that require new abstractions. Therefore, there is an opportunity to attempt to connect the traditional realms of HPC I/O optimization with the Database / Data Management community. As such, in this paper we offer some specific examples from our ongoing work in managing data structures, services, and performance at the extreme scale for scientific computing. Using the publish/subscribe model afforded by ADIOS, we demonstrate a set of services that connect data format, metadata, queries, data reduction, and high-performance delivery. The resulting publish/subscribe framework facilitates connection to on-line workflow systems to enable the dynamic capabilities that will be required for exascale science.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Tigres Workflow Library: Supporting Scientific Pipelines on HPC Systems

The growth in scientific data volumes has resulted in the need for new tools that enable users to operate on and analyze data on large-scale resources. In the last decade, a number of scientific workflow tools have emerged. These tools often target distributed environments, and often need expert help to compose and execute the workflows. Data-intensive workflows are often ad-hoc, they involve an iterative development process that includes users composing and testing their workflows on desktops, and scaling up to larger systems. In this paper, we present the design and implementation of Tigres, a workflow library that supports the iterative workflow development cycle of data-intensive workflows. Tigres provides an application programming interface to a set of programming templates i.e., sequence, parallel, split, merge, that can be used to compose and execute computational and data pipelines. We discuss the results of our evaluation of scientific and synthetic workflows showing Tigres performs with minimal template overheads (mean of 13 seconds over all experiments). We also discuss various factors (e.g., I/O performance, execution mechanisms) that affect the performance of scientific workflows on HPC systems.

97 MATHEMATICS AND COMPUTING↗