Search NASA⌕ Search

SEARCH · Search NASA

Results for “data storage and retrieval”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Viability of S3 Object Storage for the ASC Program at Sandia

Recent efforts at Sandia such as DataSEA are creating search engines that enable analysts to query the institution’s massive archive of simulation and experiment data. The benefit of this work is that analysts will be able to retrieve all historical information about a system component that the institution has amassed over the years and make better-informed decisions in current work. As DataSEA gains momentum, it faces multiple technical challenges relating to capacity storage. From a raw capacity perspective, data producers will rapidly overwhelm the system with massive amounts of data. From an accessibility perspective, analysts will expect to be able to retrieve any portion of the bulk data, from any system on the enterprise network. Sandia’s Institutional Computing is mitigating storage problems at the enterprise level by procuring new capacity storage systems that can be accessed from anywhere on the enterprise network. These systems use the simple storage service, or S3, API for data transfers. While S3 uses objects instead of files, users can access it from their desktops or Sandia’s high-performance computing (HPC) platforms. S3 is particularly well suited for bulk storage in DataSEA, as datasets can be decomposed into object that can be referenced and retrieved individually, as needed by an analyst. In this report we describe our experiences working with S3 storage and provide information about how developers can leverage Sandia’s current systems. We present performance results from two sets of experiments. First, we measure S3 throughput when exchanging data between four different HPC platforms and two different enterprise S3 storage systems on the Sandia Restricted Network (SRN). Second, we measure the performance of S3 when communicating with a custom-built Ceph storage system that was constructed from HPC components. Overall, while S3 storage is significantly slower than traditional HPC storage, it provides significant accessibility benefits that will be valuable for archiving and exploiting historical data. There are multiple opportunities that arise from this work, including enhancing DataSEA to leverage S3 for bulk storage and adding native S3 support to Sandia’s IOSS library.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1981

ESRI applications of GIS technology: Mineral resource development

The application of geographic information systems technology to large scale regional assessment related to mineral resource development, identifying candidate sites for related industry, and evaluating sites for waste disposal is discussed. Efforts to develop data bases were conducted at scales ranging from 1:3,000,000 to 1:25,000. In several instances, broad screening was conducted for large areas at a very general scale with more detailed studies subsequently undertaken in promising areas windowed out of the generalized data base. Increasingly, the systems which are developed are structured as the spatial framework for the long-term collection, storage, referencing, and retrieval of vast amounts of data about large regions. Typically, the reconnaissance data base for a large region is structured at 1:250,000 scale, data bases for smaller areas being structured at 1:25,000, 1:50,000 or 1:63,360. An integrated data base for the coterminous US was implemented at a scale of 1:3,000,000 for two separate efforts.

Derrenbacher, W.↗
NASA NTRS1986

A database management capability for Ada

The data requirements of mission critical defense systems have been increasing dramatically. Command and control, intelligence, logistics, and even weapons systems are being required to integrate, process, and share ever increasing volumes of information. To meet this need, systems are now being specified that incorporate data base management subsystems for handling storage and retrieval of information. It is expected that a large number of the next generation of mission critical systems will contain embedded data base management systems. Since the use of Ada has been mandated for most of these systems, it is important to address the issues of providing data base management capabilities that can be closely coupled with Ada. A comprehensive distributed data base management project has been investigated. The key deliverables of this project are three closely related prototype systems implemented in Ada. These three systems are discussed.

Chan, Arvola↗
NASA NTRS2012

Method for utilizing properties of the sinc(x) function for phase retrieval on nyquist-under-sampled data

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for simulating propagation of an electromagnetic field, performing phase retrieval, or sampling a band-limited function. A system practicing the method generates transformed data using a discrete Fourier transform which samples a band-limited function f(x) without interpolating or modifying received data associated with the function f(x), wherein an interval between repeated copies in a periodic extension of the function f(x) obtained from the discrete Fourier transform is associated with a sampling ratio Q, defined as a ratio of a sampling frequency to a band-limited frequency, and wherein Q is assigned a value between 1 and 2 such that substantially no aliasing occurs in the transformed data, and retrieves a phase in the received data based on the transformed data, wherein the phase is used as feedback to an optical system.

Dean, Bruce H.↗
NASA NTRS1977

AOIPS data base management systems support for GARP data sets

A data base management system is identified, developed to provide flexible access to data sets produced by GARP during its data systems tests. The content and coverage of the data base are defined and a computer-aided, interactive information storage and retrieval system, implemented to facilitate access to user specified data subsets, is described. The computer programs developed to provide the capability were implemented on the highly interactive, minicomputer-based AOIPS and are referred to as the data retrieval system (DRS). Implemented as a user interactive but menu guided system, the DRS permits users to inventory the data tape library and create duplicate or subset data sets based on a user selected window defined by time and latitude/longitude boundaries. The DRS permits users to select, display, or produce formatted hard copy of individual data items contained within the data records.

Gary, J. P.↗
DOE OSTI2020

Science-Driven Data Management for Multi-Tiered Storage (Final Report)

Scientific discovery at the exascale will not be possible without significant new research in the management, storage and retrieval over the long lifespan of the extreme amounts of data that will be produced. Our thesis is that adding application level knowledge about data to guide the actions of the storage system provides substantial benefits to the organization, storage, and access to extreme scale data, resulting in improved productivity for computational science. In this project we will demonstrate novel techniques to facilitate efficient mapping of data objects, even partitioning individual variables, from the user space onto multiple storage tiers, and enable application-guided data reductions and transformations to address capacity and bandwidth bottlenecks. Our goal is to address the associated Input/ Output (I/O) and storage challenges in the context of current and emerging storage landscapes, and expedite insights into mission critical scientific processes.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1997

Digital storage and analysis of color Doppler echocardiograms

Color Doppler flow mapping has played an important role in clinical echocardiography. Most of the clinical work, however, has been primarily qualitative. Although qualitative information is very valuable, there is considerable quantitative information stored within the velocity map that has not been extensively exploited so far. Recently, many researchers have shown interest in using the encoded velocities to address the clinical problems such as quantification of valvular regurgitation, calculation of cardiac output, and characterization of ventricular filling. In this article, we review some basic physics and engineering aspects of color Doppler echocardiography, as well as drawbacks of trying to retrieve velocities from video tape data. Digital storage, which plays a critical role in performing quantitative analysis, is discussed in some detail with special attention to velocity encoding in DICOM 3.0 (medical image storage standard) and the use of digital compression. Lossy compression can considerably reduce file size with minimal loss of information (mostly redundant); this is critical for digital storage because of the enormous amount of data generated (a 10 minute study could require 18 Gigabytes of storage capacity). Lossy JPEG compression and its impact on quantitative analysis has been studied, showing that images compressed at 27:1 using the JPEG algorithm compares favorably with directly digitized video images, the current goldstandard. Some potential applications of these velocities in analyzing the proximal convergence zones, mitral inflow, and some areas of future development are also discussed in the article.

NASA Discipline Cardiopulmonary↗
NASA NTRS2005

Apparatus Characterizes Transient Voltages in Real Time

The figure shows a prototype of a relatively inexpensive electronic monitoring apparatus that measures and records selected parameters of lightning-induced transient voltages on communication and power cables. The selected parameters, listed below, are those most relevant to the ability of lightning-induced transients to damage electronic equipment. This apparatus bridges a gap between some traditional transient-voltage recorders that record complete waveforms and other traditional transient-voltage recorders that record only peak values: By recording the most relevant parameters and only those parameters this apparatus yields more useful information than does a traditional peak-value (only) recorder while imposing much smaller data-storage and data-transmission burdens than does a traditional complete-waveform recorder. Also, relative to a complete-waveform recorder, this apparatus is more reliable and can be built at lower cost because it contains fewer electronic components. The transients generated by sources other than lightning tend to have frequency components well below 1 MHz. Most commercial transient recorders can detect and record such transients, but cannot respond rapidly enough for recording lightning-induced transient voltage peaks, which can rise from 10 to 90 percent of maximum amplitude in a fraction of a microsecond. Moreover, commercial transient recorders cannot rearm themselves rapidly enough to respond to the multiple transients that occur within milliseconds of each other on some lightning strikes. One transient recorder, designed for Kennedy Space Center earlier [ Fast Transient-Voltage Recorder (KSC- 11991), NASA Tech Briefs, Vol. 23, No. 10, page 6a (October 1999)], is capable of sampling transient voltages at peak values up to 50 V in four channels at a rate of 20 MHz. That recorder contains a trigger circuit that continuously compares the amplitudes of the signals on four channels to a preset triggering threshold. When a trigger signal is received, a volatile memory is filled with data for a total time of 200 ms. After the data are transferred to nonvolatile memory, the recorder rearms itself within 400 ms to enable recording of subsequent transients. Unfortunately, the recorded data must be retrieved through a serial communication link. Depending on the amount of data recorded, the memory can be filled before retrieval is completed. Although large amounts of data are recorded and retrieved, only a small part of the information (the selected parameters) is usually required. The present transient-voltage recorder provides the required information, without incurring the overhead associated with the recording, storage, and retrieval of complete transient-waveform data. In operation, this apparatus processes transient voltage waveforms in real time to extract and record the selected parameters. An analog-to-digital converter that operates at a speed of as much as 100 mega-samples per second is used to sample a transient waveform. A real-time comparator and peak detector are implemented by use of fast field-programmable gate arrays.

Medelius, Pedro↗
NASA NTRS1993

Systems aspects of COBE science data compression

A general approach to compression of diverse data from large scientific projects has been developed and this paper addresses the appropriate system and scientific constraints together with the algorithm development and test strategy. This framework has been implemented for the COsmic Background Explorer spacecraft (COBE) by retrofitting the existing VAS-based data management system with high-performance compression software permitting random access to the data. Algorithms which incorporate scientific knowledge and consume relatively few system resources are preferred over ad hoc methods. COBE exceeded its planned storage by a large and growing factor and the retrieval of data significantly affects the processing, delaying the availability of data for scientific usage and software test. Embedded compression software is planned to make the project tractable by reducing the data storage volume to an acceptable level during normal processing.

Freedman, I.↗
NASA NTRS1980

Optical mass memory investigation

The MASTER 1 optical mass storage system advanced working model (AWM) was designed to demonstrate recording and playback of imagery data and to enable quantitative data to be derived as to the statistical distribution of raw errors experienced through the system. The AWM consists of two subsystems, the recorder and storage and retrieval. The recorder subsystem utilizes key technologies such as an acoustic travelling wave lens to achieve recording of digital data on fiche at a rate of 30 Mbits/sec, whereas the storage and retrieval reproducer subsystem utilizes a less complex optical system that employs an acousto-optical beam deflector to achieve data readout at a 5 Mbits/sec rate. The system has the built in capability for detecting and collecting error statistics. The recorder and storage and retrieval subsystems operate independent of one another and are each constructed in modular form with each module performing independent functions. The operation of each module and its interface to other modules is controlled by one controller for both subsystems.

Source record↗
NASA NTRS1976

Data processing/display design for the space shuttle/spacelab Electromagnetic Environment Experiment (EEE)

Methods for data analysis, data compression including universal coding, storage and retrieval on random access storage devices, and display were developed and implemented on the GSFC Interdata computer. The original 64 bit per frequency band representation was reduced to 10 bits through source coding/universal coding, a compression ratio of 6.4, prior to storage. Rapid encoding/decoding was achieved by the algorithms used so that rapid random access is retained.

Davisson, L. D.↗
NASA NTRS1976

Planning for a data base system to support satellite conceptual design

The conceptual design of an automated satellite design data base system is presented. The satellite catalog in the system includes data for all earth orbital satellites funded to the hardware stage for launch between 1970 and 1980, and provides a concise compilation of satellite capabilities and design parameters. The cost of satellite subsystems and components will be added to the base. Data elements are listed and discussed. Sensor and science and applications opportunities catalogs will be included in the data system. Capabilities of the BASIS storage, retrieval, and analysis system are used in the system design.

Claydon, C. R.↗
NASA NTRS1995

Particle fallout/activity sensor

A particle fallout/activity sensor measures relative amounts of dust or other particles which collect on a mirror in an area to be monitored. The sensor includes a sensor module and a data acquisition module, both of which can be operated independently of one another or in combination with one another. The sensor module includes a housing containing the mirror, an LED assembly for illuminating the mirror and an optical detector assembly for detecting light scattered off of the mirror by dust or other particles collected thereon. A microprocessor controls operation of the sensor module's components and displays results of a measurement on an LCD display mounted on the housing. A push button switch is also mounted on the housing which permits manual initiation of a measurement. The housing is constructed of light absorbing material, such as black delrin, which minimizes detection of light by the optical detector assembly other than that scattered by dust or particles on the mirror. The data acquisition module can be connected to the sensor module and includes its own microprocessor, a timekeeper and other digital circuitry for causing the sensor module to make a measurement periodically and send the measurement data to the data acquisition module for display and storage in memory for later retrieval and transfer to a separate computer. The time tagged measurement data can also be used to determine the relative level of activity in the monitored area since this level is directly related to the amount of dust or particle fallout in the area.

Curtis, Ihlefeld M.↗
NASA NTRS1996

Evolving Requirements for Magnetic Tape Data Storage Systems

Magnetic tape data storage systems have evolved in an environment where the major applications have been back-up/restore, disaster recovery, and long term archive. Coincident with the rapidly improving price-performance of disk storage systems, the prime requirements for tape storage systems have remained: (1) low cost per MB, (2) a data rate balanced to the remaining system components. Little emphasis was given to configuring the technology components to optimize retrieval of the stored data. Emerging new applications such as network attached high speed memory (HSM), and digital libraries, place additional emphasis and requirements on the retrieval of the stored data. It is therefore desirable to consider the system to be defined both by STorage And Retrieval System (STARS) requirements. It is possible to provide comparative performance analysis of different STARS by incorporating parameters related to (1) device characteristics, and (2) application characteristics in combination with queuing theory analysis. Results of these analyses are presented here in the form of response time as a function of system configuration for two different types of devices and for a variety of applications.

Gniewek, John J.↗
NASA NTRS1985

An Innovative, Multidisciplinary Educational Program in Interactive Information Storage and Retrieval

There exists a large number of large-scale bibliographic Information Storage and Retrieval Systems containing large amounts of valuable data of interest in a wide variety of research applications. These systems are not used to capacity because the end users, i.e., the researchers, have not been trained in the techniques of accessing such systems. This thesis describes the development of a transportable, university-level course in methods of querying on-line interactive Information Storage and Retrieval systems as a solution to this problem. This course was designed to instruct upper division science and engineering students to enable these end users to directly access such systems. The course is designed to be taught by instructors who are not specialists in either computer science or research skills. It is independent of any particular IS and R system or computer hardware. The project is sponsored by NASA and conducted by the University of Southwestern Louisiana and Southern University.

Dominick, Wayne D.↗
NASA NTRS2018

Evolving Storage and Cyber Infrastructure at the NASA Center for Climate Simulation

This talk will describe recent developments at the NASA Center for Climate Simulation, which is funded by NASAs Science Mission Directorate, and supports the specialized data storage and computational needs of weather, ocean, and climate researchers, as well as astrophysicists, heliophysicists, and planetary scientists. To meet requirements for higher-resolution, higher-fidelity simulations, the NCCS augments its High Performance Computing (HPC) and storage retrieval environment. As the petabytes of model and observational data grow, the NCCS is broadening data services offerings and deploying and expanding virtualization resources for high performance analytics.

Cyberinfrastructure↗