Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data archiving software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

A land-surface Testbed for EOSDIS

The main objective of the Testbed project was to deliver satellite images via the Internet to scientific and educational users free of charge. The main method of operations was to store satellite images on a low cost tape library system, visually browse the raw satellite data, access the raw data filed, navigate the imagery through 'C' programming and X-Windows interface software, and deliver the finished image to the end user over the Internet by means of file transfer protocol methods. The conclusion is that the distribution of satellite imagery by means of the Internet is feasible, and the archiving of large data sets can be accomplished with low cost storage systems allowing multiple users.

Emery, William↗
NASA NTRS2010

ICER-3D Hyperspectral Image Compression Software

Software has been developed to implement the ICER-3D algorithm. ICER-3D effects progressive, three-dimensional (3D), wavelet-based compression of hyperspectral images. If a compressed data stream is truncated, the progressive nature of the algorithm enables reconstruction of hyperspectral data at fidelity commensurate with the given data volume. The ICER-3D software is capable of providing either lossless or lossy compression, and incorporates an error-containment scheme to limit the effects of data loss during transmission. The compression algorithm, which was derived from the ICER image compression algorithm, includes wavelet-transform, context-modeling, and entropy coding subalgorithms. The 3D wavelet decomposition structure used by ICER-3D exploits correlations in all three dimensions of sets of hyperspectral image data, while facilitating elimination of spectral ringing artifacts, using a technique summarized in "Improving 3D Wavelet-Based Compression of Spectral Images" (NPO-41381), NASA Tech Briefs, Vol. 33, No. 3 (March 2009), page 7a. Correlation is further exploited by a context-modeling subalgorithm, which exploits spectral dependencies in the wavelet-transformed hyperspectral data, using an algorithm that is summarized in "Context Modeler for Wavelet Compression of Hyperspectral Images" (NPO-43239), which follows this article. An important feature of ICER-3D is a scheme for limiting the adverse effects of loss of data during transmission. In this scheme, as in the similar scheme used by ICER, the spatial-frequency domain is partitioned into rectangular error-containment regions. In ICER-3D, the partitions extend through all the wavelength bands. The data in each partition are compressed independently of those in the other partitions, so that loss or corruption of data from any partition does not affect the other partitions. Furthermore, because compression is progressive within each partition, when data are lost, any data from that partition received prior to the loss can be used to reconstruct that partition at lower fidelity. By virtue of the compression improvement it achieves relative to previous means of onboard data compression, this software enables (1) increased return of hyperspectral scientific data in the presence of limits on the rates of transmission of data from spacecraft to Earth via radio communication links and/or (2) reduction in spacecraft radio-communication power and/or cost through reduction in the amounts of data required to be downlinked and stored onboard prior to downlink. The software is also suitable for compressing hyperspectral images for ground storage or archival purposes.

Xie, Hua↗
NASA NTRS2018

System and Method for Providing a Climate Data Persistence Service

A system, method and computer-readable storage devices for providing a climate data persistence service. A system configured to provide the service can include a climate data server that performs data and metadata storage and management functions for climate data objects, a compute-storage platform that provides the resources needed to support a climate data server, provisioning software that allows climate data server instances to be deployed as virtual climate data servers in a cloud computing environment, and a service interface, wherein persistence service capabilities are invoked by software applications running on a client device. The climate data objects can be in various formats, such as International Organization for Standards (ISO) Open Archival Information System (OAIS) Reference Model Submission Information Packages, Archive Information Packages, and Dissemination Information Packages. The climate data server can enable scalable, federated storage, management, discovery, and access, and can be tailored for particular use cases.

Schnase, John L.↗
NASA NTRS1993

Using magnetic tape technology for data migration

Magnetic tape and optical disk library units (jukeboxes) are satisfying the demand for high-capacity cost-effective storage. The choice between optical disk and magnetic tape technology must take into account the cost limitations as well as the performance and reliability requirements of the user environment. Library units require data management software in order to function in an automated and user-transparent way. The most common data management applications are backup and recovery, data migration, and archiving. The medium access patterns that these applications create will be described. Since the most user visible application is data migration, a queue simulator was developed to model its performance against a variety of library units. The major subject of this paper is the design and implementation of this simulator as well as some simulation results. The relative cost and reliability of magnetic tape versus optical disk library units is presented for completeness.

Therrien, David↗
NASA NTRS2002

Global Precipitation Measurement: Benefits of Partnering with GPM Mission - Report 2

An important goal of the Global Precipitation Measurement (GPM) mission is to maximize participation by non-NASA partners both domestic and international. A consequence of this objective is the provision for NASA to provide sufficient incentives to achieve partner buy-in and commitment to the program. NASA has identified seven specific areas in which substantive incentives will be offered: (1) partners will be offered participation in governance of GPM mission science affairs including definition of data products; (2) partners will be offered use of NASA's TDRSS capability for uplink and downlink of commands and data in regards to partner provided spacecraft; (3) partners will be offered launch support for placing partner provided spacecraft in orbit conditional upon mutually agreeable co-manifest arrangements; (4) partners will be offered direct data access at the NASA-GPM server level rather than through standard data distribution channels; (5) partners will be offered the opportunity to serve as regional data archive and distribution centers for standard GPM data products; and (6) partners will be offered the option to insert their own specialized filtering and extraction software into the GPM data processing stream or to obtain specialized subsets and products over specific areas of interest (7) partners will be offered GPM developed software tools that can be run on their platforms. Each of these incentives, either individually or in combination, represents a significant advantage to partners who may wish to participate in the GPM mission.

Stocker, Erich F.↗
NASA NTRS2013

Thermal Performance Data Services (TPDS)

Initiated as a NASA Engineering and Safety Center (NESC) assessment in 2009, the Thermal Performance Database (TPDB) was a response to the need for a centralized thermal performance data archive. The assessment was renamed Thermal Performance Data Services (TPDS) in 2012; the undertaking has had two fronts of activity: the development of a repository software application and the collection of historical thermal performance data sets from dispersed sources within the thermal performance community. This assessment has delivered a foundational tool on which additional features should be built to increase efficiency, expand the protection of critical Agency investments, and provide new discipline-advancing work opportunities. This report contains the information from the assessment.

French, Richard T.↗
NASA NTRS2012

Newly Released TRMM Version 7 Products, Other Precipitation Datasets and Data Services at NASA GES DISC

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is home of global precipitation product archives, in particular, the Tropical Rainfall Measuring Mission (TRMM) products. TRMM is a joint U.S.-Japan satellite mission to monitor tropical and subtropical (40 S - 40 N) precipitation and to estimate its associated latent heating. The TRMM satellite provides the first detailed and comprehensive dataset on the four dimensional distribution of rainfall and latent heating over vastly undersampled tropical and subtropical oceans and continents. The TRMM satellite was launched on November 27, 1997. TRMM data products are archived at and distributed by GES DISC. The newly released TRMM Version 7 consists of several changes including new parameters, new products, meta data, data structures, etc. For example, hydrometeor profiles in 2A12 now have 28 layers (14 in V6). New parameters have been added to several popular Level-3 products, such as, 3B42, 3B43. Version 2.2 of the Global Precipitation Climatology Project (GPCP) dataset has been added to the TRMM Online Visualization and Analysis System (TOVAS; URL: http://disc2.nascom.nasa.gov/Giovanni/tovas/), allowing online analysis and visualization without downloading data and software. The GPCP dataset extends back to 1979. Version 3 of the Global Precipitation Climatology Centre (GPCC) monitoring product has been updated in TOVAS as well. The product provides global gauge-based monthly rainfall along with number of gauges per grid. The dataset begins in January 1986. To facilitate data and information access and support precipitation research and applications, we have developed a Precipitation Data and Information Services Center (PDISC; URL: http://disc.gsfc.nasa.gov/precipitation). In addition to TRMM, PDISC provides current and past observational precipitation data. Users can access precipitation data archives consisting of both remote sensing and in-situ observations. Users can use these data products to conduct a wide variety of activities, including case studies, model evaluation, uncertainty investigation, etc. To support Earth science applications, PDISC provides users near-real-time precipitation products over the Internet. At PDISC, users can access tools and software. Documentation, FAQ and assistance are also available. Other capabilities include: 1) Mirador (http://mirador.gsfc.nasa.gov/), a simplified interface for searching, browsing, and ordering Earth science data at NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). Mirador is designed to be fast and easy to learn; 2)TOVAS; 3) NetCDF data download for the GIS community; 4) Data via OPeNDAP (http://disc.sci.gsfc.nasa.gov/services/opendap/). The OPeNDAP provides remote access to individual variables within datasets in a form usable by many tools, such as IDV, McIDAS-V, Panoply, Ferret and GrADS; 5) The Open Geospatial Consortium (OGC) Web Map Service (WMS) (http://disc.sci.gsfc.nasa.gov/services/wxs_ogc.shtml). The WMS is an interface that allows the use of data and enables clients to build customized maps with data coming from a different network.

Liu, Zhong↗
NASA NTRS2000

EOSDIS Terra Data Sampler #1: Western US Wildfires 2000

This CD-ROM contains sample data in HDF-EOS format from the instruments on board the Earth Observing System (EOS) Terra satellite: (1) Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER); (2) Clouds and the Earth's Radiant Energy System (CERES); (3) Multi-angle Imaging Spectroradiometer (MISR); and (4) Moderate Resolution Imaging Spectroradiometer (MODIS). Data from the Measurements of Pollution in the Troposphere (MOPITT) instrument were not available for distribution (as of October 17, 2000). The remotely sensed, coincident data for the Western US wildfires were acquired August 30, 2000. This CD-ROM provides information about the Terra mission, instruments, data, and viewing tools. It also provides the Collage tool for viewing data, and links to Web sites containing other digital data processing software. Full granules of the data on this CD-ROM and other EOS Data and Information System (EOSDIS) data products are available from the NASA Distributed Active Archive Centers (DAACs).

Perkins, Dorothy C.↗
NASA NTRSDate not supplied

Crustal Dynamics Data Information System (CDDIS) Contributions to GGOS

The Crustal Dynamics Data Information System (CDDIS) provides essential support for the Global Geodetic Observing System (GGOS) by operating a data and product archive for the main geodetic techniques. As GGOS matures and grows, the CDDIS adopts the latest data practices to strengthen its support for the community and ensure quality products are available in a timely manner. To this end, the CDDIS is continually refining its software backend, performing developments such as the 2023 implementation of a middleware processing system, which is successfully being used in operation to provide GNSS-based Upper Atmospheric Realtime Disaster Information and Alert Network (GUARDIAN) and Global Differential GPS System (GDGPS) products. In addition, the CDDIS has 1) released new data and products, 2) performed updates to facilitate navigation of the archive, and 3) made it easier to cite the data and products being added. This poster explores the breadth of work done at the CDDIS and provides highlights of the latest developments.

Justine Woo↗
NASA NTRS2007

Using Selection Pressure as an Asset to Develop Reusable, Adaptable Software Systems

The Goddard Earth Sciences Data and Information Services Center (GES DISC) at NASA has over the years developed and honed several reusable architectural components for supporting large-scale data centers with a large customer base. These include a processing system (S4PM) and an archive system (S4PA) based upon a workflow engine called the Simple Scalable Script based Science Processor (S4P) and an online data visualization and analysis system (Giovanni). These subsystems are currently reused internally in a variety of combinations to implement customized data management on behalf of instrument science teams and other science investigators. Some of these subsystems (S4P and S4PM) have also been reused by other data centers for operational science processing. Our experience has been that development and utilization of robust interoperable and reusable software systems can actually flourish in environments defined by heterogeneous commodity hardware systems the emphasis on value-added customer service and the continual goal for achieving higher cost efficiencies. The repeated internal reuse that is fostered by such an environment encourages and even forces changes to the software that make it more reusable and adaptable. Allowing and even encouraging such selective pressures to software development has been a key factor In the success of S4P and S4PM which are now available to the open source community under the NASA Open source Agreement

Berrick, Stephen↗
NASA NTRS2019

NASA Archives and Data Stewardship in the Cloud with Cumulus

NASA's Earth Observing System Data and Information System (EOSDIS) houses nearly 30PB (petabytes) of critical Earth Science data and with upcoming missions is expected to balloon to between 200PBs-300PBs over the next seven years. The magnitude of data collected makes it infeasible to download data and process it locally, forcing us to re-think how we store and work with earth science data. NASA has looked to the cloud to address this, building its open source Cumulus software to manage the ingest of diverse data in a wide variety of formats into the cloud and provide services to manage and access the data. In this talk, we will describe how Cumulus provides common features needed to manage a cloud archive in the realm of ingest, data stewardship, and cost-controlled distribution of science data to users and services.

NASA↗
NASA NTRS2018

Cumulus: NASA Archives in the Cloud

NASA's Earth Observing System Data and Information System (EOSDIS) houses nearly 30PBs of critical Earth Science data and with upcoming missions is expected to balloon to between 200PBs-300PBs over the next seven years. The magnitude of data collected makes it infeasible to download data and process it locally, forcing us to re-think how we store and work with earth science data. NASA has looked to the cloud to address this, building its open source Cumulus software to manage the ingest of diverse data in a wide variety of formats into the cloud and provide services to manage and access the data. In this talk, we will describe how Cumulus provides common features needed to manage a cloud archive in the realm of ingest, data stewardship, and cost-controlled distribution of science data to users and services.

EOSDIS↗
NASA NTRS2019

Using Big Data Technologies with Earth Science Data in HDF5: HDF5 Scalable Solutions

HDF5 (Hierarchical Data Format 5) is open-source, high-performance software that consists of an abstract data model, library, and fileformat used for storing and managing extremely large and/or complex data collections. NASA Earth Observing System (EOS) Data and Information Systems use HDF5 as an archival format to store remote sensing data from EOS satellites. HDF5 is also used to store other types of Geoscience and Strophysical data, e.g., seismic data and data from Low-Frequency Array (LOFAR) radio telescopes. Data stored in HDF5 has reached tens of petabytes and is growing at an accelerated rate.With the growing amout of HDF5 Earth Science data to analyze and process, scientists need to adopt big data technologies including new storage paradigms such as cloud and object storage. To run models and perform data analysis they also need to utilizied efficient and diverse ways to access data, from high-performance computing's (HPC) Message Passing Interface (MPI) I/O and deep memory hierarchies (DMH) to non-HPC frameworks such as Apache Hadoop, Spark, and Drill. The HDF Group continually works to enable usage of big data technologies in HDF software.

Knox, Larry↗
NASA NTRS1991

Calibration of NASA/JPL DC-8 SAR data

The paper describes a number of improvements in calibration of NASA/JPL DC-8 SAR (synthetic aperture radar) data, intended to make calibrated data from the system available to the majority of the investigators. The improvements include a move to a better site for calibration of the system at the start of each campaign season, the release of software to allow users to calibrate their data, the supply of trihedral corner reflectors and instructions on how to deploy them, and the setting up of an archive of calibrated data. Analysis of the calibration performance of the DC-8 SAR over a number of sites during 1990 suggests that the system is sufficiently stable to allow calibration of the standard data products without the presence of calibration devices within the scene. The limiting factor on DC-8 SAR data calibration would now appear to be the knowledge of the altitude of the aircraft, which determines the relationship between the measured slant range coordinates and the antenna pattern in elevation.

Freeman, A.↗
NASA NTRS1992

A land-surface testbed for EOSDIS

We propose to develop an on-line data distribution and interactive display system for the collection, archival, distribution and analysis of operational weather satellite data for applications in land surface studies. A 1,000 sq km scene of the western U.S. (centered on the Colorado Rockies) will be extracted from Advanced Very High Resolution Radiometer (AVHRR) imagery collected from morning and afternoon passes of the NOAA polar-orbiters at the direct readout stations operated by CU/CCAR. All five channels of these AVHRR data will be navigated and map registered at CU/CCAR and then be transferred to NCAR for storage in an on-line data system. Software will also be available at NCAR to process and navigate the raw AVHRR data as needed. A display workstation software, based on a Macintosh II computer, will be developed that will display and further process the AVHRR data for studies of vegetation monitoring and snowpack assessment. Various options of presently used techniques for both vegetation and snowpack monitoring will be implemented in the workstation software to provide the individual investigator with the freedom to interact with the satellite image data. The display software will be freely distributed online to interested investigators and the AVHRR data will be made available on-line to anyone interested. In addition, potential users will be sought out and connected to the on-line data archive. This experiment with an active on-line archive and interactive analysis systems will provide experience with a small scale EOSDIS.

Emery, William↗
NASA NTRS1993

A land-surface testbed for EOSDIS

We propose to develop an on-line data distribution and interactive display system for the collection, archival, distribution and analysis of operational weather satellite data for applications in land surface studies. A 1,000 square km scene of the western U.S. (centered on the Colorado Rockies) will be extracted from Advanced Very High Resolution Radiometer (AVHRR) imagery collected from morning and afternoon passes of the NOAA polar-orbiters at the direct readout stations operated by CU/CCAR. All five channels of these AVHRR data will be navigated and map registered at CU/CCAR and then be transferred to NCAR for storage in an on-line data system. Software will also be available at NCAR to process and navigate the raw AVHRR data as needed. A display workstation software, based on a Macintosh II computer, will be developed that will display and further process the AVHRR data for studies of vegetation monitoring and snowpack assessment. Various options of presently used techniques for both vegetation and snowpack monitoring will be implemented in the workstation software to provide the individual investigator with the freedom to interact with the satellite image data. The display software will be freely distrributed online to interested investigators, and the AVHRR data will be made available on-line to anyone interested. In addition, potential users will be sought out and connected to the on-line data archive. This experiment with an active on-line archive and interactive analysis systems will provide experience with a small scale EOSDIS.

Emery, William↗
NASA NTRS1994

Develop advanced nonlinear signal analysis topographical mapping system

The Space Shuttle Main Engine (SSME) has been undergoing extensive flight certification and developmental testing, which involves some 250 health monitoring measurements. Under the severe temperature, pressure, and dynamic environments sustained during operation, numerous major component failures have occurred, resulting in extensive engine hardware damage and scheduling losses. To enhance SSME safety and reliability, detailed analysis and evaluation of the measurements signal are mandatory to assess its dynamic characteristics and operational condition. Efficient and reliable signal detection techniques will reduce catastrophic system failure risks and expedite the evaluation of both flight and ground test data, and thereby reduce launch turn-around time. The basic objective of this contract are threefold: (1) develop and validate a hierarchy of innovative signal analysis techniques for nonlinear and nonstationary time-frequency analysis. Performance evaluation will be carried out through detailed analysis of extensive SSME static firing and flight data. These techniques will be incorporated into a fully automated system; (2) develop an advanced nonlinear signal analysis topographical mapping system (ATMS) to generate a Compressed SSME TOPO Data Base (CSTDB). This ATMS system will convert tremendous amount of complex vibration signals from the entire SSME test history into a bank of succinct image-like patterns while retaining all respective phase information. High compression ratio can be achieved to allow minimal storage requirement, while providing fast signature retrieval, pattern comparison, and identification capabilities; and (3) integrate the nonlinear correlation techniques into the CSTDB data base with compatible TOPO input data format. Such integrated ATMS system will provide the large test archives necessary for quick signature comparison. This study will provide timely assessment of SSME component operational status, identify probable causes of malfunction, and indicate feasible engineering solutions. The final result of this program will yield an ATMS system of nonlinear and nonstationary spectral analysis software package integrated with the Compressed SSME TOPO Data Base (CSTDB) on the same platform. This system will allow NASA engineers to retrieve any unique defect signatures and trends associated with different failure modes and anomalous phenomena over the entire SSME test history across turbo pump families.

Source record↗
NASA NTRS2012

The NASA Heliophysics Active Final Archive at the Space Physics Data Facility

The 2009 NASA Heliophysics Science Data Management Policy re-defined and extended the responsibilities of the Space Physics Data Facility (SPDF) project. Building on SPDF's established capabilities, the new policy assigned the role of active "Final Archive" for non-solar NASA Heliophysics data to SPDF. The policy also recognized and formalized the responsibilities of SPDF as a source for critical infrastructure services such as VSPO to the overall Heliophysics Data Environment (HpDE) and as a Center of Excellence for existing SPDF science-enabling services and software including CDAWeb, SSCWeb/4D Orbit Viewer, OMNIweb and CDF. We will focus this talk to the principles, strategies and planned SPDF architecture to effectively and efficiently perform these roles, with special emphasis on how SPDF will ensure the long-term preservation and ongoing online community access to all the data entrusted to SPDF. We will layout our archival philosophy and what we are advocating in our work with NASA missions both current and future, with potential providers of NASA and NASA-relevant archival data, and to make the data and metadata held by SPDF accessible to other systems and services within the overall HpOE. We will also briefly review our current services, their metrics and our current plans and priorities for their evolution.

McGuire, Robert E.↗