Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data archiving software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Open-Source Science-led Development of the AOS Mission Science Data System (SDS)

The Earth System Observatory (ESO) Atmosphere Observing System (AOS) mission will provide space-based and suborbital observations of collocated cloud, dynamic, precipitation and aerosol processing leading to improved weather, air quality, and climate predictions. The AOS Science Data System (SDS) will be a system of systems developed within the Cloud to manage the research and operational processing of AOS mission orbital and suborbital sensors and curate these data for reprocessing (e.g., in near real-time or by collection) and transfer them to a NASA Distributed Active Archive Center (DAAC) for long-term storage. Further, AOS SDS will follow guidelines provided by NASA Earth Science Data Systems (ESDS) program including standard conventions for data file formats, naming, and metadata to improve data interoperability, interpretability, usability, discovery, provenance, and spatiotemporal representativeness. The AOS mission follows NASA’s lead in making a commitment to Open-Source Science (OSS) including the sharing of data, software, and knowledge in an open and timely manner. Each of the AOS SDS system components will be developed with open-source concepts including components of SDS itself as well as AOS mission algorithms. Further, the AOS SDS assumes the role to lead and facilitate OSS activities for the AOS mission. This presentation describes the framework of the AOS SDS and its integral part in facilitating OSS within the AOS mission.

David M. Giles↗
NASA NTRS1992

An open, parallel I/O computer as the platform for high-performance, high-capacity mass storage systems

APTEC Computer Systems is a Portland, Oregon based manufacturer of I/O computers. APTEC's work in the context of high density storage media is on programs requiring real-time data capture with low latency processing and storage requirements. An example of APTEC's work in this area is the Loral/Space Telescope-Data Archival and Distribution System. This is an existing Loral AeroSys designed system, which utilizes an APTEC I/O computer. The key attributes of a system architecture that is suitable for this environment are as follows: (1) data acquisition alternatives; (2) a wide range of supported mass storage devices; (3) data processing options; (4) data availability through standard network connections; and (5) an overall system architecture (hardware and software designed for high bandwidth and low latency). APTEC's approach is outlined in this document.

Abineri, Adrian↗
NASA NTRSDate not supplied

Lunar Instrument Data Integration Into the Virtual Reality Mission Simulation System for Decision Communication and Situational Awareness

In situ resource utilization (ISRU) technologies are a key advancement required to make human habitation on the Moon and Mars viable. The upcoming Volatiles Investigating Polar Exploration Rover (VIPER) mission will provide crucial correlations between volatiles and lunar geology to understand the water content available for ISRU on the moon. The mission will require the coordination of multi-disciplinary teams across the country making real time decisions based on rover instrument data. The virtual reality Mission Simulation System (vMSS) is a virtual reality platform designed at MIT by the Resource Exploration and Science of our Cosmic Environment (RESOURCE) team to provide teams with a collaboration interface for planetary missions like VIPER. Herein we determine the integration pathway for analog based datasets that are examples of VIPER's two main instruments, the near-infrared volatile spectrometer subsystem (NIRVSS) and the neutron spectrometer subsystem (NSS), into vMSS to provide the most valuable visualization tools. Focusing on improving situational awareness, decision making, reducing task load and incorporating comments from scientists working previous analogs and on the current VIPER mission, we recommend critical elements to implement into vMSS and the best approaches for data visualization. We present a review of relevant analogs and state of the art mission software. We have developed a design concept and path to flight of analysed instrument data integrated with data maps that allow for virtual manipulation and annotation between non-co-located team members. We focus on pre-mission mapping of a priori data for improved situational awareness, layering of analysed instrument data, correlative mapping and interactive capabilities for in-mission decision making, as well as archiving and annotation tools for post-mission analysis. Finally, we lay out the roadmap for the future development of immersive sample site visualization capabilities and the use of integrated instrument data in vMSS with automated temporal and geospatial planning.

Cody Alison Paige↗
NASA NTRS2017

GES DISC Data Recipes in Jupyter Notebooks

The Earth Science Data and Information System (ESDIS) Project manages twelve Distributed Active Archive Centers (DAACs) which are geographically dispersed across the United States. The DAACs are responsible for ingesting, processing, archiving, and distributing Earth science data produced from various sources (satellites, aircraft, field measurements, etc.). In response to projections of an exponential increase in data production, there has been a recent effort to prototype various DAAC activities in the cloud computing environment. This, in turn, led to the creation of an initiative, called the Cloud Analysis Toolkit to Enable Earth Science (CATEES), to develop a Python software package in order to transition Earth science data processing to the cloud. This project, in particular, supports CATEES and has two primary goals. One, to transition data recipes created by the Goddard Earth Science Data and Information Service Center (GES DISC) into an interactive and educational environment using JupyterNotebooks. Two, to acclimate Earth scientists to cloud computing. To accomplish these goals, we create JupyterNotebooks to compartmentalize the different steps of data analysis and help users obtain and parse data from the command line. We also develop a Docker container, comprised of Jupyter Notebooks, Python dependencies, and command line tools, and configure it into an easy-to-deploy package. The end result is an end-to-end product that simulates the use case of end users working in the cloud computing environment.

discoverability↗
NASA NTRS1979

Mail LOG: Program operating instructions

The operating instructions for the software package, MAIL LOG, developed for the Scout Project Automatic Data System, SPADS, are provided. The program is written in FORTRAN for the PRIME 300 computer system. The MAIL LOG program has the following four modes of operation: (1) INPUT - putting new records into the data base (2) REVISE - changing or modifying existing records in the data base (3) SEARCH - finding special records existing in the data base (4) ARCHIVE - store or put away existing records in the data base. The output includes special printouts of records in the data base and results from the INPUT and SEARCH modes. The MAIL LOG data base consists of three main subfiles: Incoming and outgoing mail correspondence; Design Information Releases and Releases and Reports; and Drawings and Engineering orders.

Harris, D. K.↗
NASA NTRS1988

IUE's legacy for the future: The final archive and goals for its implementation

Requirements for the IUE archive, and how the signal/noise (S/N) ratio in photometrically corrected images can be enhanced considerably by cross-correlating the fixed pattern in a data image with that in a suitable flat-field image are described. From these cross-correlations it is feasible to derive an accurate geometrical correction to apply to the data image before applying the intensity transfer functions. The standard IUE processing software does not generate a sufficiently accurate geometric correction so that typical spatial errors of 1 to 2 pixels conspire with the large fixed pattern in raw images to produce significant misregistration noise. Tests on flat-field images demonstrate that an explicit geometric correction procedure can avoid most of the misregistration noise and can thereby improve the S/N ratio of IUE data by factors of 1.5 to 2.4.

Linsky, J. L.↗
NASA NTRS2007

A Comprehensive Plan for the Long-Term Calibration and Validation of Oceanic Biogeochemical Satellite Data

The primary objective of this planning document is to establish a long-term capability and validating oceanic biogeochemical satellite data. It is a pragmatic solution to a practical problem based primarily o the lessons learned from prior satellite missions. All of the plan's elements are seen to be interdependent, so a horizontal organizational scheme is anticipated wherein the overall leadership comes from the NASA Ocean Biology and Biogeochemistry (OBB) Program Manager and the entire enterprise is split into two components of equal sature: calibration and validation plus satellite data processing. The detailed elements of the activity are based on the basic tasks of the two main components plus the current objectives of the Carbon Cycle and Ecosystems Roadmap. The former is distinguished by an internal core set of responsibilities and the latter is facilitated through an external connecting-core ring of competed or contracted activities. The core elements for the calibration and validation component include a) publish protocols and performance metrics; b) verify uncertainty budgets; c) manage the development and evaluation of instrumentation; and d) coordinate international partnerships. The core elements for the satellite data processing component are e) process and reprocess multisensor data; f) acquire, distribute, and archive data products; and g) implement new data products. Both components have shared responsibilities for initializing and temporally monitoring satellite calibration. Connecting-core elements include (but are not restricted to) atmospheric correction and characterization, standards and traceability, instrument and analysis round robins, field campaigns and vicarious calibration sites, in situ database, bio-optical algorithm (and product) validation, satellite characterization and vicarious calibration, and image processing software. The plan also includes an accountability process, creating a Calibration and Validation Team (to help manage the activity), and a discussion of issues associated with the plan's scientific focus.

Hooker, Stanford B.↗
DOE OSTI2026

Terrestrial laser scanning data (Levels 0 and 1) for Pasoh, Malaysia, Sep 2024

This data package contains data from terrestrial laser scanning (TLS) at the Pasoh Forest Reserve, Malaysia. The Pasoh Forest Reserve is a facility of the Forest Research Institute Malaysia, and contains evergreen lowland dipterocarp forest. The Next-Generation Ecosystem Experiments Tropics (NGEE-Tropics) study areas at Pasoh were established to study how different species respond to climatic variation and soil water availability. Two study areas were chosen representing different topography and species. The TLS data archived here were collected to provide detailed, three-dimensional information about forest structure. Specifically, data were collected to allow tree-level characterization of woody structure and leaf area for 12 focal trees with FloraPulse and sap flux sensors, facilitating estimation of woody biomass and leaf area to allow upscaling of water content and transpiration data to the tree-level. Scan positions were not selected to provide consistent data for non-focal trees with the study areas. This data package contains the following data: - High-level files document further details of the campaign and data package: 1_CampaignSummary.csv provides details about the campaign and study site, 2_ScanAreasDetail.csv provides details about each separate scan area (groups of scans post-processed into a single point cloud), 3_TerrestrialLidarSensor.csv provides further technical details about the Riegl VZ-400i TLS sensor, TLS_CSV_dd.csv is a CSV Data Dictionary providing information about the fields in CSV files following the ESS-DIVE CSV File Formatting Guidelines Reporting Format, TLS_flmd.csv is a File Level Metadata file providing information about each file in the data package following the ESS-DIVE File Level Metadata Reporting Format, and README.txt is a text file describing the overall project and file structure. - Level 0 data are the raw data (.PROJ folders) as recorded by the Riegl VZ-400i TLS instrument before scan co-registration and post-processing with the Riegl's proprietary RiSCAN PRO software, which requires a license. - Level 1 data contain post-processed, co-registered data from each scan area. The "PointClouds" folder for each scan area contains a .las file with 1 cm resolution point cloud data exported from RiSCAN PRO. These are the main files likely to be of interest to most users and can be further processed with any software capable of manipulating .las files (e.g. Python, R CloudCompare). The "Project Information" folder contains log files from post-processing in RiSCAN PRO that may be of interest to users who want to see detailed records of post-processing, including all PDF reports generated by RiSCAN PRO. The "ScanPositions" folder contains information about the final position of all TLS scans, after post-processing, in multiple formats. The file ScanPositions_*.csv provides final geo-referenced scan positions, and the file SOP_backup_*.csv can be used in RiSCAN PRO to restore the co-registered scan positions if users wish to re-process raw data (Level 0 .PROJ folders) with RiSCAN PRO software (e.g., subsample to a different resolution, exclude a certain scan position, or apply different filters on reflectance or deviation values) without redoing time-consuming co-registration steps.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2016

Explore Earth Science Datasets for STEM with the NASA GES DISC Online Visualization and Analysis Tool, Giovanni

The NASA Goddard Earth Sciences (GES) Data and Information Services Center(DISC) is one of twelve NASA Science Mission Directorate (SMD) Data Centers that provide Earth science data, information, and services to users around the world including research and application scientists, students, citizen scientists, etc. The GESDISC is the home (archive) of remote sensing datasets for NASA Precipitation and Hydrology, Atmospheric Composition and Dynamics, etc. To facilitate Earth science data access, the GES DISC has been developing user-friendly data services for users at different levels in different countries. Among them, the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni, http:giovanni.gsfc.nasa.gov) allows users to explore satellite-based datasets using sophisticated analyses and visualization without downloading data and software, which is particularly suitable for novices (such as students) to use NASA datasets in STEM (science, technology, engineering and mathematics) activities. In this presentation, we will briefly introduce Giovanni along with examples for STEM activities.

precipitation↗
NASA NTRS1997

Operation of the Planetary Plasma Interactions Node of the Planetary Data System

Five years ago NASA selected the Planetary Plasma Interactions (PPI) Node at UCLA to help the scientific community locate, access and preserve particles and fields data from planetary missions. We propose to continue to serve for 5 more years. During the first five years we have served the scientific community by providing them with high quality data products. We worked with missions and individual scientists to secure the highest quality data possible and to thoroughly document it. We validated the data, placed it on long lasting media and made sure it was properly archived for future use. So far we have prepared and archived over 10(exp 11) bytes of data from 26 instruments on 4 spacecraft. We have produced 106 CD-ROMs with peer reviewed data. In so doing, we have developed an efficient system to prepare and archive the data and thereby have been able to steadily increase the rate at which the data are produced. Although we produced a substantial archive during the initial five years, we have an even larger amount of work in progress. This includes preparing CD-ROM data sets with all of the Voyager, Pioneer and Ulysses data at Jupiter and Saturn. We will have the Jupiter data ready for the Galileo encounter in December, 1995. We are also completing the Pioneer Venus data restoration. The Galileo Venus archive and radio science data from Magellan will be prepared early in the next period. We are assisting the Small Bodies Node of PDS in the preparation of comet data and will be archiving the asteroid data from Galileo. We will be moving in several new directions as well. We will archive the PPI Node's first Earth based data with data from the International Jupiter Watch and Hubble data taken in support of Ulysses particles and field observations. We will work with the Cassini mission in archive planning efforts. For the inner planets we will begin an archive of Mars data starting with Phobos data and will support the US and Russian Mars missions in the late 1990's. We will restore the Mercury data from Mariner 10 and prepare the lunar data from Clementine in time for the lunar data analysis program in 1995. We will work with the Discovery mission teams to plan their archive and have already started with one, NEAR. Finally we will begin archiving our first heliospheric data from Voyager, Galileo, and Mars observers. We will continue to serve the science community by providing access to the data products. During the past 19 months we have filled nearly 6000 requests for on-line and CD-ROM data. The data delivered directly by the PPI Node has been - 5 x 10(exp 11) bytes. In addition to providing the data, we have provided users with software tools to manage and read the data which are computer, operating system and format independent. We have developed scalable systems so that the same software we use to manage and access the data for the entire PPI Node can be used by individual investigators to manage the data on a single CD-ROM, thereby greatly reducing the software development effort for both the PPI Node and users. We deliver this software with the disks. Recent technical advances have made it possible for us to serve a broader community than before. In the next five year period we plan to extend our outreach to the general public and in particular to increase our support for education. Since planetary plasma data are varied and require expertise in many areas the PPI Node will continue to be distributed. In addition to the primary node at UCLA, the PPI Node has three subnodes with an Outer Planets Subnode at the University of Iowa, an Inner Planets Subnode at UCLA, and a Radio Science Subnode at Stanford University. During the first two years of the renewal period there will be a Radio Astronomy Data Node at GSFC. These organizations will provide scientific expertise on the data, participate in node data selection activities and help with data restoration and mission activities.

Walker, Raymond J.↗
NASA NTRS2004

A Complex Two Dimensional Spectroscopic Database: The HST Treasury Archive for Eta Carinae

The HST Treasury Program on eta Carinae produced one of the most complex and ambitious spectroscopic datasets obtained with the Hubble Space Telescope. The archival database includes the STIS/CCD and MAMA data during its recent spectroscopic event in 2003, plus all of the STIS observations of the object and its ejecta obtained during its spectroscopic cycle beginning in 1998. The database permits users to query via instrument, target, date, wavelength range, position angle, and position and allows the user to display the spectrum and download it in fits format. We have also developed software for ease of extraction and display and analysis. The user can go from an initial query to doing science in only 20 minutes! A demonstration of the database and software will be available at the poster.

Humphreys, R. M.↗
NASA NTRS1994

The design of a petabyte archive and distribution system for the NASA ECS project

The NASA EOS Data and Information System (EOSDIS) Core System (ECS) will contain one of the largest data management systems ever built - the ECS Science and Data Processing System (SDPS). SDPS is designed to support long term Global Change Research by acquiring, producing, and storing earth science data, and by providing efficient means for accessing and manipulating that data. The first two releases of SDPS, Release A and Release B, will be operational in 1997 and 1998, respectively. Release B will be deployed at eight Distributed Active Archiving Centers (DAAC's). Individual DAAC's will archive different collections of earth science data, and will vary in archive capacity. The storage and management of these data collections is the responsibility of the SDPS Data Server subsystem. It is anticipated that by the year 2001, the Data Server subsystem at the Goddard DAAC must support a near-line data storage capacity of one petabyte. The development of SDPS is a system integration effort in which COTS products will be used in favor of custom components in very possible way. Some software and hardware capabilities required to meet ECS data volume and storage management requirements beyond 1999 are not yet supported by available COTS products. The ECS project will not undertake major custom development efforts to provide these capabilities. Instead, SDPS and its Data Server subsystem are designed to support initial implementations with current products, and provide an evolutionary framework that facilitates the introduction of advanced COTS products as they become available. This paper provides a high-level description of the Data Server subsystem design from a COTS integration standpoint, and discussed some of the major issues driving the design. The paper focuses on features of the design that will make the system scalable and adaptable to changing technologies.

Caulk, Parris M.↗
NASA NTRS2004

Detectors for the James Webb Space Telescope Near-Infrared Spectrograph

The Near-Infrared Spectrograph (NIRSpec) is the James Webb Space Telescope's primary near-infrared spectrograph. NASA is providing the NIRSpec detector subsystem, which consists of the focal plane array, focal plane electronics, cable harnesses, and software. The focal plane array comprises two closely-butted lambda (sub co) approximately 5 micrometer Rockwell HAWAII- 2RG sensor chip assemblies. After briefly describing the NIRSpec instrument, we summarize some of the driving requirements for the detector subsystem, discuss the baseline architecture (and alternatives), and presents some recent detector test results including a description of a newly identified noise component that we have found in some archival JWST test data. We dub this new noise component, which appears to be similar to classical two-state popcorn noise in many aspects, "popcorn mesa noise." We close with the current status of the detector subsystem development effort.

Rauscher, Bernard J.↗
NASA NTRS2022

An Advanced Open-Source Platform for Air Quality Analysis, Visualization, and Prediction

Ambient air pollution is the largest environmental health risk factor, leading to several million premature deaths globally per year. The challenge of combating poor air quality is exacerbated by growing urban populations, changing emissions, and a warming climate. While there have been many advances monitoring and modeling of atmospheric composition, reflected in the dramatic increase in archived Earth Observations, there is no single measurement or method that alone can provide an accurate depiction of the entire atmosphere. The rapidly growing collections of observational and modeling data require us to be smarter about what data to include, and how such data is used. In recent years, NASA has invested significantly in advancing the concepts for Analytics Collaborative Framework (ACF) [5] and New Observing Strategies (NOS) [4] to tackle our software infrastructure need for harmonized data management and dynamic acquisition of diverse measurements for on-demand, interactive, multivariate analysis, and access [3]. It is not enough to have a big data, standalone analytics solution; it is critical that we start integrating data from remote sensing, modeling, and in-situ networks in a harmonized manner that enables timely and data-driven decision-making for air quality management. This work presents the design and development of an Air Quality Analytics Collaborative Framework (AQ ACF), as part of NASA’s Advanced Information Systems Technology (AIST) effort, to establish a data, machine-learning, and numerically driven platform for air quality analysis, visualization, and prediction.

Liu, Qian↗
NASA NTRS2009

Service Management Database for DSN Equipment

This data- and event-driven persistent storage system leverages the use of commercial software provided by Oracle for portability, ease of maintenance, scalability, and ease of integration with embedded, client-server, and multi-tiered applications. In this role, the Service Management Database (SMDB) is a key component of the overall end-to-end process involved in the scheduling, preparation, and configuration of the Deep Space Network (DSN) equipment needed to perform the various telecommunication services the DSN provides to its customers worldwide. SMDB makes efficient use of triggers, stored procedures, queuing functions, e-mail capabilities, data management, and Java integration features provided by the Oracle relational database management system. SMDB uses a third normal form schema design that allows for simple data maintenance procedures and thin layers of integration with client applications. The software provides an integrated event logging system with ability to publish events to a JMS messaging system for synchronous and asynchronous delivery to subscribed applications. It provides a structured classification of events and application-level messages stored in database tables that are accessible by monitoring applications for real-time monitoring or for troubleshooting and analysis over historical archives.

Zendejas, Silvino↗
NASA NTRS2014

Earth Science Informatics Comes of Age

The volume and complexity of Earth science data have steadily increased, placing ever-greater demands on researchers, software developers and data managers tasked with handling such data. Additional demands arise from requirements being levied by funding agencies and governments to better manage, preserve and provide open access to data. Fortunately, over the past 10-15 years significant advances in information technology, such as increased processing power, advanced programming languages, more sophisticated and practical standards, and near-ubiquitous internet access have made the jobs of those acquiring, processing, distributing and archiving data easier. These advances have also led to an increasing number of individuals entering the field of informatics as it applies to Geoscience and Remote Sensing. Informatics is the science and technology of applying computers and computational methods to the systematic analysis, management, interchange, and representation of data, information, and knowledge. Informatics also encompasses the use of computers and computational methods to support decisionmaking and other applications for societal benefits.

Jodha, Siri↗
NASA NTRSDate not supplied

Enabling Space Biological Knowledge Discovery Through Image and Video Data Sharing

Increased biomedical risks associated with deep space crewed missions (cis-Lunar, Mars transit/surface) require development of health countermeasures, novel ecosystem support, risk modeling, and fundamental space biological knowledge discovery. Molecular-omics, physiological-phenotypic-behavioral, and environmental-radiation telemetry data from space biological and health studies are needed for reuse by scientists to address these tasks. The data as well as space-relevant biospecimens are being made more findable, accessible, interoperable, and reusable through NASA’s Open Science Data Repository (OSDR). This new OSDR umbrella grouping includes NASA GeneLab, the NASA Ames Life Sciences Data Archive (ALSDA), and the NASA Biological Institutional Scientific Collection. The OSDR system design appropriately handles metadata and processed-tabular results from ALSDA studies collected from space experiments. But raw and processed ALSDA bioimage and video datasets require an expansion of OSDR’s data architecture to handle ingestion, curation, and egress. The academic-industry bioimaging field saw a scientific renaissance in the past several years through leveraging open-source software, international collaborations, machine learning, and other open science/programming approaches. As crewed missions and more biological experiments are on the deep space horizon, OSDR is embracing data stewardship through listening to feedback from subject matter experts and designing an expanded architecture which is appropriate for NASA’s goals to enable analysis and reuse of bioimaging and video data for the public science community.Discovery Through Image and Video Data Sharing

space biology↗
NASA NTRS2008

MISR Instrument Data Visualization

The MISR Interactive eXplorer (MINX) software functions both as a general-purpose tool to visualize Multiangle Imaging SpectroRadiometer (MISR) instrument data, and as a specialized tool to analyze properties of smoke, dust, and volcanic plumes. It includes high-level options to create map views of MISR orbit locations; scrollable, single-camera RGB (red-greenblue) images of MISR level 1B2 (L1B2) radiance data; and animations of the nine MISR camera images that provide a 3D perspective of the scenes that MISR has acquired. NASA Tech Briefs, September 2008 55 The plume height capability provides an accurate estimate of the injection height of plumes that is needed by air quality and climate modelers. MISR provides global high-quality stereo height information, and this program uses that information to perform detailed height retrievals of aerosol plumes. Users can interactively digitize smoke, dust, or volcanic plumes and automatically retrieve heights and winds, and can also archive MISR albedos and aerosol properties, as well as fire power and brightness temperatures associated with smoke plumes derived from Moderate Resolution Imaging Spectroradiometer (MODIS) data. Some of the specialized options in MINX enable the user to do other tasks. Users can display plots of top-of-atmosphere bidirectional reflectance factors (BRFs) versus camera-angle for selected pixels. Images and animations can be saved to disk in various formats. Also, users can apply a geometric registration correction to warp camera images when the standard processing correction is inadequate. It is possible to difference the images of two MISR orbits that share a path (identical ground track), as well as to construct pseudo-color images by assigning different combinations of MISR channels (angle or spectral band) to the RGB display channels. This software is an interactive application written in IDL and compiled into an IDL Virtual Machine (VM) ".sav" file.

Nelson, David↗