Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data archiving software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Development and Operations of the Astrophysics Data System

The ADS was represented at the AAS meeting with 3 poster papers and a demonstration booth. We have setup a mirror site of the Vizier data base system at the CDS. functionality of the ADC at Goddard. This will replace the Preparations for the APS and LPSC meetings in March started. We will have demonstrations at both meetings. Preparations for the APS and LPSC meetings in March continued. We will have demonstrations at both meetings. The ADS was represented with a poster at the joint AGUEGU meeting in Nice, France. Discussions about the on-going collaboration between the ADS and the CDS in Strasbourg, France were held in Strasbourg. The ADS was invited to organize a session about the ADS and its mirror sites at the next United Nations Workshop on Basic Space Sciences in the Developing World. Efforts are under way to enter the tables of contents of all conference proceedings in the SA0 library into the ADS. This requires copying the tables of contents from all volumes in the library and have them typed in. This will greatly enhance the coverage of the literature in the ADS. We started the development of a search system for the full text of all scanned material in the ADS. This will eventually allow our users search capabilities that so far do not exist in any form. I order to enable the full text searching, we have purchased OCR software and are in the process of OCRing the scanned pages in the ADS. Efforts are in progress to handle the inclusion of data set identifiers in article manuscripts. The ADS will be the central system that will allow the journals to verify data set identifiers. The "master verifier" has been implemented in prototype form at the ADS. We started to include more journals in Geosciences/Geophysics in the ADS. The Royal astronomical Society has decided to archive their on-line journals in the ADS three years after publication. We have started to process these older on-line articles in order to archive them in the ADS. Our mirror site in Korea now has a full article mirror. We developed XML output capability in the ADS. This will make it easier to exchange data with other data systems. We started the development of new indexing software that will eventually reduce the indexing time for a database from days to hours or less. The ADS was represented at the IAU General Assembly with a poster. Discussions with the IAU management were held about extending the ADS IAU collaborations.

Murray, Stephen S.↗
NASA NTRS2004

A Low-Cost Clustered Archive Approach for Storing Remote Sensing Data

As part of NASA's Earth Observing System (EOS) Data and Information System (EOSDIS), the Moderate Resolution Imaging Spectrometer (MODIS) Data Processing System (MODAPS) is now processing data from two instruments on the EOS flag ship spacecraft Terra and Aqua. Between the two, MODAPS is generating over 1 Terrabyte of data per day and has surpassed 3 Petabytes of total data. The bulk of the data is stored near-line in StorageTek Powerhorn tape jukeboxes. Accessing data that has been moved to tape involves submitting an order, scheduling the tape, and waiting for the data to become available. I am developing a low cost clustered archive that could enable storing a very large amount of data such as the MODIS data described above in an organized fashion on a cluster of commodity hardware using low cost SATA hard drives such that the files are directly available online. The system takes full advantage of Open Source software, using the GNULinux operating system, PostgreSQL relational database and the Apache HTTP Server. This poster session will depict my approach, the interface to the archive, a brief discussion of the internals of the system and some performance numbers from my prototyping. I will also describe various costs and benefits of this approach versus the traditional large tape jukebox approach currently in use in the EOSDIS.

Tilmes, Curt↗
NASA NTRS1993

Develop advanced nonlinear signal analysis topographical mapping system

The SSME has been undergoing extensive flight certification and developmental testing, which involves some 250 health monitoring measurements. Under the severe temperature pressure, and dynamic environments sustained during operation, numerous major component failures have occurred, resulting in extensive engine hardware damage and scheduling losses. To enhance SSME safety and reliability, detailed analysis and evaluation of the measurements signal are mandatory to assess its dynamic characteristics and operational condition. Efficient and reliable signal detection techniques will reduce catastrophic system failure risks and expedite the evaluation of both flight and ground test data, and thereby reduce launch turn-around time. The basic objective of this contract are threefold: (1) Develop and validate a hierarchy of innovative signal analysis techniques for nonlinear and nonstationary time-frequency analysis. Performance evaluation will be carried out through detailed analysis of extensive SSME static firing and flight data. These techniques will be incorporated into a fully automated system. (2) Develop an advanced nonlinear signal analysis topographical mapping system (ATMS) to generate a Compressed SSME TOPO Data Base (CSTDB). This ATMS system will convert tremendous amounts of complex vibration signals from the entire SSME test history into a bank of succinct image-like patterns while retaining all respective phase information. A high compression ratio can be achieved to allow the minimal storage requirement, while providing fast signature retrieval, pattern comparison, and identification capabilities. (3) Integrate the nonlinear correlation techniques into the CSTDB data base with compatible TOPO input data format. Such integrated ATMS system will provide the large test archives necessary for a quick signature comparison. This study will provide timely assessment of SSME component operational status, identify probable causes of malfunction, and indicate feasible engineering solutions. The final result of this program will yield an ATMS system of nonlinear and nonstationary spectral analysis software package integrated with the Compressed SSME TOPO Data Base (CSTDB) on the same platform. This system will allow NASA engineers to retrieve any unique defect signatures and trends associated with different failure modes and anomalous phenomena over the entire SSME test history across turbo pump families.

Jong, Jen-Yi↗
NASA NTRS1988

Low cost real time interactive analysis system

Efforts continue to develop a low cost real time interactive analysis system for the reception of satellite data. A multi-purpose ingest hardware software frame formatter was demonstrated for GOES and TIROS data and work is proceeding on extending the capability to receive GMS data. A similar system was proposed as an archival and analysis system for use with INSAT data and studies are underway to modify the system to receive the planned SeaWiFS (ocean color) data. This system was proposed as the core of a number of international programs in support of U.S. AID activities. Systems delivered or nearing final testing are listed.

Stetina, F.↗
NASA NTRS2000

BOREAS AFM-08 ECMWF Hourly Surface and Upper Air Data for the SSA and NSA

The Boreal Ecosystem-Atmosphere Study (BOREAS) Airborne Fluxes and Meteorology (AFM)-8 team focused on modeling efforts to improve the understanding of the diurnal evolution of the convective boundary layer over the boreal forest. This data set contains hourly data from the European Center for for Medium-Range Weather Forecasts (ECMWF) operational model from below the surface to the top of the atmosphere, including the model fluxes at the surface. Spatially, the data cover a pair of the points that enclose the rawinsonde sites at Candle Lake, Saskatchewan, in the Southern Study Area (SSA) and Thompson, Manitoba, in the Northern Study Area (NSA). Temporally, the data include the two time periods of 13 May 1994 to 30 Sept 1994 and 01 Mar 1996 to 31 Mar 1997. The data are stored in tabular ASCII files. The number of records in the upper air data files may exceed 20,000, causing a problem for some software packages. The ECMWF hourly surface and upper air data are available from the Earth Observing System Data and Information System (EOSDIS) Oak Ridge National Laboratory (ORNL) Distributed Active Archive Center (DAAC). The data files are available on a CD-ROM (see document number 20010000884).

Viterbo, Pedro↗
NASA NTRSDate not supplied

Extravehicular Activity Mission System Software (EMSS) - Enabling Human Planetary Exploration Data Within The Broader Planetary Data Ecosystem

The planetary science community is once again on the verge of generating, capturing and analyzing human planetary exploration data, this time via the Artemis program. Artemis missions will involve robotic missions in addition to human extravehicular activity (EVA) where crew will be generating scientific data [1]. Present-day robotic mission data expectations for data archiving involves ingesting data into the Planetary Data System (PDS), but how might PDS be leveraged/adapted/ready (or not) for human spaceflight mission data, particularly EVA data that includes non-scientific data that provides important context to the scientific data gathered on the lunar surface? This question has broader implications than what this abstract can answer, but we wanted to pose the question to 1) get conversations started and 2) highlight how operations software data handling could play a role in overall data curation.

M J Miller↗
NASA NTRS1991

Computing and data processing

The applications of computers and data processing to astronomy are discussed. Among the topics covered are the emerging national information infrastructure, workstations and supercomputers, supertelescopes, digital astronomy, astrophysics in a numerical laboratory, community software, archiving of ground-based observations, dynamical simulations of complex systems, plasma astrophysics, and the remote control of fourth dimension supercomputers.

Smarr, Larry↗
NASA NTRS1992

GammaCore: The Compton Observatory research environment

The Compton Observatory Science Support Center (COSSC) is developing a coherent analysis environment for the analysis of Compton and other gamma-ray astronomy data. This environment, GammaCore, allows the astronomer to access the data analysis systems developed at the Principal Investigator (PI) sites for the four Compton Observatory instruments. In addition users have access to standard astronomical tools such as IRAF, IDL, and XANADU. The user interface of GammaCore is the AGCL (AnswerGarden Command Language), developed at the AXAF Science Center. The parameter interface supported by the AGCL allows GammaCore to access all PI software systems in a uniform fashion. These systems are quite different, having been developed independently on heterogeneous systems without much concern for general portability. The data kibitzer concept, where a window running in a specific PI environment is controlled by the AGCL, has been used extensively. Users can choose to view what is going on in the native environment, to use the window to control PI software directly, or to ignore the PI systems entirely and to work only through the homogeneous AGCL interface. Software developed at the COSSC is also integrated within GammaCore. Extensive facilities for conversions of PI data formats to and from FITS have been developed. Access to the Compton data archive and catalogs will also be completely integrated with the GammaCore. Users can retrieve any publicly available Compton data. This paper examines the issues that have arisen in attempting to meld these widely diverse systems. The advantages and limitations of the parameter interface and the kibitzer are discussed along with issues of data portability, documentation, and the feasibility of multi-instrument analysis. Limited capabilities are now available within GammaCore with significant enhancements planned over the coming year. An implementation including all PI systems will be available within that time. Instructions on how to access GammaCore and how to get more information are given.

Mcglynn, T. A.↗
NASA NTRS2004

Aura Atmospheric Data Products and Their Availability from NASA Goddard Earth Sciences DAAC

NASA's EOS-Aura spacecraft was launched successfully on July 15, 2004. The four instruments onboard the spacecraft are the Microwave Limb Sounder (MLS), the Ozone Monitoring Instrument (OMI), the Tropospheric Emission Spectrometer (TES), and the High Resolution Dynamics Limb Sounder (HBDLS). The Aura instruments are designed to gather earth sciences measurements across the ultraviolet, visible, infra-red, thermal and microwave regions of the electromagnetic spectrum. Aura will provide over 70 distinct standard atmospheric data products for use in ozone layer and surface UV-B monitoring, air quality forecast, and atmospheric chemistry and climate change studies (http://eosaura.gsfc.nasa.gov/). These products include earth-atmosphere radiances and solar spectral irradiances; total column, tropospheric, and profiles of ozone and other trace gases, surface W-B flux; clouds and aerosol characteristics; and temperature, geopotential height, and water vapor profiles. The MLS, OMI, and HIRDLS data products will be archived at the NASA Goddard Earth Sciences (GES) Distributed Active Archive Center (DAAC), while data from TES will be archived at NASA Langley Research Center DAAC. Some of the standard products which have gone through quick preliminary checks are already archived at the GES DAAC (http://daac.nsfc.nasa.gov/) and are available to the Aura science team and data validation team members for data validation; and to the application and visualization software developers, for testing their application modules. Once data are corrected for obvious calibration problems and partially validated using in-situ observations, they would be made available to the broader user community. This presentation will provide details of the whole suite of Aura atmospheric data products, and the time line of the availability of the rest of the preliminary products and of the partially validated provisional products. Software and took available for data access, visualization, and data mining will also be discussed.

Ahmad, S.↗
NASA NTRS2007

AIRSAR Web-Based Data Processing

The AIRSAR automated, Web-based data processing and distribution system is an integrated, end-to-end synthetic aperture radar (SAR) processing system. Designed to function under limited resources and rigorous demands, AIRSAR eliminates operational errors and provides for paperless archiving. Also, it provides a yearly tune-up of the processor on flight missions, as well as quality assurance with new radar modes and anomalous data compensation. The software fully integrates a Web-based SAR data-user request subsystem, a data processing system to automatically generate co-registered multi-frequency images from both polarimetric and interferometric data collection modes in 80/40/20 MHz bandwidth, an automated verification quality assurance subsystem, and an automatic data distribution system for use in the remote-sensor community. Features include Survey Automation Processing in which the software can automatically generate a quick-look image from an entire 90-GB SAR raw data 32-MB/s tape overnight without operator intervention. Also, the software allows product ordering and distribution via a Web-based user request system. To make AIRSAR more user friendly, it has been designed to let users search by entering the desired mission flight line (Missions Searching), or to search for any mission flight line by entering the desired latitude and longitude (Map Searching). For precision image automation processing, the software generates the products according to each data processing request stored in the database via a Queue management system. Users are able to have automatic generation of coregistered multi-frequency images as the software generates polarimetric and/or interferometric SAR data processing in ground and/or slant projection according to user processing requests for one of the 12 radar modes.

Chu, Anhua↗
NASA NTRS2019

EARTHDATA PUB: A Data Publication Workflow Solution for NASA’s EOSDIS

Each NASA Distributed Active Archive Center (DAAC) faces the challenge of dealing with an increasingly diverse number of publishable data products from diverse data producers. Data producers, on the other hand, may experience pain points when interacting with the EOSDIS for the first time or when publishing different data at different DAACs. As a result, there has been a growing need to develop a common software framework that serves as a common interface for data producers, rigorously defines the data publication procedure for DAAC staff, facilitates the management of various data publication processes, and tracks the progress of data publication. This software should also account for the different configurations at different DAACs. Currently, two primary data publication workflow and tracking tools exist in operation at the EOSDIS: Semi-Automated ingest System (SAuS) and Data Publication workflow Portal (DAPPeR). However, neither tool is cloud-ready. Automated data processing could be managed by Cumulus, an EOSDIS cloud-based data ingest, archive and management system. However, Cumulus does not support manual tasks or on-premise implementations. We propose to develop the Earthdata Publication Minimum Viable Product (Earthdata Pub MVP) -- a cloud-hosted solution that works with both cloud and on-premise systems and implements the communications and exchange requirements generated by the Earthdata Pub information architecture team.

Rice, Justin L.↗
NASA NTRS1985

Studies of lightning data in conjunction with geostationary satellite data

Since January, work has been proceeding on the first phase of this project: the creation of an extensive real-time lightning data base accessible via the Space Science and Engineering Center McIdas system. The purpose of this endeavor is two-fold: to enhance the availability and ease of access to lightning data among the various networks, governmental and research agencies; and to test the feasiblity and desirability of such efforts in succeeding years. The final steps in the creation of the necessary communications links, hardware, and software are in the process of being completed. Operations ground rules for access among the various users have been discussed and are being refined. While the research planned for the last year of the project will rely for the most part on archived, quality-controlled data from the various networks, the real-time data will provide a valuable first-look at potentially interesting case studies. For this purpose, tools are being developed on McIdas for display and analysis of the data as they become available. In conjunction with concurrent GOES real-time imagery, strike locations can be plotted, gridded and contoured, or displayed in various statistical formats including frequency distributions, histograms, and scatter plots. The user may also perform these functions in relation to arbitrarily defined areas on the satellite image. By mid-May these preparations for the access and analysis of real-time lightning data are expected to be complete.

Auvine, B.↗
NASA NTRS1995

Radio-science performance analysis software

The Radio Science Systems Group (RSSG) provides various support functions for several flight project radio-science teams. Among these support functions are uplink and sequence planning, real-time operations monitoring and support, data validation, archiving and distribution functions, and data processing and analysis. This article describes the support functions that encompass radio-science data performance analysis. The primary tool used by the RSSG to fulfill this support function is the STBLTY program set. STBLTY is used to reconstruct observable frequencies and calculate model frequencies, frequency residuals, frequency stability in terms of Allan deviation, reconstructed phase, frequency and phase power spectral density, and frequency drift rates. In the case of one-way data, using an ultrastable oscillator (USO) as a frequency reference, the program set computes the spacecraft transmitted frequency and maintains a database containing the in-flight history of the USO measurements. The program set also produces graphical displays. Some examples and discussions on operating the program set on Galileo and Ulysses data will be presented.

Morabito, D. D.↗
NASA NTRS1992

FITS data conversion efforts at the Compton Observatory Science Support Center

The Compton Gamma Ray Observatory (CGRO) is an active, earth orbiting satellite consisting of four gamma-ray telescopes. Each telescope is maintained by an independent principal investigator (PI) team, and each PI team has devised separate data formats to handle the needs of their particular instrument. As mandated by NASA, the Compton Observatory Science Support Center (COSSC) intends to archive and distribute PI data to the public in FITS (Flexible Image Transport System) format. To accomplish this task, we at COSSC have been developing a set of general purpose software tools that facilitate the transformation of non-FITS formatted data into FITS format. These tools, known as ToFU (To FITS Utilities), serve as the kernel of our CGRO data conversion software. This presentation describes the problems encountered in transcribing large amounts of data into a standard FITS form and the capabilities of the COSSC-built conversion software designed to perform the transformations.

Jennings, D. G.↗
NASA NTRS1996

Implementation of a Campuswide Distributed Mass Storage Service: the Dream Versus Reality

In 1990, a technical team at NASA Lewis Research Center, Cleveland, Ohio, began defining a Mass Storage Service to pro- wide long-term archival storage, short-term storage for very large files, distributed Network File System access, and backup services for critical data dw resides on workstations and personal computers. Because of software availability and budgets, the total service was phased in over dm years. During the process of building the service from the commercial technologies available, our Mass Storage Team refined the original vision and learned from the problems and mistakes that occurred. We also enhanced some technologies to better meet the needs of users and system administrators. This report describes our team's journey from dream to reality, outlines some of the problem areas that still exist, and suggests some solutions.

Prahst, Stephen↗
NASA NTRSDate not supplied

Biomass Harmonization and SAR Analysis with the Multi-mission Algorithm and Analysis Platform (MAAP)

The Multi‐mission Algorithm and Analysis Platform (MAAP) is a collaborative effort between NASA and the European Space Agency (ESA) to support above ground biomass (AGB) research in an open science framework. MAAP brings together relevant data, algorithms, and computing capabilities in a common cloud environment to address the challenges of sharing and processing data from field, airborne and satellite measurements. MAAP was publicly released in October 2021, providing computing capabilities co-located with the data, a collaborative coding and analysis environment, and a set of interoperable tools and algorithms developed to support the estimation and visualization of data. MAAP has allowed scientists from both North America and Europe to collaborate on the generation and analysis/visualization of data derived from multiple, discipline-adjacent missions in an open, collaborative environment that has reached beyond traditional scientific investigation. MAAP has been used to support multiple scientific activities. To date, existing LiDAR data from multiple platforms has been calibrated with field measurements and combined for more comprehensive and accurate estimates of above ground biomass AGB; these LiDAR platforms include airborne (e.g. LVIS), the International Space Station (NASA’s Global Ecosystem Dynamics Investigation (GEDI), and satellites (e.g. ICESat-2). The current challenge is to effectively and seamlessly combine the aforementioned LiDAR-based data with new data sources such as P-band RADAR from ESA’s upcoming BIOMASS mission, existing ESA Sentinel-1 C-band SAR, and the 30 PB/yr of high cadence global coverage L-band SAR data from the upcoming NASA-ISRO SAR (NISAR) mission. Recent analysis using MAAP merged ICESat-2 and optical data (Harmonized Landsat Sentinel) produced the most comprehensively precise estimate of boreal-wide AGB to date. Another effort using MAAP is the production and open distribution of global comparisons of AGB map estimates, including from ICESat-2 and GEDI, to bolster stakeholder uptake for policy applications. These map estimates will feed into the Intergovernmental Panel on Climate Change (IPCC) database, likely aiding the next Global Carbon Stocktake of the UNFCCC. Furthermore, the biomass retrieval intercomparison exercise BRIX-2 could benefit from the MAAP providing standardized test cases (based on airborne campaign and spaceborne data) allowing the community to develop and apply retrieval algorithms based on these test cases, while forthcoming SAR data training curricula could also use the MAAP as a teaching and learning platform. The MAAP is meeting the challenges inherent in international, open science collaboration and large scale computing with a platform that is entirely open source and cloud native, using open standards for data access, manipulation, protocols, and formats. The MAAP data system consists of a dedicated data store whose data is indexed in an online catalog conforming to established metadata, application programmatic interfaces (APIs), and service interface standards, using an implementation of the open sourced NASA Common Metadata Repository. Federation of user identities allows users from either NASA or ESA to access and consume services from the other using a unified metadata catalog for the data utilized across the ESA and NASA MAAP platforms. Similarly, we are exploring how to increase interoperability to achieve a common approach to packaging, orchestrating and executing algorithms, with interoperable access to data for subsetting, fast browse, and cloud-optimized access, all using interoperable standards such as those from the Open Geospatial Consortium (OGC). Designed for interoperability, ESA and NASA utilize a common architecture for the software platform. It provides a cloud-based algorithm development environment (ADE) that enables scientists to develop algorithms collaboratively with access to the MAAP data catalog as well as other data archives. MAAP provides an Eclipse Che-based ADE supporting both Python and R languages, popular in this biomass community. Algorithms developed and containerized within the ADE can be deployed to run to thousands of computational nodes in the MAAP’s data processing system (DPS), dramatically speeding up processing and giving scientists a rapid, iterative turnaround of results. NASA’s implementation of the DPS is based on the Hybrid Science Data System (HySDS) framework, used by NASA flight projects to produce Earth science standard products.

cloud computing↗
NASA NTRS2007

System for Secure Integration of Aviation Data

The Aviation Data Integration System (ADIS) of Ames Research Center has been established to promote analysis of aviation data by airlines and other interested users for purposes of enhancing the quality (especially safety) of flight operations. The ADIS is a system of computer hardware and software for collecting, integrating, and disseminating aviation data pertaining to flights and specified flight events that involve one or more airline(s). The ADIS is secure in the sense that care is taken to ensure the integrity of sources of collected data and to verify the authorizations of requesters to receive data. Most importantly, the ADIS removes a disincentive to collection and exchange of useful data by providing for automatic removal of information that could be used to identify specific flights and crewmembers. Such information, denoted sensitive information, includes flight data (here signifying data collected by sensors aboard an aircraft during flight), weather data for a specified route on a specified date, date and time, and any other information traceable to a specific flight. The removal of information that could be used to perform such tracing is called "deidentification." Airlines are often reluctant to keep flight data in identifiable form because of concerns about loss of anonymity. Hence, one of the things needed to promote retention and analysis of aviation data is an automated means of de-identification of archived flight data to enable integration of flight data with non-flight aviation data while preserving anonymity. Preferably, such an automated means would enable end users of the data to continue to use pre-existing data-analysis software to identify anomalies in flight data without identifying a specific anomalous flight. It would then also be possible to perform statistical analyses of integrated data. These needs are satisfied by the ADIS, which enables an end user to request aviation data associated with de-identified flight data. The ADIS includes client software integrated with other software running on flight-operations quality-assurance (FOQA) computers for purposes of analyzing data to study specified types of events or exceedences (departures of flight parameters from normal ranges). In addition to ADIS client software, ADIS includes server hardware and software that provide services to the ADIS clients via the Internet (see figure). The ADIS server receives and integrates flight and non-flight data pertaining to flights from multiple sources. The server accepts data updates from authorized sources only and responds to requests from authorized users only. In order to satisfy security requirements established by the airlines, (1) an ADIS client must not be accessible from the Internet by an unauthorized user and (2) non-flight data as airport terminal information system (ATIS) and weather data must be displayed without any identifying flight information. ADIS hardware and software architecture as well as encryption and data display scheme are designed to meet these requirements. When a user requests one or more selected aviation data characteristics associated with an event (e.g., a collision, near miss, equipment malfunction, or exceedence), the ADIS client augments the request with date and time information from encrypted files and submits the augmented request to the server. Once the user s authorization has been verified, the server returns the requested information in de-identified form.

Kulkarni, Deepak↗