Search NASA⌕ Search

SEARCH · Search NASA

Results for “series metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

27 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2015

NASA'S Earth Science Data Stewardship Activities

NASA has been collecting Earth observation data for over 50 years using instruments on board satellites, aircraft and ground-based systems. With the inception of the Earth Observing System (EOS) Program in 1990, NASA established the Earth Science Data and Information System (ESDIS) Project and initiated development of the Earth Observing System Data and Information System (EOSDIS). A set of Distributed Active Archive Centers (DAACs) was established at locations based on science discipline expertise. Today, EOSDIS consists of 12 DAACs and 12 Science Investigator-led Processing Systems (SIPS), processing data from the EOS missions, as well as the Suomi National Polar Orbiting Partnership mission, and other satellite and airborne missions. The DAACs archive and distribute the vast majority of data from NASA’s Earth science missions, with data holdings exceeding 12 petabytes The data held by EOSDIS are available to all users consistent with NASA’s free and open data policy, which has been in effect since 1990. The EOSDIS archives consist of raw instrument data counts (level 0 data), as well as higher level standard products (e.g., geophysical parameters, products mapped to standard spatio-temporal grids, results of Earth system models using multi-instrument observations, and long time series of Earth System Data Records resulting from multiple satellite observations of a given type of phenomenon). EOSDIS data stewardship responsibilities include ensuring that the data and information content are reliable, of high quality, easily accessible, and usable for as long as they are considered to be of value.

metadata↗
NASA NTRS2022

Revisiting the Solar Research Cyberinfrastructure Needs: A White Paper of Findings and Recommendations

Solar and Heliosphere physics are areas of remarkable data-driven discoveries. Recent advances in high cadence, high-resolution multiwavelength observations, growing amounts of data from realistic modeling, and operational needs for uninterrupted science-quality data coverage generate the demand for a solar metadata standardization and overall healthy data infrastructure. This white paper is prepared as an effort of the working group “Uniform Semantics and Syntax of Solar Observations and Events” created within the “Towards Integration of Heliophysics Data, Modeling, and Analysis Tools” EarthCube Research Coordination Network (@HDMIEC RCN), with primary objectives to discuss current advances and identify future needs for the solar research cyberinfrastructure. The white paper summarizes presentations and discussions held during the special working group session at the EarthCube Annual Meeting on June 19th, 2020, as well as community contribution gathered during a series of preceding workshops and subsequent RCN working group sessions. The authors provide examples of the current standing of the solar research cyberinfrastructure, and describe the problems related to current data handling approaches. The list of the top-level recommendations agreed by the authors of the current white paper is presented at the beginning of the paper.

SMD↗
NASA NTRS2011

Determining the Completeness of the Nimbus Meteorological Data Archive

NASA launched the Nimbus series of meteorological satellites in the 1960s and 70s. These satellites carried instruments for making observations of the Earth in the visible, infrared, ultraviolet, and microwave wavelengths. The original data archive consisted of a combination of digital data written to 7-track computer tapes and on various film media. Many of these data sets are now being migrated from the old media to the GES DISC modern online archive. The process involves recovering the digital data files from tape as well as scanning images of the data from film strips. Some of the challenges of archiving the Nimbus data include the lack of any metadata from these old data sets. Metadata standards and self-describing data files did not exist at that time, and files were written on now obsolete hardware systems and outdated file formats. This requires creating metadata by reading the contents of the old data files. Some digital data files were corrupted over time, or were possibly improperly copied at the time of creation. Thus there are data gaps in the collections. The film strips were stored in boxes and are now being scanned as JPEG-2000 images. The only information describing these images is what was written on them when they were originally created, and sometimes this information is incomplete or missing. We have the ability to cross-reference the scanned images against the digital data files to determine which of these best represents the data set from the various missions, or to see how complete the data sets are. In this presentation we compared data files and scanned images from the Nimbus-2 High-Resolution Infrared Radiometer (HRIR) for September 1966 to determine whether the data and images are properly archived with correct metadata.

Johnson, James↗
NASA NTRS2018

GMI-IPS: Python Processing Software for Aircraft Campaigns

NASA's Atmospheric Tomography Mission (ATom) seeks to understand the impact of anthropogenic air pollution on gases in the Earth's atmosphere. Four flight campaigns are being deployed on a seasonal basis to establish a continuous global-scale data set intended to improve the representation of chemically reactive gases in global atmospheric chemistry models. The Global Modeling Initiative (GMI), is creating chemical transport simulations on a global scale for each of the ATom flight campaigns. To meet the computational demands required to translate the GMI simulation data to grids associated with the flights from the ATom campaigns, the GMI ICARTT Processing Software (GMI-IPS) has been developed and is providing key functionality for data processing and analysis in this ongoing effort. The GMI-IPS is written in Python and provides computational kernels for data interpolation and visualization tasks on GMI simulation data. A key feature of the GMI-IPS, is its ability to read ICARTT files, a text-based file format for airborne instrument data, and extract the required flight information that defines regional and temporal grid parameters associated with an ATom flight. Perhaps most importantly, the GMI-IPS creates ICARTT files containing GMI simulated data, which are used in collaboration with ATom instrument teams and other modeling groups. The initial main task of the GMI-IPS is to interpolate GMI model data to the finer temporal resolution (1-10 seconds) of a given flight. The model data includes basic fields such as temperature and pressure, but the main focus of this effort is to provide species concentrations of chemical gases for ATom flights. The software, which uses parallel computation techniques for data intensive tasks, linearly interpolates each of the model fields to the time resolution of the flight. The temporally interpolated data is then saved to disk, and is used to create additional derived quantities. In order to translate the GMI model data to the spatial grid of the flight path as defined by the pressure, latitude, and longitude points at each flight time record, a weighted average is then calculated from the nearest neighbors in two dimensions (latitude, longitude). Using SciPya's Regular Grid Interpolator, interpolation functions are generated for the GMI model grid and the calculated weighted averages. The flight path points are then extracted from the ATom ICARTT instrument file, and are sent to the multi-dimensional interpolating functions to generate GMI field quantities along the spatial path of the flight. The interpolated field quantities are then written to a ICARTT data file, which is stored for further manipulation. The GMI-IPS is aware of a generic ATom ICARTT header format, containing basic information for all flight campaigns. The GMI-IPS includes logic to edit metadata for the derived field quantities, as well as modify the generic header data such as processing dates and associated instrument files. The ICARTT interpolated data is then appended to the modified header data, and the ICARTT processing is complete for the given flight and ready for collaboration. The output ICARTT data adheres to the ICARTT file format standards V1.1. The visualization component of the GMI-IPS uses Matplotlib extensively and has several functions ranging in complexity. First, it creates a model background curtain for the flight (time versus model eta levels) with the interpolated flight data superimposed on the curtain. Secondly, it creates a time-series plot of the interpolated flight data. Lastly, the visualization component creates averaged 2D model slices (longitude versus latitude) with overlaid flight track circles at key pressure levels. The GMI-IPS consists of a handful of classes and supporting functionality that have been generalized to be compatible with any ICARTT file that adheres to the base class definition. The base class represents a generic ICARTT entry, only defining a single time entry and 3D spatial positioning parameters. Other classes inherit from this base class; several classes for input ICARTT instrument files, which contain the necessary flight positioning information as a basis for data processing, as well as other classes for output ICARTT files, which contain the interpolated model data. Utility classes provide functionality for routine procedures such as: comparing field names among ICARTT files, reading ICARTT entries from a data file and storing them in data structures, and returning a reduced spatial grid based on a collection of ICARTT entries. Although the GMI-IPS is compatible with GMI model data, it can be adapted with reasonable effort for any simulation that creates Hierarchical Data Format (HDF) files. The same can be said of its adaptability to ICARTT files outside of the context of the ATom mission. The GMI-IPS contains just under 30,000 lines of code, eight classes, and a dozen drivers and utility programs. It is maintained with GIT source code management and has been used to deliver processed GMI model data for the ATom campaigns that have taken place to date.

Damon, M. R.↗
NASA NTRS2017

Recovering Nimbus Era Observations at the NASA GES DISC

Between 1964 and 1978, NASA launched a series of seven Nimbus meteorological satellites which provided Earth observations for 30 years. These satellites, carrying a total of 33 instruments to observe the Earth at visible, infrared, ultraviolet, and microwave wavelengths, revolutionized weather forecasting, provided early observations of ocean color and atmospheric ozone, and prototyped location-based search and rescue capabilities. The Nimbus series paved the way for a number of currently operational systems such as the EOS (Earth Observation System) Terra, Aqua, and Aura platforms. The original data archive includes both magnetic tapes and film media. These media are well past their expected end of life, placing at risk valuable data that are critical to extending the history of Earth observations back in time. GES DISC (Goddard Earth Sciences Data and Information Services Center) has been incorporating these data into a modern online archive by recovering the digital data files from the tapes, and scanning images of the data from film strips. The digital data products were written on obsolete hardware systems in outdated file formats, and in the absence of metadata standards at that time, were often written in proprietary file structures. Through a tedious and laborious process, oft-corrupted data are recovered, and incomplete metadata and documentation are reconstructed.

data recovery↗
NASA NTRS2019

WWAO-WSWC Workshop Report 2019 Final

EXECUTIVE SUMMARY The Western States Water Council (WSWC) and the NASA Western Water Applications Office (WWAO) hosted a joint workshop on technology transfer for water management in the Western U.S. The goals of the workshop were to understand how different agencies approach the technology transfer and research to operations (R2O) process, identify best practices, and discuss existing barriers to successful technology infusion into operational water resource management systems at the state and federal level. The workshop took place August 7-9, 2019 in Irvine, CA. Key outcomes of the meeting include the following:• U.S. Rep. Grace Napolitano provided opening remarks for the workshop, where she highlighted the critical value of water data and the importance of collaboration between state and federal agencies in working to advance the use of water data in water management, planning and policy. • A total of 33 participants (including remote participants) were part of the workshop. They included principal investigators and project teams supported by NASA (Cyanobacteria Assessment Network, Evapotranspiration for Western States, Evaporative Stress Index, the Airborne Snow Observatory, Satellite-based Snow Water Equivalent in the Sierra Nevadas, and Fallowed Area Mapping) as well as representatives from federal (USGS, NOAA, USBR, EPA) and state (CA, WY, OR, NE) agency partners. • One main outcome of the meeting was the consensus that successful transitions of new applications and new technologies into operations require careful planning, effective communication within and across institutions, resources and considerable time investments. In addition, there was broad agreement that significant lead time is often required to allow for identification of financial and technical resources to sustain operational use of new data, information and tools.• The meeting included remarks from U.S. Rep. Napolitano and discussions during presentations and breakout groups about key opportunities to develop best practices and streamline the technology transfer process. • For example, one key set of best practices that emerged revolved around the the importance of building trust and establishing clear lines of communication between the research and operational institutions. The conversations led to defining two key components of trust-building. The first aspect is purely technical. It requires effectively demonstrating that the proposed application meets the end user’s needs in terms of accuracy, format, resolution, latency, metadata and documentation. The second aspect of building trust involves developing sustained, productive and mutually-beneficial relationships with the partner operational agency. The best practices presented here span both the technical as well as the relational aspects of cultivating trust. • This workshop served as a first step in developing a broader community discussion around R2O in western water management. Many of the best practices and lessons learned described in this report represent starting places for action within the WWAO, WSWC and our colleagues’ institutions. • Effective implementation of the best practices that emerged from this workshop will require sustained investments of time, resources and transition planning. In recognition of this, the WSWC and the WWAO proposed continuation of discussions begun at the workshop through a series of semi-annual or annual workshops.

Wilardson, Tony↗
NASA NTRS2023

Commercial Smallsat Data Acquisition Program On-ramp #2 Airbus U.S. Synthetic Aperture Radar (SAR) Evaluation Report

In 2017, NASA’s Earth Science Division (ESD) launched the Private-Sector Small Constellation Satellite Data Product Pilot, now referred to as the Commercial Smallsat Data Acquisition (CSDA) program. The objective of CSDA is to identify, evaluate, and acquire commercial remote sensing data that support NASA’s Earth science research and application activities. The Pilot successfully concluded in early 2020, when CSDA transitioned into a sustained program with on-ramping opportunities for new vendors as the industry emerges with new candidates and capabilities. In October 2019, a Request for Information (RFI) seeking capability statements from parties interested in providing data from spaceborne platforms was released for the CSDA on-ramp #2 evaluations. To be responsive to the RFI, the commercial satellite constellations had to consist of three or more operating spacecraft actively collecting data in a non-geostationary orbit with full latitudinal coverage and be U.S. companies. Two vendors responded to the RFI and were evaluated by a committee composed of NASA ESD leadership, program managers, and scientists. Both vendors satisfied the RFI requirements and were asked to respond to a Request for Proposal (RFP). After review of the proposals, NASA entered into a Blanket Purchase Agreement (BPA) with Airbus Defense and Space GEO, Inc. (Airbus) U.S. in September 2021 and with BlackSky Geospatial Solutions, Inc. (BlackSky) in November 2021. In this report, CSDA provides an evaluation of the usefulness of data provided by the Airbus U.S. Synthetic Aperture Radar (SAR) satellite constellation, consisting of TerraSAR-X (launched in 2007), TanDEM-X (launched in 2010), and PAZ (launched in 2018), for advancing NASA’s Earth system science research and applications. The evaluation of the BlackSky commercial data will be provided in a separate report. To conduct the Airbus evaluation, NASA’s ESD augmented 13 existing research projects that could potentially benefit from, and had the expertise to evaluate, the commercial data being considered for longer-term purchase. Investigators from NASA’s Research and Analysis Program science focus areas and from NASA’s Applied Sciences Program elements participated in the evaluation. A summary of the research areas evaluated by the Principal Investigator (PI) teams is presented in Figure 3. CSDA also funded a dedicated activity to evaluate the satellite data quality (calibration and geolocation) independently by assessing the accuracy of data from Airbus. Evaluation activities were carried out by the selected PIs from December 7, 2022, to December 7, 2023. Delivery of datasets requested by the researchers began in January 2023. The vendors were evaluated on the accessibility of data, accuracy and completeness of metadata, and promptness and quality of user support services. Datasets purchased during the evaluation have been archived by NASA and will be made available to current and future government-funded researchers in accordance with the End User License Agreement (EULA). This synthesis report distills and integrates the findings of research reports commissioned by NASA for the Airbus evaluation. This report also includes recommendations that inform the way ahead for the program. The scientific results from the evaluations demonstrated that the commercial data from Airbus were able to advance NASA research and applications. However, the PIs encountered limitations that diminished the usefulness of the data due to the amount of effort that was required to access, preprocess, and analyze these data. One significant issue encountered was the limited spatial and temporal coverage of the data in the Airbus archive that could be used to conduct time series analyses or assessments over large spatial scales. Overall, however, the utility and the quality of the evaluated data outweighed the difficulties encountered, and NASA has concluded that the Airbus SAR data would complement NASA’s existing Earth observation capabilities and Airbus U.S. would qualify to participate in the sustained phase of the program.

Batuhan Osmanoglu↗
NASA NTRS2020

Connecting Users and Applications with Po.daac Hosted GHRSST Data

The 80+ GHRSST public datasets represent a rich resource for sea surface temperature research and applications given their time series length, resolution, spatial coverage, varying measurement types and processing levels, and availability in the full spectrum of PO.DAAC tools and services ecosystem. The PO.DAAC has created a publicly accessible recipe suite for the user community to perform straightforward yet powerful computations on GHRSST data using python recipes, Jupyter notebooks, R, Matlab, and the NCO programming language. These recipes include numerical computations for regional and global SST trends, anomaly derivations, EOF analysis, climate signal reproduction, and ocean phenology. For example, one recipe reproduces a famous SST based warming figure from the Fourth National Climate Assessment (USA) while another focuses on quantifying the regional changes in ocean SST phenology. Most are python-based while some contain hybrid calls and leverage the NCO programming interface too. All are available on the PO.DAAC user forum (https://podaac.jpl.nasa.gov/forum/) and/or via the open source NASA GitHub repository (https://github.com/nasa/podaac_tools_and_services). Several are available in the Jupyter notebook framework including podaacypy (https://github.com/nasa/podaacpy), a recipe for GHRSST granule metadata discovery and application, and more recently a Jupyter notebook developed to support data analysis and visualization of a cloud-based Zarr formatted Level 4 MUR dataset in the AWS Open Data Registry. Throughout the summer of 2020, the PO.DAAC intends to add and migrate more of its numerical recipes to the Jupyter notebook framework and publish them on its open source GitHub repository.

Gentemann, Chelle↗
NASA NTRS2008

Standardizing Interfaces for External Access to Data and Processing for the NASA Ozone Product Evaluation and Test Element (PEATE)

NASA's traditional science data processing systems have focused on specific missions, and providing data access, processing and services to the funded science teams of those specific missions. Recently NASA has been modifying this stance, changing the focus from Missions to Measurements. Where a specific Mission has a discrete beginning and end, the Measurement considers long term data continuity across multiple missions. Total Column Ozone, a critical measurement of atmospheric composition, has been monitored for'decades on a series of Total Ozone Mapping Spectrometer (TOMS) instruments. Some important European missions also monitor ozone, including the Global Ozone Monitoring Experiment (GOME) and SCIAMACHY. With the U.S.IEuropean cooperative launch of the Dutch Ozone Monitoring Instrument (OMI) on NASA Aura satellite, and the GOME-2 instrumental on MetOp, the ozone monitoring record has been further extended. In conjunction with the U.S. Department of Defense (DoD) and the National Oceanic and Atmospheric Administration (NOAA), NASA is now preparing to evaluate data and algorithms for the next generation Ozone Mapping and Profiler Suite (OMPS) which will launch on the National Polar-orbiting Operational Environmental Satellite System (NPOESS) Preparatory Project (NPP) in 2010. NASA is constructing the Science Data Segment (SDS) which is comprised of several elements to evaluate the various NPP data products and algorithms. The NPP SDS Ozone Product Evaluation and Test Element (PEATE) will build on the heritage of the TOMS and OM1 mission based processing systems. The overall measurement based system that will encompass these efforts is the Atmospheric Composition Processing System (ACPS). We have extended the system to include access to publically available data sets from other instruments where feasible, including non-NASA missions as appropriate. The heritage system was largely monolithic providing a very controlled processing flow from data.ingest of satellite data to the ultimate archive of specific operational data products. The ACPS allows more open access with standard protocols including HTTP, SOAPIXML, RSS and various REST incarnations. External entities can be granted access to various modules within the system, including an extended data archive, metadata searching, production planning and processing. Data access is provided with very fine grained access control. It is possible to easily designate certain datasets as being available to the public, or restricted to groups of researchers, or limited strictly to the originator. This can be used, for example, to release one's best validated data to the public, but restrict the "new version" of data processed with a new, unproven algorithm until it is ready. Similarly, the system can provide access to algorithms, both as modifiable source code (where possible) and fully integrated executable Algorithm Plugin Packages (APPs). This enables researchers to download publically released versions of the processing algorithms and easily reproduce the processing remotely, while interacting with the ACPS. The algorithms can be modified allowing better experimentation and rapid improvement. The modified algorithms can be easily integrated back into the production system for large scale bulk processing to evaluate improvements. The system includes complete provenance tracking of algorithms, data and the entire processing environment. The origin of any data or algorithms is recorded and the entire history of the processing chains are stored such that a researcher can understand the entire data flow. Provenance is captured in a form suitable for the system to guarantee scientific reproducability of any data product it distributes even in cases where the physical data products themselves have been deleted due to space constraints. We are currently working on Semantic Web ontologies for representing the various provenance information. A new web site focusing on consolidating informaon about the measurement, processing system, and data access has been established to encourage interaction with the overall scientific community. We will describe the system, its data processing capabilities, and the methods the community can use to interact with the standard interfaces of the system.

Tilmes, Curt A.↗