SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The NASA GES DISC is the primary archive for the Aura OMI products. After 20 years of operations, the OMI instrument is nearing its end and entering Phase-F (closeout). It is time to begin the process of preserving the data, documentation, software and associated information that were produced by the OMI project over its lifetime.
Remote Sensing data generation by NASA to study Earth s geophysical processes was initiated in 1960 with the launch of the first Television Infrared Observation Satellite Program (TIROS), to develop a meteorological satellite information system. What would be deemed as a primitive data set by today s standards, early Earth science missions were the foundation upon which today s remote sensing instruments have built their scientific success, and tomorrow s instruments will yield science not yet imagined. NASA Scientific Data Stewardship requirements have been documented to ensure the long term preservation and usability of remote sensing science data. In recent years, the Federation of Earth Science Information Partners and NASA s Earth Science Data System Working Groups have organized committees that specifically examine standards, processes, and ontologies that can best be employed for the preservation of remote sensing data, supporting documentation, and data provenance information. This presentation describes the activities, issues, and implementations, guided by the NASA Earth Science Data Preservation Content Specification (423-SPEC-001), for preserving instrument characteristics, and data processing and science information generated for 20 Earth science instruments, spanning 40 years of geophysical measurements, at the NASA s Goddard Earth Sciences Data and Information Services Center (GES DISC). In addition, unanticipated preservation/implementation questions and issues in the implementation process are presented.
Data lifecycle management awareness is common today; planners are more likely to consider lifecycle issues at mission start. NASA remote sensing missions are typically subject to life cycle management plans of the Distributed Active Archive Center (DAAC), and NASA invests in these national centers for the long-term safeguarding and benefit of future generations. As stewards of older missions, it is incumbent upon us to ensure that a comprehensive enough set of information is being preserved to prevent the risk for information loss. This risk is greater when the original data experts have moved on or are no longer available. Preservation of items like documentation related to processing algorithms, pre-flight calibration data, or input-output configuration parameters used in product generation, are examples of digital artifacts that are sometimes not fully preserved. This is the grey area of information preservation; the importance of these items is not always clear and requires careful consideration. Missing important metadata about intermediate steps used to derive a product could lead to serious challenges in the reproducibility of results or conclusions. Organizations are rapidly recognizing that the focus of life-cycle preservation needs to be enlarged from the strict raw data to the more encompassing arena of information lifecycle management. By understanding what constitutes information, and the complexities involved, we are better equipped to deliver longer lasting value about the original data and derived knowledge (information) from them. The NASA Earth Science Data Preservation Content Specification is an attempt to define the content necessary for long-term preservation. It requires new lifecycle infrastructure approach along with content repositories to accommodate artifacts other than just raw data. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) setup an open-source Preservation System capable of long-term archive of digital content to augment its raw data holding. This repository is being used for such missions as HIRDLS, UARS, TOMS, OMI, among others. We will provide a status of this implementation; report on challenges, lessons learned, and detail our plans for future evolution to include other missions and services.
For datasets to be usable, many pieces of information in addition to the data themselves are essential. During the active parts of the lifecycle of dataset generating projects, the needed information is usually accessible through individuals familiar with the various aspects of the projects. However, the utility of datasets tends to outlive the lives of projects, by several decades in many cases. Thus it is essential to capture all the relevant information about the datasets, data, metadata and associate knowledge that is sufficient to read, understand, interpret and reuse the datasets, while the projects are still active. The capture and preservation should be such that the data are usable when no consultation is available from the original project participants. Identification of specific categories of content through an international standard is beneficial to the user communities of the future, so that projects involving Earth observations and generating data products can consistently plan for preservation and future usability of the project outcomes. While there are existing standards that address archival and preservation in general, there are no existing international standards or specifications today to address what content should be preserved. The standard, ISO 19165-1, titled "Geographic Information - Preservation of digital data and metadata Part 1: Fundamentals" considers geographic information preservation in general. It acknowledges that "specific content items needed to preserve the full provenance and context of the data and associated metadata depend on the needs of the designated community and types of datasets (e.g., maps, remotely sensed data from satellites and airborne instruments, physical samples). Follow-up parts to this standard may be developed detailing content items appropriate to individual disciplines." NASA proposed an extension to this standard, titled "Geographic information -- Preservation of digital data and metadata -- Part 2: Content specifications for Earth observation data and derived digital products." The development of this extension is in progress with participation by an international team representing nine countries. The purpose of this paper is to introduce this standard and report on its status.
As NASA prepares for the follow on to EOS through the deployment of many new space and airborne-based assets envisioned under the ESO initiative, the need for ensuring user access to the very large and diverse archive of past missions remains a critical endeavor for current and future research. The ESDIS team has been working for many years on developing strategies and implementations that will make the linkage between the data held in our archives and the emerging missions possible. The focus on how best to preserve these past data has proven to be a collaborative effort, and one that may well benefit from the adoption of innovative technical tools that enable organizational efficiencies. This poster reveals both aspects of NASA’s EOS data repository strategy by discussing the collaborative community efforts of development of ISO 19165-2 – an international standard supported by a team from many countries around the world, the use by EOSDIS DAACs of a Preservation Content Specification, and ongoing work with mission teams to ensure mission data and all associated information are preserved and available for current and future use.
Many NASA Earth Observing System (EOS) have either already reached the end of their active life or are nearing it. Preservmissionsation of data products is a fairly well defined task for the NASA EOS Data Centers or DAACs.The Goddard Earth Sciences Data and Information Services Center (GES-DISC) has implemented a repository system, which is capable of long-term archive of documentation artifacts and other associated digital content. The existing GES-DISC Repository System is based on Fedora Commons, an open-source repository management software, for cost savings and flexibility.The first mission to utilize the GES-DISC Repository System was the High Resolution Dynamics Limb Sounder (HIRDLS) on the Aura spacecraft. Since then, the GES DISC has gathered documentation from the UARS and TOMS into the Repository. The Microwave Limb Sounder (MLS) team has begun delivering some early pre-launch documents to the GES-DISC Repository System as well. Other missions in planning or progress include AIRS, OMI, SORCE, SNPP Sounder, and TRMM.
Long-term data sets with data from many missions are needed to study trends and validate model results that are typical in Earth System Science research. Data and derived products originate from multiple missions (spaceborne, airborne and/or in situ) and from multiple organizations. During the missions as well as well past their termination, it is essential to preserve the data and products to support future studies. Key aspects of preservation are: preserving bits and ensuring data are uncorrupted, preserving understandability with appropriate documentation, and preserving reproducibility of science with appropriate documentation and other artifacts. Computer technology provides adequate standards to ensure that, with proper engineering, bits are preserved as hardware evolves. However, to ensure understandability and reproducibility, it is essential to plan ahead to preserve all the relevant data and information. There are currently no standards to identify the content that needs to be preserved, leading to non-uniformity in content and users not being sure of whether preserved content is comprehensive. Each project, program or agency can specify the items to be preserved as a part of its data management requirements. However, broader community consensus that cuts across organizational or national boundaries would be needed to ensure comprehensiveness, uniformity and long-term utility of archived data. The Federation of Earth Science Information Partners (ESIP), a diverse network of scientists, data stewards and technology developers, has a forum for ESIP members to collaborate on data preservation issues. During early 2011, members discussed the importance of developing a Provenance and Context Content Standard (PCCS) and developed an initial list of content items. This list is based on the outcome of a NASA and NOAA meeting held in 1998 under the auspices of the USGCRP, documentation requirements from NOAA and our experience with some of the NASA Earth science missions. The items are categorized into the following 8 high level categories: Preflight/Pre-Operations, Products (Data), Product Documentation, Mission Calibration, Product Software, Algorithm Input, Validation, Software Tools.
Information preserving data compression systems with coding algorithm developed for noiseless channel conditions
Various remote sensing agencies of the world have created a data rich environment for research and applications over the last three decades. Especially over the last decade, the volume and variety of data useful for Earth system science have increased quite rapidly. One of the key purposes of collecting these data and generating useful digital products containing derived geophysical parameters is to study the long-term trends in the Earth s behavior. Long-term observational data and derived products are essential for validating results from models that predict the future behavior of the Earth system. Given the significant resources expended in gathering the observational data and developing the derived products, it is important to preserve them for the benefit of future generations of users. Preservation involves maintaining the bits with no loss (or loss within scientifically acceptable bounds) as they move across systems as well as over time, ensuring readability over time, and providing for long-term understandability and repeatability of previously obtained results. In order to ensure long-term understandability and repeatability, it is necessary to identify all items of content that must be preserved and plan for such preservation. This paper discusses the need for a standard enumerating and describing such content items and reports on the progress made by NASA and the Federation of Earth Science Information Partners (ESIP Federation) in the U.S. towards such a standard.
The purpose of this study was to determine the optimum points of evaluation of data compressed by means of polynomial smoothing. It is shown that a set y of m statistically independent observations Y(t sub 1), Y(t sub 2), ... Y(t sub m) of a quantity X(t), which can be described by a (n-1)th degree polynomial in time, may be represented by a set Z of n statistically independent compressed observations Z (tau sub 1), Z (tau sub 2),...Z (tau sub n), such that The compressed set Z has the same information content as the observed set Y. the times tau sub 1, tau sub 2,.. tau sub n are the zeros of an nth degree polynomial P sub n, to whose definition and properties the bulk of this report is devoted. The polynomials P sub n are defined as functions of the observation times t sub 1, t sub 2,.. t sub n, and it is interesting to note that if the observation times are continuously distributed the polynomials P sub n degenerate to legendre polynomials. The proposed data compression scheme is a little more complex than those usually employed, but has the advantage of preserving all the information content of the original observations.
No abstract available
The National Aeronautic and Space Administration (NASA) Commercial Smallsat Data Acquisition (CSDA) Program was established to identify, evaluate, and acquire data from commercial satellite companies that support and complement NASA’s Earth sciences missions and research goals. Since becoming a sustained program in 2020, CSDA has been developing a data system to provide scalable, efficient, continuous, and repeatable data management processes for all commercial data acquired by NASA. This includes providing access to commercial data to approved science investigators through NASA developed and vendor operated interfaces. As the commercial data user community and volume of data acquired by CSDA has grown, the team managing these data has adapted technologies and services to better address the data and information search, discovery, and access needs of the science user community. This presentation will provide an overview of CSDA Program data system activities including data availability, development of CSDA user interfaces, challenges in managing large-volume diverse datasets, and long-term data preservation activities to retain data for scientific reproducibility.
Digital data, acquired by the National Landsat Remote Sensing Program, document nearly two decades of global agricultural, environmental, and sociological change. The data were widely applied and continue to be essential to a variety of geologic, hydrologic, agronomic, and strategic programs and studies by governmental, academic, and commercial researchers. Landsat data were acquired by five observatories that use primarily two digital sensor systems. The Multispectral Scanner (MSS) was onboard all five Landsats, which have orbited over 19 years; the higher resolution Thematic Mapper (TM) sensor acquired data for the last 9 years on Landsats 4 and 5 only. The National Land Remote Sensing Archive preserves the 800,000 scenes, which total more than 60 terabytes of data, on master tapes that are steadily deteriorating. Data are stored at two terabytes of data, on master tapes that are steadily deteriorating. Data are stored at two locations (Sioux Falls, South Dakota and Landover, Maryland), in three archive formats. The U.S. Geological Survey's EROS Data Center has initiated a project to consolidate and convert, over the next 4 years, two of the archive formats from antiquated instrumentation tape to rotary-recorded cassette magnetic tape. The third archive format, consisting of 300,000 scenes of MSS data acquired from 1972 through 1978, will not be converted because of budgetary constraints. This data preservation project augments EDC's experience in data archiving and information management, expertise that is critical to EDC's role as a Distributed Active Archive Center for the Earth Observing System, a new and much larger national earth science program.
This paper describes a prototype suite of Excel-based tools that could be used for estimating lifecycle costs for newly planned or modified long-term archival facilities. These tools may also prove valuable for monitoring the long-term performance of such facilities once operational. Cost estimation is by analogy, using statistical curve-fitting techniques across a database of comparable data activities. The database currently includes 29 operational data centers ranging from small (2 FTEs) to large (66 FTEs), and is readily expandable to include additional activities specifically involving data preservation and added value. Each comparable data center is described in terms of its staffing, throughput workload, archival and distribution requirements, levels of user service, overall complexity, degree of automation, and other data, comprising 94 distinct descriptors in all. The descriptors were developed by normalizing heterogeneous data from the various centers and mapping them into an Excel framework consistent with the OAIS reference model. A user-friendly tool is provided for generating input to and updating the comparables database. This tool can also be used to benchmark the performance (in terms of cost versus throughput) of an operational data center, and to update the staffing, cost and workload data on a periodic basis. The comparables database could thus provide a history of staffing and throughput over time, as a means of performance monitoring and providing feedback for continuous improvement. Ancillary tools are also provided for performing "what-if' cost exercises for planning purposes, and for graphical display of data and results. We provide a high-level description of the tools; present our experiences and observations on gathering the information and maintaining the database; and discuss how this tool set might be applied to long term archives.
The "Grid" is an emerging infrastructure for coordinating access across autonomous organizations to distributed, heterogeneous computation and data resources. Data grids are being built around the world as the next generation data handling systems for sharing, publishing, and preserving data residing on storage systems located in multiple administrative domains. A data grid provides logical namespaces for users, digital entities and storage resources to create persistent identifiers for controlling access, enabling discovery, and managing wide area latencies. This paper introduces data grids and describes data grid use cases. The relevance of data grids to digital libraries and persistent archives is demonstrated, and research issues in data grids and grid dataflow management systems are discussed.
PCM TV photographic data communication for grand tour of outer planets, emphasizing adaptive information-preserving data compression system for optimal performance
A local change in intensity (edge) is a characteristic that is preserved when an image is filtered through a bandpass filter. Primal sketch representations of images, using the bandpass-filtered data, have become a common process since Marr proposed his model for early human vision. Here, researchers move beyond the primal sketch extraction to the recovery of intensity and reflectance representations using only the bandpass-filtered data. Assessing the response of an ideal step edge to the Laplacian of Gaussian (NAb/A squared G) filter, they found that the resulting filtered data preserves the original change of intensity that created the edge in addition to the edge location. Using the filtered data, they can construct the primal sketches and recover the original (relative) intensity levels between the boundaries. It was found that the result of filtering an ideal step edge with the Intensity-Dependent Spatial Summation (IDS) filter preserves the actual intensity on both sides of the edge, in addition to the edge location. The IDS filter also preserves the reflectance ratio at the edge location. Therefore, one can recover the intensity levels between the edge boundaries as well as the (relative) reflectance representation. The recovery of the reflectance representation is of special interest as it erases shadowing degradations and other dependencies on temporal illumination. This method offers a new approach to low-level vision processing as well as to high data-compression coding. High compression can be gained by transmitting only the information associated with the edge location (edge primitives) that is necessary for the recovery