Search NASA⌕ Search

SEARCH · Search NASA

Results for “data preservation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Multivariate Pointwise Information-Driven Data Sampling and Visualization

With increasing computing capabilities of modern supercomputers, the size of the data generated from the scientific simulations is growing rapidly. As a result, application scientists need effective data summarization techniques that can reduce large-scale multivariate spatiotemporal data sets while preserving the important data properties so that the reduced data can answer domain-specific queries involving multiple variables with sufficient accuracy. While analyzing complex scientific events, domain experts often analyze and visualize two or more variables together to obtain a better understanding of the characteristics of the data features. Therefore, data summarization techniques are required to analyze multi-variable relationships in detail and then perform data reduction such that the important features involving multiple variables are preserved in the reduced data. To achieve this, in this work, we propose a data sub-sampling algorithm for performing statistical data summarization that leverages pointwise information theoretic measures to quantify the statistical association of data points considering multiple variables and generates a sub-sampled data that preserves the statistical association among multi-variables. Using such reduced sampled data, we show that multivariate feature query and analysis can be done effectively. The efficacy of the proposed multivariate association driven sampling algorithm is presented by applying it on several scientific data sets.

97 MATHEMATICS AND COMPUTING↗

Genesis Solar Wind Sample Curation Documentation

Introduction: A scientist with experience as a sample science analyst, provider of flight hardware for multiple missions, and senior engineer in an ISO 2000-rated manufacturing plant has described the timeline of key participants in any PI-led sample return mission, the breadth of the organizations involved [1,2], and, of interest to this meeting, choosing the types of data to preserve and issues of future data accessibility. This work broadens that perspective by giving similar lessons from Genesis sample curation point-of-view. Curation participation regarding data gathering was part of the mission review process from the beginning. Genesis’ story illustrates outcome of several choices about types of data to record and preserve. Precision analysis of solar wind atoms captured in pure, ultraclean substrates is the driving science goal; therefore, detailed documentation was captured from all mission and curation phases and from investigator laboratories because these processes affect the final analytical results [3]. Pre-flight: Design and fabrication of the spacecraft. Like many modern small sample return missions, Genesis was a tightly managed team integrated across science, engineering and curation. Communication across the team was excellent, and, for the most part, the hands-on engineering technicians understood the impacts of “small choices” they routinely make, and the eyes-on oversight of manufacturing processes by scientists was mindful of details. The payload was designed by the Jet Propulsion Laboratory and the spacecraft by Lockheed Martin. Solar wind collectors and instruments were fabricated by multiple vendors and laboratories. The main portion of the payload was assembled at JSC. Fabrication procedures and contamination-control data (with witness coupons) were stored primarily at JSC. The original composition, dimensions and configuration of components, results of thermal testing, etc. are still needed for interpretation of analytical data. At times, these must be estimated from secondary information acquired pre-flight. Moreover, some files (e.g., original 3-D models and early Powerpoint) cannot be opened using software. Archived curation data includes 2-D drawings, material usage lists, QA documentation and analyses of consumables used during fabrication. Important chemical information still resides in archived hardware, paints and lubricants, material coupons, cleaning coupons, environmental witness plates and reference materials from manufacturing facilities. Purity and cleanliness of collector substrates. Semi-conductor vendors provided surface cleanliness data and some purity data. Purity for specific elements of interest was verified by science team members in their laboratories [4]. Curation archived procurement and shipping records, analysis reports, and non-proprietary fabrication data. A physical archive of flight collector reference materials is maintained for future use so additional data can be collected as analytical techniques improve. These are of increased value due to the hard landing upon re-entry. Cleaning and cleanliness assessments of flight hardware. Cleaning of the science canister payload was performed at JSC in a dedicated ISO 4 cleanroom using ultrapure water (UPW). The cleanliness of this UPW was monitored throughout processing. The archive for the clean lab also includes airborne particle counts, airborne molecular and inorganic contamination measurements as well as cleanroom construction material coupons and witness coupons. Hardware cleanliness was assessed by particle counts in rinse water batches. This information is recorded in batch cleaning forms and logbooks, and are, perhaps, of decreased value due to the hard landing. Post-flight: Curation-generated data. The curation handling history of each Genesis sample is documented in a typical astromaterials sample database which captures sample location, physical description and characterization data. Samples have a “shelf life”. Crucial to the preservation of samples is ongoing documentation of the sample environment, initially under curatorial control but is now a separate facility function with requires coordination. PI-generated data. Data on sample characterization and cleaning techniques continues to be generated by sample users [5]. These are often captured in LPSC abstracts, but these “engineering” results often are not publishable as stand-alone papers. We are actively looking for ways to make this information more accessible to users. Ion implants into samples have aided science return and can be shared among investigators. These (and similar) materials should be added to the curatorial collection with appropriate process and characterization data generated externally. Summary: Complete data archives for returned astromaterial samples must be broad in types and formats, and inclusive of environmental monitoring.

Genesis↗

Locally adaptive vector quantization: Data compression with feature preservation

A study of a locally adaptive vector quantization (LAVQ) algorithm for data compression is presented. This algorithm provides high-speed one-pass compression and is fully adaptable to any data source and does not require a priori knowledge of the source statistics. Therefore, LAVQ is a universal data compression algorithm. The basic algorithm and several modifications to improve performance are discussed. These modifications are nonlinear quantization, coarse quantization of the codebook, and lossless compression of the output. Performance of LAVQ on various images using irreversible (lossy) coding is comparable to that of the Linde-Buzo-Gray algorithm, but LAVQ has a much higher speed; thus this algorithm has potential for real-time video compression. Unlike most other image compression algorithms, LAVQ preserves fine detail in images. LAVQ's performance as a lossless data compression algorithm is comparable to that of Lempel-Ziv-based algorithms, but LAVQ uses far less memory during the coding process.

Cheung, K. M.↗

Fiber-Optic Sensing for Earthquake Hazards Research, Monitoring, and Early Warning

The use of fiber‐optic sensing systems in seismology has exploded in the past decade. Despite an ever‐growing library of ground‐breaking studies, questions remain about the potential of fiber‐optic sensing technologies as tools for advancing if not revolutionizing earthquake‐hazards‐related research, monitoring, and early warning systems. A working group convened to explore these topics; we comprehensively examined the application of fiber optics in various aspects of earthquake hazards, encompassing earthquake source processes, crustal imaging, data archiving, and technological challenges. There is great potential for fiber‐optic systems to advance earthquake monitoring and understanding, but to fully unlock their capabilities requires continued progress in key areas of research and development, including instrument testing and validation, increased dynamic range for applications focused on larger earthquakes, and continued improvement in subsurface and source imaging methods. A key current stumbling block results from the lack of clear data archiving requirements, and we propose an initial strategy that balances data volume requirements with preserving key data for a broad range of future studies. In addition, we demonstrate the potential for fiber‐optic sensing to impact monitoring efforts by documenting the data completeness in a number of long‐term experiments. Finally, we outline the features of a instrument testing facility that would enable progress toward reliable and standardized distributed acoustic sensing data. Overcoming these current obstacles would facilitate progress in fiber‐optic sensing and unlock its potential application to a broad range of earthquake hazard problems.

58 GEOSCIENCES↗

Long-Term Preservation of NASA Heliophysics Data and Access: Where We Were and Where We're Going

The importance of ensuring preservation and useful access to the unique science potential of past, present and future NASA solar and space physics (i.e. heliophysics) data has been recognized since the inception of NASA but remains challenging. In this talk, I will briefly review the history of this topic and and then discuss the present NASA model for heliophysics science data management, including key current resources for finding and using data projects like the Space Physics Data Facility. I will highlight expected future directions, building on working elements of the present program and exploiting new technology, to further improve the data environment, address existing issues and anticipate emerging challenges.

McGuire, Robert E.↗

Preserving NASA Historic and Current Mission Data and Adding Value to These for Future Researchers

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has been actively involved in many aspects of ensuring the long-term preservation of NASA earth science data and knowledge. This involves both the recovery and preservation of early NASA meteorological and other earth observation data, as well as preserving the more recent Earth Observation System (EOS) mission data sets which continue or have reached their end of lifetime. The GES DISC adds value to these preserved data by adding metadata and making the data available online to future researchers. The early NASA meteorological and earth observation data sets from the 1960s and 70s were originally archived on magnetic tapes, and visualizations of these data were preserved on 70-mm film. As these media have aged, their contents have been at risk of permanent loss. NASA has given the task of preserving these early data sets to the GES DISC and making these data sets easily available to the public. The data from these early missions are potentially useful to climate researchers as these are some of the only global measurements made at their time. These old data on magnetic tapes and film strips do not contain easily readable metadata, and so to add value the GES DISC has added digital metadata to them so that the data are searchable and findable. The GES DISC is also involved in preserving the data and knowledge from the EOS era missions. The GES DISC follows the guidelines developed for the preservation of data as specified in the NASA EOS Data and Information System (EOSDIS) Earth Science Data Preservation Content Specification (423-SPEC-001) document. To date, the GES DISC has consulted with the data science teams from the following missions: UARS, Earth Probe TOMS, Aura HIRDLS, and SORCE, in order to properly preserve their data and accompanying documentation. The GES DISC is also currently working with the EOS science teams from TRMM, AIRS, MLS, OMI and additional missions to ensure that the relevant documents and data sets are properly archived for future researchers. A standardized procedure for mission data preservation following 423-SPEC-001 makes preservation among the many NASA EOSDIS data centers uniform, so that these could be transitioned easily to a common EOSDIS preservation repository. This presentation will give an overview of the preservation and recovery of the old NASA historical data sets archived at the GES DISC, as well as the data and documentation preservation efforts of the EOS era missions.

James Johnson↗

Data-driven, structure-preserving approximations to entropy-based moment closures for kinetic equations

In this study, we present a data-driven approach for approximating entropy-based closures of moment systems from kinetic equations. The proposed closure learns the entropy function by fitting the map between the moments and the entropy of the moment system, and thus does not depend on the spacetime discretization of the moment system or specific problem configurations such as initial and boundary conditions. With convex and C 2 approximations, this data-driven closure inherits several structural properties from entropy-based closures, such as entropy dissipation, hyperbolicity, and H-Theorem. We construct convex approximations to the Maxwell–Boltzmann entropy using convex splines and neural networks, test them on the plane source benchmark problem for linear transport in slab geometry, and compare the results to the standard, entropy-based systems which solve a convex optimization problem to find the closure. Numerical results indicate that these data-driven closures provide accurate solutions in much less computation time than that required by the optimization routine.

97 MATHEMATICS AND COMPUTING↗

Adaptive continuity-preserving simplification of street networks

Street network data is widely used to study human-based activities and urban structure. Often, these data are geared towards transportation applications, which require highly granular, directed graphs that capture the complex relationships of potential traffic patterns. While this level of network detail is critical for certain fine-grained mobility models, it represents a hindrance for studies concerned with the morphology of the street network. For the latter case, street network simplification — the process of converting a highly granular input network into its most simple morphological form — is a necessary, but highly tedious preprocessing step, especially when conducted manually. In this manuscript, we develop and present a novel adaptive algorithm for simplifying street networks that is both fully automated and able to mimic results obtained through a manual simplification routine. The algorithm — available in the neatnet Python package — outperforms current state-of-the-art procedures when comparing those methods to manually, human-simplified data, while preserving network continuity.

Python↗

Lunar Data Node: Apollo Data Restoration and Archiving Update

The Lunar Data Node (LDN) of the Planetary Data System (PDS) is responsible for the restoration and archiving of Apollo data. The LDN is located at the National Space Science Data Center (NSSDC), which holds much of the extant Apollo data on microfilm, microfiche, hard-copy documents, and magnetic tapes in older formats. The goal of the restoration effort is to convert the data into user-accessible PDS formats, create a full set of explanatory supporting data (metadata), archive the full data sets through PDS, and post the data online at the PDS Geosciences Node. This will both enable easy use of the data by current researchers and ensure that the data and metadata are securely preserved for future use. We are also attempting to locate and preserve Apollo data which were never archived at NSSDC. We will give a progress report on the data sets we have been restoring and future work.

Williams, David R.↗

Evaluation of Multi-Fidelity Soil Moisture Products Across the Continental United States

We have aggregated the most recent soil moisture datasets from a diverse range of sources, encompassing the Continental United States (CONUS). These sources encompass gridded data from remote sensing products, reanalysis products, machine learning-based projects, and land surface modeling products. Additionally, we have obtained and processed in-situ soil moisture observations from the International Soil Moisture Network. The collected datasets exhibit variations in both temporal and spatial resolutions. Among the 20 datasets, six are available at a spatial resolution of 0.25 degrees, while three are at a coarser spatial resolution of 25 km. To minimize spatial interpolation, we conducted data uncertainty evaluations at the 0.25-degree spatial resolution. For our data evaluations, we maintained a monthly temporal resolution, which effectively captures soil moisture seasonality and interannual variability. Our data processing strategy preserves the raw data and interpolated data at their original temporal resolutions. Datasets with higher temporal resolutions, including daily, three-hourly, and hourly datasets, are set aside for subsequent analyses. These analyses will delve into topics such as soil moisture changes and recovery during extreme weather events. Furthermore, we have processed auxiliary data to enhance our evaluation, leveraging tools such as Google Earth Engine. This includes incorporating topography data, land use land cover data, Köppen-Geiger climate classification, and more to provide a comprehensive assessment from multiple sources.

Li, Lingcheng↗

Tiling as a Durable Abstraction for Parallelism and Data Locality

Tiling is a useful loop transformation for expressing parallelism and data locality. Automated tiling transformations that preserve data-locality are increasingly important due to hardware trends towards massive parallelism and the increasing costs of data movement relative to the cost of computing. We propose TiDA as a durable tiling abstraction that centralizes parameterized tiling information within array data types with minimal changes to the source code. The data layout information can be used by the compiler and runtime to automatically manage parallelism, optimize data locality, and schedule tasks intelligently. In this study, we present the design features and early interface of TiDA along with some preliminary results.

Unat, Didem↗

Standards and Best Practices - Two NASA Examples

Formal international standards as well as promotion of community or recommended practices have their place in ensuring "FAIRness" of data. Data management in NASA's Earth Observation System Data and Information System (EOSDIS) has benefited from both of these avenues to a significant extent. The purpose of this paper is to present one example of each of these, which promote (re)usability. The first is an ISO standard for specifying preservation content from Earth observation missions. The work on this started in 2011, informally within the Earth Science Information Partners (ESIP) in the US, while the European Space Agency (ESA) was leading an effort on Long-Term Data Preservation (LTDP). Resulting from the ESIP discussions was NASA's Preservation Content Specification, which was applied in 2012 as a requirement for NASA's new missions. ESA's Preserved Data Set Content (PDSC) document was codified into a document adopted by the Committee on Earth Observation Satellites (CEOS). It was recognized that it would be useful to combine PCS and PDSC into an ISO standard to ensure consistency in data preservation on a broader international scale. This standard, numbered ISO 19165-2 has been under development since mid-2017. The second is an example of developing recommendations for "best practices" within more limited (still fairly broad) communities. A Data Product Developers' Guide (DPDG) is currently being developed by one of NASA's Earth Science Data System Working Groups (ESDSWGs). It is for use by developers of products to be derived from Earth observation data to improve product (re)usability. One of the challenges in developing the guide is the fact that there are already many applicable standards and guides. The relevant information needs to be selected and expressed in a succinct manner, with appropriate pointers to references. The DPDG aims to compile the most applicable parts of earlier guides into a single document outlining the typical development process for Earth Science data products. Standards and best practices formally endorsed by the Earth Science Data and Information System (ESDIS) Standards Office (ESO), outputs from ESDSWGs (e.g., Dataset Interoperability Working Group, and Data Quality Working Group), and recommendations from Distributed Active Archive Centers and data producers are emphasized.

Ramapriyan, Hampapuram↗

Finding, Organizing, and Preserving Legacy Nuclear Test Monitoring Data—Examples from the Livermore Nevada Network

A challenge for geophysical research is the relatively short time for which continuous digital data have been available. There are many events of research interest where much of the data were collected by analog or temporary digital means. These legacy data require special efforts to find, organize, and preserve for future research. Here, we focus on one type of legacy data, nuclear explosions, and associated events (e.g., collapses, aftershocks) of high interest for nuclear monitoring research and development. For nuclear tests conducted in Nevada, the Lawrence Livermore Nevada Network recorded valuable data on a variety of formats. In this article, we briefly review this network, and describe one set of data that previously was available via CD-ROM by request, and since 2018 has become available through Incorporated Research Institutions for Seismology as assembled dataset 18-001.

58 GEOSCIENCES↗

Catalyzing deep decarbonization with federated battery diagnosis and prognosis for better data management in energy storage systems

Industrial data analytics methods play a central role in improving energy storage performance and efficiency, impacting the future of electrified transportation and renewable electricity generation. However, significant challenges hinder the large-scale deployment of batteries. Conventional methods rely on centralized collection and processing of fleet-level data, leading to database size issues and privacy concerns due to potential data breaches. To enable scalable deployment of battery management systems, this article proposes a federated battery diagnosis and prognosis model, which distributes the processing of battery standard current-voltage-time-usage data in a privacy-preserving manner. Instead of transferring the raw data, this approach communicates only the locally processed parameters, thus reducing communication load and preserving data confidentiality. The federated model offers a paradigm shift in battery health management through privacy-preserving distributed methods for battery data processing and lifetime prediction, ensuring the reliable and sustainable deployment of lithium-ion batteries in a rapidly evolving world.

asset health management↗