Search NASA⌕ Search

SEARCH · Search NASA

Results for “Dataset Curation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

An Update on the Geothermal Data Repository's Data Standards and Pipelines: Geospatial Data and Distributed Acoustic Sensing Data

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has implemented data standards and automated data pipelines for the following data types: 1) drilling data, 2) geospatial datasets, and 3) DAS data. An additional data pipeline is proposed for stimulation data. These data standards and pipelines are intended to improve the real-world applicability of geothermal machine learning outputs through improving the quality of data. More specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, allowing more time to be spent on actual research. By automating this process, the burden of standardization is taken off of the user, overall increasing the availability of standardized data. This paper provides an update on the GDR's transition toward data standardization through automated data pipelines and calls for feedback from the community on how we can improve this process.

cloud-optimized↗
DOE OSTI2023

An Update on the Geothermal Data Repository's Data Standards and Pipelines: Geospatial Data and Distributed Acoustic Sensing Data

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has implemented or is currently implementing data standards and automated data pipelines for the following geothermal data types: 1) drilling data, 2) geospatial datasets, and 3) Distributed Acoustic Sensing (DAS) data. These data standards and pipelines are intended to improve the real-world applicability of geothermal machine learning outputs through improving the quality of data. More specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, allowing more time to be spent on actual research. By automating this process, the burden of standardization is taken off of the user, overall increasing the availability of standardized data. This paper provides an update on the GDR's transition toward data standardization through automated data pipelines and calls for feedback from the community on how the GDR team can improve this process.

cloud-optimized↗
DOE OSTI2024

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository: Preprint

The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and aiding users in accessing data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data.

accessibility↗
DOE OSTI2025

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository

The Department of Energy's (DOE's) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and helping users access data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data. This paper provides an update on recent improvements made to the GDR's data lakes and automated data pipelines, including: (1) streamlining the data lake intake process, (2) better educating users on the process and requirements through a new data lakes page, (3) adding data lake direct access links to GDR data lake submission pages, (4) implementing a DAS data pipeline to convert DAS data uploaded in SEG-Y format to a standardized hierarchical data format v5 (HDF5), (5) extending this pipeline to encompass data in the GDR data lake, (6) adding metadata requirements for geospatial data, (7) making user interface/user experience (UX) enhancements to the data pipelines' documentation pages, and (8) improving the GDR's data standards and pipelines pages to better guide users in ensuring that their data is standardized by the GDR's automated data pipelines. 2024 Geothermal Resources Council. All rights reserved.

accessibility↗
DOE OSTI2024

From RNNs to Foundation Models: An Empirical Study on Commercial Building Energy Consumption

Accurate short-term energy consumption forecasting for commercial buildings is crucial for smart grid operations. While smart meters and deep learning models enable forecasting using past data from multiple buildings, data heterogeneity from diverse buildings can reduce model performance. The impact of increasing dataset heterogeneity in time series forecasting, while keeping size and model constant, is understudied. We tackle this issue using the ComStock dataset, which provides synthetic energy consumption data for U.S. commercial buildings. Two curated subsets, identical in size and region but differing in building type diversity, are used to assess the performance of various time series forecasting models, including finetuned open-source foundation models (FMs). The results show that dataset heterogeneity and model architecture have a greater impact on post-training forecasting performance than the parameter count. Moreover, despite the higher computational cost, finetuned FMs demonstrate competitive performance compared to base models trained from scratch.

commercial buildings↗
DOE OSTI2024

Organ-delimited gene regulatory networks provide high accuracy in candidate transcription factor selection across diverse processes

Organ-specific gene expression datasets that include hundreds to thousands of experiments allow the reconstruction of organ-level gene regulatory networks (GRNs). However, creating such datasets is greatly hampered by the requirements of extensive and tedious manual curation. Here, we trained a supervised classification model that can accurately classify the organ-of-origin for a plant transcriptome. This K-Nearest Neighbor-based multiclass classifier was used to create organ-specific gene expression datasets for the leaf, root, shoot, flower, and seed in Arabidopsis thaliana . A GRN inference approach was used to determine the: i. influential transcription factors (TFs) in each organ and, ii. most influential TFs for specific biological processes in that organ. These genome-wide, organ-delimited GRNs (OD-GRNs), recalled many known regulators of organ development and processes operating in those organs. Importantly, many previously unknown TF regulators were uncovered as potential regulators of these processes. As a proof-of-concept, we focused on experimentally validating the predicted TF regulators of lipid biosynthesis in seeds, an important food and biofuel trait. Of the top 20 predicted TFs, eight are known regulators of seed oil content, e.g., WRI1, LEC1, FUS3. Importantly, we validated our prediction of MybS2, TGA4, SPL12, AGL18, and DiV2 as regulators of seed lipid biosynthesis. We elucidated the molecular mechanism of MybS2 and show that it induces purple acid phosphatase family genes and lipid synthesis genes to enhance seed lipid content. This general approach has the potential to be extended to any species with sufficiently large gene expression datasets to find unique regulators of any trait-of-interest.

09 BIOMASS FUELS↗
NASA NTRSDate not supplied

Beyond Fair: Engagement, Data Usability, and Open Community Productivity through the NASA Open Science Data Repository

The FAIR principle (findable, accessible, interoperable, and reusable) governs the storage and sharing of NASA space biology and health data[1]. These guiding principles maximize reuse of data and the reproducibility of scientific findings. The NASA Open Science Data Repository (OSDR; an expansion of NASA GeneLab) was built on the FAIR principles and houses over 500 studies and close to 1000 datasets from decades of space life sciences experiments. OSDR embodies the FAIR principles through data governance that includes mediated, embargoed, and fully open access data. The FAIR data governance principles were recently proposed to be expanded to encompass a FAIREST framework for assessing research data repositories (FAIR + Engagement, Social connections, and Trust)[2]. FAIREST emphasizes the importance of data repositories engaging with the scientific community and gaining the trust of researchers regarding data quality. Trust also refers to the TRUST principles developed for assessment of digital repositories: Transparency, Responsibility, User Focus, Sustainability, Technology[3]. We present the “Open Science for Life in Space” Analysis Working Groups (AWGs) as evidence regarding the power of engagement, social connections, and trust which has enhanced OSDR’s capabilities and productivity. AWG members engage in two main activities. One, members provide feedback on OSDR scientific standards for data ingestion, curation, and reuse (study, subject and assay metadata; processing pipelines; dataset formats and uniformed structures for machine-readability). Two, AWG members collaborate to mine-reuse OSDR data to conduct scientific analysis. With nearly 800 active members, the AWGs have resulted in 32 publications re-using OSDR data and contributed many papers in two major special issues in Cell (2020) and Nature (2024). AWGs also serve as networking groups, facilitate social connections between researchers at all levels of experience, and also have a social online ‘Forum’ used to keep members informed on projects and opportunities. This community-centric, productive, and trustworthy data culture has resulted in a broader effect with international space agencies, academics, and the commercial space sector wanting to submit their data to OSDR. Ten studies of Inspiration 4 data were recently publicly released by OSDR, as were some JAXA human data. Coming up soon in OSDR are data submissions from the European Space Agency, Virgin Galactic PIs, and SpaceX Polaris Dawn. A major benefit of OSDR is the array of standardized and uniformly formatted data (which was developed through AWG member consensus), from which visualization tools, analysis tools, and machine learning models can be built or trained. This talk will cover the Multi-Study Visualization Tool, the Environmental Data Application, RadLab, and a UCSF-NSF funded knowledge graph biomedical health discovery tool ‘SPOKE’ currently being integrated with OSDR. OSDR also provides training programs in bioinformatics and machine learning to improve the scientific community’s awareness of data availability and to boost their ability to perform data analysis. The increasing engagement of the scientific community and the public with technologies powered by artificial intelligence (AI) heightens the need for data analysis to be transparent. The AI for Life in Space initiative leverages the data products provided in OSDR to train AI models, with an emphasis on explainable and trustworthy AI, which would not be possible without FAIR data and metadata. Overall, here we will demonstrate the importance for NASA life sciences data repositories to adhere to the FAIREST framework, by providing examples and success stories from different aspects of OSDR.

data↗
DOE OSTI2023

Gulf of Mexico Risk Analysis Database (GoMRAD)

The Gulf of Mexico Risk Analysis Database is comprehensive Esri geodatabase of vector layers, raster layers, and tables curated for risk analysis within the offshore Gulf of Mexico. Datasets include bathymetry, seafloor characteristics (channels, anomalies, faults, etc.), MetOcean data (wind speed, wave height, etc.), ocean current data, sediment data, and machine learning training regions used in NETL's Ocean & Geohazard Analysis (OGA) tool. This database serves as a compliment to the OGA tool by providing many of the datasets used in the design of the OGA tool, including regions used for machine learning. This database also serves as a valuable resource for risk analysis studies within the offshore Gulf of Mexico. This work was completed under the Advanced Offshore Research Portfolio, FWP Number: 1022476.

BOEM,Bathymetry,Gulf Of Mexico,Machine Learning,Me↗
DOE OSTI2026

RTN-124: Photometric Redshifts for the Vera C. Rubin Observatory Data Preview

We present the photometric redshifts (photo-z) inferred using algorithms implemented in the Redshift Assessment Infrastructure Layers (RAIL) for the NSF-DOE Vera C. Rubin Observatory Data Preview 2 (DP2). We produce a compilation of reference redshift catalog using spectroscopic, grism and many band photometric redshift dataset hosted on the LIneA Photo-z Server. We curate training and testing set for assessing the scientific and technical performance of Rubin photo-z. The algorithm applied to the object catalog are FlexZBoost, BPZ, kNN, GPz, DNF and TPz; with a combination of 6-band and 4-band photo-z depending on availability of u and y photometry. The redshift point estimates and uncertainty estimation in tabular format through the Large Survey DataBase (LSDB).

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2019

Cross Kingdom Analysis of Data Within the GeneLab Repository Identifies a Potential Conserved Response of Life to the Stress Associated with Spaceflight

It is important to determine the health risks and potential survival for astronauts associated with long-term space missions. This entails not only understanding the impact the space environment will have on humans, but also how it will affect other organisms needed for humans to survive in space such as plants. In addition, it has been reported in the literature that hundreds of genes seem to be conserved and/or transferred between different organisms from bacteria, archaea, fungi, microorganisms, and plants to animals. Since space travel involves humans in a closed environment over a long period of time, we hypothesize that potential conserved biological factors will occur between the different organisms in that environment possibly due to transfer of genes. Determining the conserved factors that are commonly being regulated in space can shed insight into possible universal master regulators and also determine the symbiotic relationship between the organisms in space. Utilizing NASA's GeneLab Data Repository (a rapidly expanding, curated clustering of spaceflight-related ‘omics-level datasets for all organisms), we were able to uncover a novel pathway and factors that were commonly shared between humans, mice, plants, C. Elegans, and drosophilas. Through ChIP-Seq enrichment analysis techniques utilizing various GeneLab datasets from each species that were flown in space, we found the following factors to be conserved across all species: oxidative stress, DNA damage (through GABPA/NRFs and NFY), SIX5, GTF2B and glutamine synthetase. Such commonalities would likely reflect the effects of factors such as microgravity and the increased radiation exposure inherent in spaceflight on basic physical processes shared by all biological systems at the cellular level. Differences between organismal responses revealed by GeneLab's data should also help understand the unique reactions to life in space that arise from the very different lifestyles of microbes, animals and plants.

Barker, Richard↗
NASA NTRSDate not supplied

NASA GeneLab Platform Utilized for Biological Response to Space Radiation in Animal Models

Ionizing radiation from Galactic Cosmic Rays (GCR) is one of the major risk factors that will impact the health of astronauts on extended missions outside the protective effects of Earth’s magnetic field. The NASA GeneLab project has detailed information on radiation exposure using animal models with curated dosimetry information for spaceflight experiments. We analyzed multiple GeneLab omics datasets associated with both ground-based and spaceflight radiation studies that included in vivo and in vitro approaches. A range of ions from protons to iron particles with doses from 0.1 Gy to 1.0 Gy for ground studies and samples flown in Low Earth Orbit (LEO) with total doses of 1.0 mGy to 30 mGy were utilized From this analysis we were able to identify distinct biological signatures associating specific ions with specific biological responses due to radiation exposure in space. For example, we discovered changes in mitochondrial function, ribosomal assembly, and immune pathways as a function of dose. We provided a summary of how the GeneLab’s rich database of omics experiments with animal models can be used to generate novel hypotheses to better understand human health risks from GCR exposures.

Afshin Beheshti↗
NASA NTRSDate not supplied

NASA GeneLab Platform Utilized for Space Radiation Dosimetry Biological Response Compared to Radiation Ground Studies

Ionizing radiation from Galactic Cosmic Rays (GCR) is one of the major risk factors that will impact the health of astronauts on extended missions outside the protective effects of Earth’s magnetic field. The NASA GeneLab project has detailed information on radiation exposure using animal models with curated dosimetry information for spaceflight experiments. We analyzed multiple GeneLab omics datasets associated with both ground-based and spaceflight radiation studies that included in vivo and in vitro approaches. A range of ions from protons to iron particles with doses from 0.1 Gy to 1.0 Gy for ground studies and samples flown in Low Earth Orbit (LEO) with total doses of 1.0 mGy to 30 mGy were utilized From this analysis we were able to identify distinct biological signatures associating specific ions with specific biological responses due to radiation exposure in space. For example, we discovered changes in mitochondrial function, ribosomal assembly, and immune pathways as a function of dose. We provided a summary of how the GeneLab’s rich database of omics experiments with animal models can be used to generate novel hypotheses to better understand human health risks from GCR exposures.

Afshin Beheshti↗
DOE OSTI2026

Arm and shoulder muscle segmentation in axial MRI with UNet deep learning model

Quantifying individual upper-limb muscle volumes from MRI provides key insight into muscle-specific strength, deficits, and adaptations. Manual delineation is the gold standard but time‑intensive, and the performance of current deep learning approaches, particularly for small or anatomically complex muscles, remains incompletely characterized. We evaluated a state‑of‑the‑art deep learning framework across the entire upper limb and analyzed factors governing segmentation performance, with attention to the forearm. Three previously published MRI datasets (1.5 T, 3D GRE T1‑weighted; total n = 39) spanning young, middle‑aged, and older adults were curated and quality‑checked, including expert manual segmentations for 31 muscles. Following multiclass mask reconstruction, we trained three 3D nnU‑Net multiclass models matched to the muscle subsets present across datasets, using five‑fold cross‑validation and a composite Dice Similarity Coefficient (DSC) + cross entropy loss. Segmentation accuracy was assessed with DSC. Performance varied across muscles (mean DSC = 0.806 ± 0.098), ranging from 0.920 (Deltoid) to 0.461 (Extensor pollicis brevis). In uncertainty‑weighted regressions, muscle volume was positively associated with DSC (R2 = 0.36, p < 0.001), whereas training segmentation count and muscle orientation showed negligible associations (R2 ≤ 0.06). A weighted mixed‑effects model identified volume as the strongest evaluated predictor, explaining 23.9% of variance in DSC; orientation and training count each contributed <1%, leaving 61.5% unexplained. These results indicate that deep learning–based segmentation can accurately quantify muscle volume for many upper‑limb muscles but remains constrained for small, low‑contrast forearm muscles.

Gillespie, Samuel↗
NASA NTRS2023

GLORIA - A Globally Representative Hyperspectral In Situ Dataset for Optical Sensing of Water Quality

The development of algorithms for remote sensing of water quality (RSWQ) requires a large amount of in situ data to account for the bio-geo-optical diversity of inland and coastal waters. The GLObal Reflectance community dataset for Imaging and optical sensing of Aquatic environments (GLORIA) includes 7,572 curated hyperspectral remote sensing reflectance measurements at 1 nm intervals within the 350 to 900 nm wavelength range. In addition, at least one co-located water quality measurement of chlorophyll a , total suspended solids, absorption by dissolved substances, and Secchi depth, is provided. The data were contributed by researchers affiliated with 59 institutions worldwide and come from 450 different water bodies, making GLORIA the de-facto state of knowledge of in situ coastal and inland aquatic optical diversity. Each measurement is documented with comprehensive methodological details, allowing users to evaluate fitness-for-purpose, and providing a reference for practitioners planning similar measurements. We provide open and free access to this dataset with the goal of enabling scientific and technological advancement towards operational regional and global RSWQ monitoring.

remote sensing of water quality↗
DOE OSTI2021

Call to Action for Global Access to and Harmonization of Quality Information of Individual Earth Science Datasets

Knowledge about the quality of data and metadata is important to support informed decisions on the (re)use of individual datasets and is an essential part of the ecosystem that supports open science. Quality assessments reflect the reliability and usability of data. They need to be consistently curated, fully traceable, and adequately documented, as these are crucial for sound decision- and policy-making efforts that rely on data. Quality assessments also need to be consistently represented and readily integrated across systems and tools to allow for improved sharing of information on quality at the dataset level for individual quality attribute or dimension. Although the need for assessing the quality of data and associated information is well recognized, methodologies for an evaluation framework and presentation of resultant quality information to end users may not have been comprehensively addressed within and across disciplines. Global interdisciplinary domain experts have come together to systematically explore needs, challenges and impacts of consistently curating and representing quality information through the entire lifecycle of a dataset. This paper describes the findings of that effort, argues the importance of sharing dataset quality information, calls for community action to develop practical guidelines, and outlines community recommendations for developing such guidelines. Practical guidelines will allow for global access to and harmonization of quality information at the level of individual Earth science datasets, which in turn will support open science.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2024

Mapping use cases and dataset needs for benchmarking buildings data

A perennial challenge in buildings research is the lack of high-quality datasets that can be relied upon for a wide array of tasks, including model calibration and improving energy efficiency and load flexibility. Instrumenting a building for data collection is resource intensive, so it is important to be methodical in the approach and ensure that resulting data are flexible and useful for a broad range of analyses. This study aims to fill the gaps in characterizing potential use cases for buildings datasets and mapping them to dataset needs using a well-defined data infrastructure. Here, we have developed a systematic mapping strategy between buildings dataset needs and use cases to help streamline the processes of efficiently targeting datasets, designing building sensing systems, and determining buildings research use cases. We selected 14 prospective use cases and 11 refined buildings data categories for developing the preliminary dataset-needs-to-use-cases mapping matrix (‘DN-UC mapping matrix’) with generic ‘Tags’—a detailed sub-level of data categories extracted by justifying the needs of an aspect of the datasets to use cases. We present two example applications of the developed mapping matrix to demonstrate use of the mapping matrix and its effectiveness.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Comprehensive analysis of common polymers using hyphenated TGA-FTIR-GC/MS and Raman spectroscopy towards a database for micro- and nanoplastics identification, characterization, and quantitation

Environmental contamination by micro- and nanoplastics (MNPs) is well documented with potential for their increased accumulation globally. Growing public concern over environmental, ecological, and human exposure to MNPs has led to exponential increase in publications, news articles, and reports. Significant knowledge gap exists in standardized analytical methods for the identification and quantification of MNPs from real world environmental samples. Here, in this study, we report comprehensive datasets utilizing thermogravimetric analyzer (TGA) coupled to a Fourier transformed infrared spectrometer (FTIR) and a gas chromatography/mass spectrometer (GC/MS) with corresponding Raman spectral data for the most common polymers documented to be present in the environment (35 plastics of 12 polymer types), to serve as a base line reference for the identification and quantitation of MNPs. Various parameters for TGA-FTIR-GC/MS data acquisition were optimized. Commercial consumer plastic product compositions were identified using this analytical database. Case studies to showcase the utility of the method for polymer mixtures analysis is included. This dataset would serve towards the development of a collaborative, global, comprehensive, and curated public database for the identification of various MNPs and mixtures.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2024

Electricity and natural gas tariffs at United States wastewater treatment plants

Abstract Wastewater treatment plants (WWTPs) are large electricity and natural gas consumers with untapped potential to recover carbon-neutral biogas and provide energy services for the grid. Techno-economic analysis of emerging energy recovery and management technologies is critical to understanding their commercial viability, but quantifying their energy cost savings potential is stymied by a lack of well curated, nationally representative electricity and natural gas tariff data. We present a dataset of electricity tariffs for the 100 largest WWTPs in the Clean Watershed Needs Survey (CWNS) and natural gas tariffs for the 54 of 100 WWTPs with on-site cogeneration. We manually collected tariffs from each utility’s website and implemented data checks to ensure their validity. The dataset includes facility metadata, electricity tariffs, and natural gas tariffs (where cogeneration is present). Tariffs are current as of November 2021. We provide code for technical validation along with a sample simulation.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗