Search NASA⌕ Search

SEARCH · Search NASA

Results for “FAIR dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Making Dataset Quality Information FAIR: Supporting Open-Source Science and Enhancing (Re)Use and Trustworthiness of Scientific Data

- Quality information should be documented and readily shared within and across domains. - Sharing of dataset quality information supports open science and trustworthiness of scientific data. - Dataset quality is more than data quality. - Quality tends to be domain-specific and context-dependent. - Community guidelines provide practical steps towards FAIR dataset quality information.

Ge Peng↗
DOE OSTI2024

Riverine dissolved organic matter transformations increase with watershed area, water residence time, and Damköhler numbers in nested watersheds

Abstract Quantifying the relative influence of factors and processes controlling riverine ecosystem function is essential to predicting future conditions under global change. Dissolved organic matter (DOM) is a fundamental component of riverine ecosystems that fuels microbial food webs, influences nutrient and light availability, and represents a significant carbon flux globally. The heterogeneous nature of DOM molecular composition and its propensity for interaction (i.e., functional diversity) can characterize riverine ecosystem function across spatiotemporal scales. To investigate fundamental drivers of DOM diversity, we collected seasonal water samples from 42 nested locations within five watersheds spanning multiple watershed sizes (~ 5 to 30,000 km 2 ) across the United States. Patterns in DOM molecular richness, aromaticity, relative abundance of N-containing formulas, and putative biochemical transformations derived from high-resolution mass spectrometry were assessed across gradients of explanatory variables associated with watershed characteristics (e.g., watershed area, water residence time, land cover). We found that putative biochemical transformations were more strongly related to explanatory variables across watersheds than common bulk DOM parameters and that watershed area, surface water residence time and derived Damköhler numbers representing DOM reactivity timescales were strong predictors of DOM diversity. The data also indicate that catchment-specific land cover factors can significantly influence DOM diversity in diverging directions. Overall, the results highlight the importance of considering water residence time and land cover when interpreting longitudinal patterns in DOM chemistry and the continued challenge of identifying generalizable drivers that are transferable across watershed and regional scales for application in Earth system models. This work also introduces a Findable Accessible Interoperable Reusable (FAIR) dataset (> 300 samples) to the community for future syntheses.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Philympics 2021: Prophage Predictions Perplex Programs

Most bacterial genomes contain integrated bacteriophages—prophages—in various states of decay. Many are active and able to excise from the genome and replicate, while others are cryptic prophages, remnants of their former selves. Over the last two decades, many computational tools have been developed to identify the prophage components of bacterial genomes, and it is a particularly active area for the application of machine learning approaches. However, progress is hindered and comparisons thwarted because there are no manually curated bacterial genomes that can be used to test new prophage prediction algorithms. Here, we present a library of gold-standard bacterial genome annotations that include manually curated prophage annotations, and a computational framework to compare the predictions from different algorithms. We use this suite to compare all extant stand-alone prophage prediction algorithms to identify their strengths and weaknesses. We provide a FAIR dataset for prophage identification, and demonstrate the accuracy, precision, recall, and f 1 score from the analysis of seven different algorithms for the prediction of prophages. We discuss caveats and concerns in this analysis and how those concerns may be mitigated.

Roach, Michael J.↗
DOE OSTI2021

ESS-DIVE Reporting Format for Hydrologic Monitoring Data and Metadata

The ESS-DIVE Reporting Format (RF) for Hydrologic Monitoring Data and Metadata was designed for water level, temperature, electrical conductivity, specific conductance, dissolved oxygen, and pH measured in situ using sensors deployed in water bodies. It can also be applied to a broader suite of variables beyond its original scope. The RF is intended to provide clear and easy guidance to data generators aiming to upload consistent findable, accessible, interoperable, and reusable (FAIR) datasets. The RF includes instructions (pdf), a term guide (csv), templates (csv and xlsx), and a list of recommended vocabulary (csv) to equip the data generator with all of the tools needed to successfully apply the RF. There is also a crosswalk to some key data resources and standards (csv). Users should cite this dataset when using the RF but are encouraged to access the RF files on GitHub (https://github.com/ess-dive-community/essdive-hydrologic-monitoring) where the RF documentation is maintained and updated on the ESS-DIVE Community Space GitHub. Questions and suggestions for improvement can be submitted as GitHub issues.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A library of AI-assisted FAIR water cycle and related disturbance datasets to enable model training, parameterization and validation

This whitepaper is responsive to focal area Data acquisition and assimilation enabled by machine learning, AI, and advanced methods. Here we describe how FAIR (Findable, Accessible, Reusable, Interoperable) datasets related to water cycle extremes are essential for successful implementation of ML in Earth System and other models. We also describe how AI can be used to acquire and integrate water cycle data related to extreme events to create a library of FAIR datasets for training and evaluating algorithms.

58 GEOSCIENCES↗
DOE OSTI2021

Ontology-Enriched Specifications Enabling Findable, Accessible, Interoperable, and Reusable Marine Metagenomic Datasets in Cyberinfrastructure Systems

Marine microbial ecology requires the systematic comparison of biogeochemical and sequence data to analyze environmental influences on the distribution and variability of microbial communities. With ever-increasing quantities of metagenomic data, there is a growing need to make datasets Findable, Accessible, Interoperable, and Reusable (FAIR) across diverse ecosystems. FAIR data is essential to developing analytical frameworks that integrate microbiological, genomic, ecological, oceanographic, and computational methods. Although community standards defining the minimal metadata required to accompany sequence data exist, they haven’t been consistently used across projects, precluding interoperability. Moreover, these data are not machine-actionable or discoverable by cyberinfrastructure systems. By making ‘omic and physicochemical datasets FAIR to machine systems, we can enable sequence data discovery and reuse based on machine-readable descriptions of environments or physicochemical gradients. In this work, we developed a novel technical specification for dataset encapsulation for the FAIR reuse of marine metagenomic and physicochemical datasets within cyberinfrastructure systems. This includes using Frictionless Data Packages enriched with terminology from environmental and life-science ontologies to annotate measured variables, their units, and the measurement devices used. This approach was implemented in Planet Microbe, a cyberinfrastructure platform and marine metagenomic web-portal. Here, we discuss the data properties built into the specification to make global ocean datasets FAIR within the Planet Microbe portal. We additionally discuss the selection of, and contributions to marine-science ontologies used within the specification. Finally, we use the system to discover data by which to answer various biological questions about environments, physicochemical gradients, and microbial communities in meta-analyses. This work represents a future direction in marine metagenomic research by proposing a specification for FAIR dataset encapsulation that, if adopted within cyberinfrastructure systems, would automate the discovery, exchange, and re-use of data needed to answer broader reaching questions than originally intended.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

The DECADE cosmic shear project IV: cosmological constraints from 107 million galaxies across 5,400 deg$^2$ of the sky

We present cosmological constraints from the Dark Energy Camera All Data Everywhere (DECADE) cosmic shear analysis. This work uses shape measurements for 107 million galaxies measured through Dark Energy Camera (DECam) imaging of $5,\!412$ deg$^2$ of sky that is outside the Dark Energy Survey (DES) footprint. We derive constraints on the cosmological parameters $S_8 = 0.791^{+0.027}_{-0.032}$ and $Ω_{\rm m} =0.269^{+0.034}_{-0.050}$ for the $Λ$CDM model, which are consistent with those from other weak lensing surveys and from the cosmic microwave background. We combine our results with cosmic shear results from DES Y3 at the likelihood level, since the two datasets span independent areas on the sky. The combined measurements, which cover $\approx\! 10,\!000$ deg$^2$, prefer $S_8 = 0.791 \pm 0.023$ and $Ω_{\rm m} = 0.277^{+0.034}_{-0.046}$ under the $Λ$CDM model. These results are the culmination of a series of rigorous studies that characterize and validate the DECADE dataset and the associated analysis methodologies (Anbajagane et. al 2025a,b,c). Overall, the DECADE project demonstrates that the cosmic shear analysis methods employed in Stage-III weak lensing surveys can provide robust cosmological constraints for fairly inhomogeneous datasets. This opens the possibility of using data that have been previously categorized as ``unusable'' for cosmic shear analyses, thereby increasing the statistical power of upcoming weak lensing surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

The globus compute dataset: An open function-as-a-service dataset from the edge to the cloud

Here we present a unique function-as-a-service (FaaS) dataset capturing the use of the Globus Compute (previously funcX) platform. Globus Compute implements a federated model via which users may deploy endpoints on arbitrary remote computers, from the edge to high performance computing (HPC) cluster, and they may then invoke Python functions on those endpoints via a reliable cloud -hosted service. The dataset covers 31 weeks and includes 2121472 task submissions from 252 users executed on 580 remote computing endpoints. It includes 277386 registered functions. We describe the dataset and various observations, some that are similar to other FaaS datasets, for example, that 74% of tasks run for less than 1 s, and some that are unique to Globus Compute, for example, that endpoints are used in different ways and that the majority of functions are related to scientific computing and machine learning. To the best of our knowledge, this dataset represents the first federated FaaS dataset that includes user workloads, distributed computing endpoints, and analysis of registered function bodies. We expect the dataset to be useful for researching FaaS architectures, workload modeling, container warming, and other distributed computing architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

FAIR Data and Interpretable AI Framework for Architectured Metamaterials (Final Report)

This research program established a transformative framework for the discovery and design of mechanical metamaterials, which are architected structures engineered to control physical phenomena like sound and vibration in ways natural materials cannot. To overcome the traditional reliance on trial-and-error, the project developed an interpretable Artificial Intelligence (AI) framework that moves beyond "black box" models to reveal the specific geometric patterns—such as "unit-cell templates"—that govern a material’s performance. A major breakthrough was the development of a hierarchical design method, which allows a single material to block vibrations across multiple frequency ranges simultaneously by layering patterns at different scales without them interfering with one another. This was further expanded to include irregular, graph-based designs that use spanning tree algorithms to ensure structural connectivity while allowing for customized, direction-dependent properties like stiffness and acoustic impedance. Beyond design, the project addressed the practicalities of real-world production by developing uncertainty quantification techniques that account for manufacturing defects and material variability, reducing the need for expensive physical testing by orders of magnitude. To speed up the discovery process, the team implemented Gaussian Process Regression and other surrogate models that provide accurate performance predictions at a fraction of the traditional computational cost. The AI-generated designs were successfully validated through fabrication of physical samples and wave propagation experiments, confirming their ability to accurately guide or reflect waves as predicted. By contributing these tools and high-quality FAIR benchmark datasets to the wider scientific community, this work provides a scalable foundation for advancing technologies in aerospace vibration control, medical imaging, and noise reduction.

36 MATERIALS SCIENCE↗
DOE OSTI2022

A FAIR and AI-ready Higgs boson decay dataset

Abstract To enable the reusability of massive scientific datasets by humans and machines, researchers aim to adhere to the principles of findability, accessibility, interoperability, and reusability (FAIR) for data and artificial intelligence (AI) models. This article provides a domain-agnostic, step-by-step assessment guide to evaluate whether or not a given dataset meets these principles. We demonstrate how to use this guide to evaluate the FAIRness of an open simulated dataset produced by the CMS Collaboration at the CERN Large Hadron Collider. This dataset consists of Higgs boson decays and quark and gluon background, and is available through the CERN Open Data Portal. We use additional available tools to assess the FAIRness of this dataset, and incorporate feedback from members of the FAIR community to validate our results. This article is accompanied by a Jupyter notebook to visualize and explore this dataset. This study marks the first in a planned series of articles that will guide scientists in the creation of FAIR AI models and datasets in high energy particle physics.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

An open-access database and analysis tool for perovskite solar cells based on the FAIR data principles

Large datasets are now ubiquitous as technology enables higher-throughput experiments, but rarely can a research field truly benefit from the research data generated due to inconsistent formatting, undocumented storage or improper dissemination. Here we extract all the meaningful device data from peer-reviewed papers on metal-halide perovskite solar cells published so far and make them available in a database. We collect data from over 42,400 photovoltaic devices with up to 100 parameters per device. We then develop open-source and accessible procedures to analyse the data, providing examples of insights that can be gleaned from the analysis of a large dataset. The database, graphics and analysis tools are made available to the community and will continue to evolve as an open-source initiative. This approach of extensively capturing the progress of an entire field, including sorting, interactive exploration and graphical representation of the data, will be applicable to many fields in materials science, engineering and biosciences.

14 SOLAR ENERGY↗
DOE OSTI2025

Discriminative versus generative approaches to simulation-based inference

Most of the fundamental, emergent, and phenomenological parameters of particle and nuclear physics are determined through parametric template fits. Simulations are used to populate histograms which are then matched to data. This approach is inherently lossy, since histograms are binned and low-dimensional. Deep learning has enabled unbinned and high-dimensional parameter estimation through neural likelihood(-ratio) estimation. We compare two approaches for neural simulation-based inference (NSBI): one based on discriminative learning (classification) and one based on generative modeling. These two approaches are directly evaluated on the same datasets, with a similar level of hyperparameter optimization in both cases. In addition to a Gaussian dataset, we study NSBI using a Higgs boson dataset from the FAIR Universe Challenge. We find that both the direct likelihood and likelihood ratio estimation are able to effectively extract parameters with reasonable uncertainties. For the numerical examples and within the set of hyperparameters studied, we found that the likelihood ratio method is more accurate and/or precise. Both methods have a significant spread from the network training and would require ensembling or other mitigation strategies in practice.

high energy physics↗
DOE OSTI2022

Community Based Data of Uranium Adsorption onto Quartz

This data upload includes a compilation of experiments for quantifying uranium adsorption onto quartz. The provided .csv file has been compiled from the literature in a findable, accessible, interoperable, reusable (FAIR) data format. This was accomplished using the Lawrence Livermore National Laboratory Surface Complexation Database Converter (SCDC) code written in the R programming language (free licensing available at https://ipo.llnl.gov/technologies/software/llnl-surface-complexation-database-converter-scdc). This constitutes all current data compiled on uranium-quartz interactions in the L-SCIE (LLNL Surface Complexation/Ion Exchange) database (as of 02/28/2022). This data was used to develop surface complexation models that fit the global community dataset (https://doi.org/10.1021/acs.est.1c07109). The FAIR-formatted dataset also enables the implementation of alternative machine-learning approaches that can be explored in the future.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

National-scale remotely sensed lake trophic state from 1984 through 2020

Lake trophic state is a key ecosystem property that integrates a lake’s physical, chemical, and biological processes. Despite the importance of trophic state as a gauge of lake water quality, standardized and machine-readable observations are uncommon. Remote sensing presents an opportunity to detect and analyze lake trophic state with reproducible, robust methods across time and space. We used Landsat surface reflectance data to create the first compendium of annual lake trophic state for 55,662 lakes of at least 10 ha in area throughout the contiguous United States from 1984 through 2020. The dataset was constructed with FAIR data principles (Findable, Accessible, Interoperable, and Reproducible) in mind, where data are publicly available, relational keys from parent datasets are retained, and all data wrangling and modeling routines are scripted for future reuse. Together, this resource offers critical data to address basic and applied research questions about lake water quality at a suite of spatial and temporal scales.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Automated Collection of Scientific Publications Linked to NASA Earth Science Datasets

NASA's Earth Observing System Data and Information System (EOSDIS) began dataset Digital Object Identifier (DOI) registration in 2012. The number of dataset DOIs registered as of January of 2023 exceeds 11,000. As the research community becomes aware of the importance of sharing data through Open Science and optimizing data reuse through Findability, Accessibility, Interoperability, and Reuse (FAIR) data management principles, datasets are increasingly being cited in scientific publications. When datasets are cited explicitly by DOI within published works, automated methods can be developed for collecting these published works from a variety of bibliometric sources. The coverage of the sources varies, so each source can collect citations that are only available within it. Using major citation databases such as Scopus and Web of Science, the Google Scholar search engine, the CrossRef Open Citation Index, and the dataset DOI registry DataCite, we present an automated workflow for dataset citation collection. By harvesting citations automatically, a citation library is created explicitly linking EOSDIS datasets to publications that cite them. Using Zotero, a free and open-source citation manager, we demonstrate how to access and browse this library by the tags indicating bibliometric sources, dataset DOI, and the dataset archive center. We also demonstrate temporary trends in the number of publications harvested from bibliometric sources.

Infometrics↗
DOE OSTI2025

The DECADE cosmic shear project III: validation of analysis pipeline using spatially inhomogeneous data

We present the pipeline for the cosmic shear analysis of the Dark Energy Camera All Data Everywhere (DECADE) weak lensing dataset: a catalog consisting of 107 million galaxies observed by the Dark Energy Camera (DECam) in the northern Galactic cap. The catalog derives from a large number of disparate observing programs and is therefore more inhomogeneous across the sky compared to existing lensing surveys. First, we use simulated data-vectors to show the sensitivity of our constraints to different analysis choices in our inference pipeline, including sensitivity to residual systematics. Next we use simulations to validate our covariance modeling for inhomogeneous datasets. Finally, we show that our choices in the end-to-end cosmic shear pipeline are robust against inhomogeneities in the survey, by extracting relative shifts in the cosmology constraints across different subsets of the footprint/catalog and showing they are all consistent within 1σ to 2σ. This is done for forty-six subsets of the data and is carried out in a fully consistent manner: for each subset of the data, we re-derive the photometric redshift estimates, shear calibrations, survey transfer functions, the data vector, measurement covariance, and finally, the cosmological constraints. Our results show that existing analysis methods for weak lensing cosmology can be fairly resilient towards inhomogeneous datasets. This also motivates exploring a wider range of image data for pursuing such cosmological constraints.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Surface Complexation/Ion Exchange Hybrid Model for Radionuclide Sorption to Clay Minerals (M4SF-23LL010301062)

This progress report (Level 4 Milestone Number M4SF-23LL010301062) summarizes research conducted at Lawrence Livermore National Laboratory (LLNL) within the Argillite International Collaborations Activity Number SF-23LL01030106. The activity is focused on our long-term commitment to engaging our partners in international nuclear waste repository research. The focus of this milestone is the establishment of international collaborations for surface complexation modeling and the associated impacts of unlocking larger, community-based datasets. More specifically, we are developing a database framework for Spent Fuel and Waste and Science Technology (SFWST) that is aligned with the Helmholtz Zentrum Dresden Rossendorf (HZDR) sorption database development group in support of the database needs of the SFWST program. In our FY22 effort, we described a detailed analysis of U(VI) sorption to quartz through both traditional surface complexation modeling and through a hybrid ML framework. In FY23, effort was placed on publication of these results and expansion of the LLNL surface complexation and ion exchange database (L-SCIE) in order to assess mineral-based radionuclide retardation under a wider variety of geochemical conditions (e.g., ionic strength, varying electrolyte compositions). Efforts were initiated to expand L-SCIE to include radionuclide surface complexation and ion exchange to clays that are relevant to subsurface geochemical processes occurring at nuclear waste repositories. In particular, a large source of sorption data for clays resides at the Paul Scherrer Institute (PSI) (work primarily by Bradbury and Baeyens) and we initiated discussions on how to retrieve those data and apply FAIR principles to those datasets. In addition to L-SCIE development, two hybrid models that incorporate AI/ML were investigated and compared to discern the most promising approaches for accurate and precise estimations of radionuclide retardation. Key considerations for future model development include (1) the ability to reduce computational burden on determining retardation coefficients for PA and (2) the ability to quantify and predict radionuclide-mineral partitioning at a more efficient, rapid pace due to automated workflows. Upon the careful consideration of the most effective modeling approaches, we are identifying ways to implement these approaches into PA. Ultimately, the data science-based workflows will provide a major incentive for other institutions to adopt a FAIR-formatted, interoperable database. LLNL will play a key role in disseminating sorption data and acting as good data stewards by updating the database in a consistent format and assessing the quality of the newly assimilated data in an organized fashion. To this end, all data and workflows are open access and made available on the LLNL Seaborg research website (https://seaborg.llnl.gov/resources/geochemical-databases-modeling-codes).

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2025

Fair Concurrent Training of Multiple Models in Federated Learning

Federated learning (FL) enables collaborative learning across multiple clients. In most FL work, all clients train a single learning task. However, the recent proliferation of FL applications may increasingly require multiple FL tasks to be trained simultaneously, sharing clients’ computing resources, which we call Multiple-Model Federated Learning (MMFL). Current MMFL algorithms use naïve average-based client-task allocation schemes that often lead to unfair performance when FL tasks have heterogeneous difficulty levels, as the more difficult tasks may need more client participation to train effectively. Furthermore, in the MMFL setting, we face a further challenge that some clients may prefer training specific tasks to others, and may not even be willing to train other tasks, e.g., due to high computational costs, which may exacerbate unfairness in training outcomes across tasks. We address both challenges by firstly designing FedFairMMFL, a difficulty-aware algorithm that dynamically allocates clients to tasks in each training round, based on the tasks’ current performance levels. We provide guarantees on the resulting task fairness and FedFairMMFL’s convergence rate. We then propose novel auction designs that incentivizes clients to train multiple tasks, so as to fairly distribute clients’ training efforts across the tasks, and extend our convergence guarantees to this setting. Here, we finally evaluate our algorithm with multiple sets of learning tasks on real world datasets, showing that our algorithm improves fairness by improving the final model accuracy and convergence speed of the worst performing tasks, while maintaining the average accuracy across tasks.

Federated learning↗