Search NASA⌕ Search

SEARCH · Search NASA

Results for “Open Data Repository”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Panorama 360 (Final Report)

This is the final technical report for the DOE-funded Panorama 360 project. Panorama 360 provided a resource for the collection, analysis, and sharing of performance data about end-to-end scientific workflows executing on DOE facilities. The work focused on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: 1. A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); 2. A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; 3. A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and 4. Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗

Panorama 360 (Final Report)

This final technical report from the lead institution, USC grant #DE-SC0012636, serves as the final technical report for collaborative institution UNC-CH grant #DE-SC0012390. The goal was to develop a repository and associated capabilities for data collection, ingestion, and analysis for a broad class of DOE applications that span experimental and simulation science workflows. In particular, this work focuses on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: (1) A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); (2) A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; (3) A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and (4) Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗

NNFDivergence

The code implements f divergence regularization for neural networks in the Python-based Pytorch framework. The methods are the main focus but the repository will also contain examples that operate on purely synthetic "toy" data or on openly available, public data from NASA.

Klein, Natalie [@lanl]↗

DOE Repository Metadata Profile (DRMP): A Metadata Framework for Advancing Interoperability and AI Readiness Across Scientific Repositories

The Department of Energy (DOE) funds a diverse and distributed ecosystem of repositories that steward scientific data, publications, and software across its research programs, user facilities, and national laboratories. While significant progress has been made in standardizing dataset-level metadata, the metadata describing repositories themselves (their identity, governance, access interfaces, policies, and technical capabilities) remains inconsistent and fragmented across DOE-funded systems. This variability limits discoverability, interoperability, automated validation, and AI-driven analysis, all of which are increasingly essential for modern scientific workflows. To address this gap, the DOE Data Curation Working Group (DCWG) developed the DOE Repository Metadata Profile (DRMP). The DRMP is a practical, community-driven framework that defines how repositories can describe themselves in a consistent, machine-actionable, and scalable manner. The DRMP is not a new metadata schema. Instead, it is a mapping profile and structured element set capturing the essential characteristics of DOE repositories. It harmonizes repository-level metadata across six widely adopted community schemas: RE3Data; DCAT-US v3; Schema.org; Dublin Core; DataCite 4.6; and PREMIS 3.0. This harmonization eliminates reinvention and enables interoperability within DOE and across the broader scientific ecosystem. A core objective of the DRMP is to reduce burden on repositories by allowing them to reuse their existing metadata through a Rosetta-style crosswalk rather than redesigning local implementations. The profile introduces a three-level conformance model that supports incremental adoption: • Level 1 – Minimum Viable Record (MVR): foundational identification elements required for workflows, project registration, and basic repository presence. • Level 2 – Interoperable: structured metadata enabling alignment with national and international discovery systems. • Level 3 – AI-Ready: enhanced provenance, policy transparency, fixity, semantic context, and capabilities that support automated reasoning, model training governance, and machine-assisted curation. To support implementation, the DRMP includes JSON Schema definitions, OpenAPI patterns, and MCP templates that allow repositories to publish machine-readable metadata directly within existing platforms. These resources are modular and lightweight, enabling adoption without major architectural change. Adopting the DRMP enables repositories to: • Enhance discoverability and interoperability by aligning identifiers, classifications, and descriptive elements across widely used schema standards. • Support federated discovery and cross-registration across DOE systems, Data.gov, and international catalogs. • Enable AI agents and workflow orchestration systems to interpret repository-level metadata within the American Science Cloud (AmSC) through Model Context Protocol (MCP)-based context publication. • Demonstrate alignment with DOE’s open science, stewardship, and FAIR data priorities. This guidance represents a community-driven step forward. Through voluntary adoption and continued feedback, the DRMP advances a cohesive, machine-actionable description of DOE repositories that supports FAIR data practices, preparing the infrastructure for AI-enabled research, and strengthening the discoverability and reuse of DOE’s scientific outputs.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Hourly Electricity Demand Profiles for Each County in the Contiguous United States

This dataset provides estimated hourly electricity demand for each county in the contiguous United States from 2016-2023. The demand profiles represent the sum of two components: (1) Weighted averages of reported hourly demand profiles for North American Electric Reliability Corporation balancing authority (BA) regions and subregions, scaled to match annual estimates of county-level retail sales and direct use of electricity and weighted by the estimated percentage of county load served by each BA region or subregion. (2) Weighted averages of modeled hourly, county- and sector-level distributed photovoltaic (DPV) capacity factor profiles, scaled to match annual estimates of on-site consumption of DPV-generated electricity for each county and weighted by the percentage of consumption attributable to each sector Annual county-level retail sales are estimated by aggregating utility-reported sales to the state level and allocating the results to counties according to each county's share of state population. Annual county-level direct use is calculated by aggregating power plant-reported direct use values. Annual county-level on-site consumption of DPV-generated electricity is estimated by aggregating utility-reported net metering data to determine the amount of DPV-generated electricity sold back to the grid for each state, subtracting those values from modeled state-level DPV generation estimates, and allocating the results to counties according to each county's share of statewide modeled DPV generation. The open-source Python code used to develop this dataset is available at "Historical Load Data Repository" link below.

14 SOLAR ENERGY↗

Globally Gridded Groundwater Extraction Volumes and Costs under Six Depletion and Ponded Depth Targets

This repository contains simulated outputs from superwell – a hydro-economic tool for long-term assessment of groundwater cost and supply – providing globally gridded groundwater extractable volumes and associated unit costs ($/km³) for accessible groundwater production, based on a variety of user-defined depletion and ponded depth scenarios. Key model documentation: Niazi, H., Ferencz, S. B., Graham, N. T., Yoon, J., Wild, T. B., Hejazi, M., Watson, D. J., & Vernon, C. R. (2025). Long-term hydro-economic analysis tool for evaluating global groundwater cost and supply: Superwell v1.1. Geoscientific Model Development, 18(5), 1737-1767. https://doi.org/10.5194/gmd-18-1737-2025 Find the source code of the superwell model on GitHub: https://github.com/JGCRI/superwell Repository Overview Main output: superwell_outputs.7z contains 6 files (4.5 GB) named as superwell_py_deep_all_0.*PD_0.*DL.csv. These files present superwell outputs of global groundwater extraction volumes and cost estimates on a 0.5° scale for six scenarios with different Ponded Depth (PD; 0.3 and 0.6 m) and Depletion Limit (DL; 5%, 25%, and 40% of available volume) targets over the entire pumping lifetime of a grid cell superwell_py_deep_all_0.3PD_0.25DL_sample_100.csv contains superwell outputs for 100 data points sampled to match the global inputs' distribution superwell_py_deep_all_0.3PD_0.25DL_Grid_72548.csv contains superwell output for a single grid cell concept_v5.png provides an overview of the superwell workflow Outputs Description year_number: year of pumping depletion_limit: set depletion limit (DL) as a volume fraction of total available groundwater Mappings: continent, country, gcam_basin_id, Basin_long_name, grid_id: geographic identifiers and basin information Inputs: grid_area (km²): area of the grid cell whyclass: hydrogeological classification of the aquifer permeability (m/day), porosity (%), total_thickness (m), depth_to_water (m): aquifer properties. The geo-processed input data has been published separately: https://doi.org/10.57931/2307831 Model outputs: orig_aqfr_sat_thickness (m), aqfr_sat_thickness (m): original and remaining/instantaneous saturated thickness of the aquifer hydraulic_conductivity (m/day), transmissivity (m²/day): hydraulic properties of the aquifer radius_of_influence (m), areal_extent (km²): well radius and area of influence from the center of the well number_of_wells (-): number of wells in a grid cell determined by a ratio of well area and grid area max_drawdown (m), drawdown (m), drawdown_interference (m): well and aquifer drawdown during extraction total_head (m): total lift for the groundwater (depth to water plus drawdown) total_well_length (m): total depth of wells drilled well_yield (m³/day): pumping rate or well yield power (kW), energy (kWh): power and energy required for pumping groundwater Volume Outputs: volume_produced_perwell (m³), cumulative_vol_produced_perwell (m³): production volume metrics per well volume_produced_allwells (m³), cumulative_vol_produced_allwells (m³): aggregate extraction volumes for all wells in a grid cell available_volume (m³): available groundwater in storage for the grid cell as determined by aquifer properties depleted_vol_fraction: fraction of total volume pumped over available volumes in a grid cell (same as depletion limit) Cost Outputs: well_installation_cost ($): well installation cost based on the hydrogeological complexity of the aquifer annual_capital_cost, maintenance_cost, nonenergy_cost ($): nonenergy costs energy_cost_rate ($/kWh): electricity rate energy_cost ($): energy cost of pumping groundwater total_cost_perwell ($), total_cost_allwells ($): total annual energy and non-energy cost for each and all wells in a grid cell a unit_cost ($/m³), unit_cost_per_km3 ($/km³), unit_cost_per_acreft ($/acre-ft): total cost of pumping a unit of groundwater, indicated for different spatial units Key Resources Model documentation: Niazi, H., Ferencz, S., Graham, N., Yoon, J., Wild, T., Hejazi, M., Watson, D., & Vernon, C. (2024; In-prep). Long-term Hydro-economic Assessment Tool for Evaluating Global Groundwater Cost and Supply: Superwell v1. Geoscientific Model Development. Input data: Niazi, H., Watson, D., Hejazi, M., Yonkofski, C., Ferencz, S., Vernon, C., Graham, N., Wild, T., & Yoon, J. (2024). Global Geo-processed Data of Aquifer Properties by 0.5° Grid, Country and Water Basins. MSD-LIVE Data repository. https://doi.org/10.57931/2307831 superwell source code: https://github.com/JGCRI/superwell Cite as Niazi, H., Ferencz, S., Yoon, J., Graham, N., Wild, T., Hejazi, M., Watson, D., & Vernon, C. (2024). Globally Gridded Groundwater Extraction Volumes and Costs under Six Depletion and Ponded Depth Targets. MSD-LIVE Data repository. https://doi.org/10.57931/2307832 Contact Reach out to Hassan Niazi or Stephen Ferencz or open an issue in the superwell repository for questions or suggestions.

Earth Systems↗

MSD CoP Webinar: Using Meta-Repositories To Facilitate Open Science in MSD Research

Context: This webinar was hosted by the MultiSector Dynamics Community of Practice (MSD CoP; https://multisectordynamics.org). Abstract: In this webinar, we will describe the use of a GitHub meta-repository, for documenting and disseminating the tools and data supporting MSD publications. The goal is to make it easier for others to understand the flow of data and code through your experiment and to be able to reproduce your results and figures with only the information you have provided for them. The webinar will cover the role of open science in the MSD community, the origins and purpose of meta-repositories, and step-by-step instructions and best practices for building a meta-repository starting from the GitHub template (https://github.com/IMMM-SFA/metarepo). We will also discuss how to leverage MSD-LIVE (https://msdlive.org/) in your meta-repository, provide links to numerous examples you can learn from, and discuss the role of meta-repositories in the IM3 project's open science mandates. Presenters : Chris R. Vernon, Casey D. Burleyson, Jennie Rice, and Mengqi Zhao Moderator: Pat M. Reed (MSD CoP Facilitation Team) This webinar was held on: February 22nd, 2024 from 2-3 PM ET

Open Science↗

GeoThermalCloud: A Machine Learning Tool for Discovery, Exploration, and Development of Hidden Geothermal Resources

In this 25 minute presentation, we showcase our open source “GeoThermalCloud” tool for identifying hidden geothermal resources using a publicly available dataset for southwestern New Mexico. The presenters include Bulbul Ahmmed and Luke Frash. All of the visuals use source material from LA-UR approved publications and this work falls under the Earth Sciences DUSA. The code shown in this video is already released with LANL approval in open source format on GitHub and DockerHub. The audio in this video includes only material on the topics of geothermal energy and machine learning applied to geothermal energy. The primary machine learning method used is LANL’s Non-negative Matrix Factorization “NMFk” method. Modeling work also mentions LANL’s Geothermal Design Tool “GeoDT” which is another approved open source code that has been released by LANL. This work was performed for DOE Geothermal Technologies Office (DE-EE-3.1.8.1). The host for the released video is intended to be YouTube or a suitable perpetual data repository such as GDR.

15 GEOTHERMAL ENERGY↗

Empowering Geothermal Research: The Geothermal Data Repository's New AI Research Assistant: Preprint

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has integrated a Large Language Model (LLM) with the metadata and supporting documents associated with GDR datasets to create an Artificially Intelligent (AI) research assistant. By leveraging work done to make GDR metadata machine-readable and an open-source LLM integration model called the Energy Language Model, developed by the National Renewable Energy Laboratory, AskGDR serves as a virtual research assistant to GDR users. It provides answers to a variety of user-provided questions using natural language processing and generative machine learning. Users can get answers to questions about specific datasets, including inquiries about the equipment, assumptions and methodologies used in the origination of the data; or more abstract questions, such as the applicability of data to specific research fields. AskGDR improves the discoverability of geothermal data by helping guide users to datasets beyond simple keyword searches. It enables users to find data based on properties of the data, discover information contained within supporting documents, and explore data from projects related to their research objectives.

access↗

Empowering Geothermal Research: The Geothermal Data Repository's New AI Research Assistant

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has integrated a Large Language Model (LLM) with the metadata and supporting documents associated with GDR datasets to create an Artificially Intelligent (AI) research assistant. By leveraging work done to make GDR metadata machine-readable and an open-source LLM integration model called the Energy Language Model, developed by the National Renewable Energy Laboratory, AskGDR serves as a virtual research assistant to GDR users. It provides answers to a variety of user-provided questions using natural language processing and generative machine learning. Users can get answers to questions about specific datasets, including inquiries about the equipment, assumptions and methodologies used in the origination of the data; or more abstract questions, such as the applicability of data to specific research fields. AskGDR improves the discoverability of geothermal data by helping guide users to datasets beyond simple keyword searches. It enables users to find data based on properties of the data, discover information contained within supporting documents, and explore data from projects related to their research objectives. This paper will outline the development, integration, output, and efficacy of the AskGDR LLM, including adherence to scientific rigor through improvements designed to increase the accuracy of generated answers, avoid speculation, and provide proper references for all resources used.

access↗

CORE: A Global Aggregation Service for Open Access Papers

This paper introduces CORE, a widely used scholarly service, which provides access to the world’s largest collection of open access research publications, acquired from a global network of repositories and journals. CORE was created with the goal of enabling text and data mining of scientific literature and thus supporting scientific discovery, but it is now used in a wide range of use cases within higher education, industry, not-for-profit organisations, as well as by the general public. Through the provided services, CORE powers innovative use cases, such as plagiarism detection, in market-leading third-party organisations. CORE has played a pivotal role in the global move towards universal open access by making scientific knowledge more easily and freely discoverable. In this paper, we describe CORE’s continuously growing dataset and the motivation behind its creation, present the challenges associated with systematically gathering research papers from thousands of data providers worldwide at scale, and introduce the novel solutions that were developed to overcome these challenges. The paper then provides an in-depth discussion of the services and tools built on top of the aggregated data and finally examines several use cases that have leveraged the CORE dataset and services.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

geothermal_osr (Energy Predictor for Geothermal Open Source Reservoir) [SWR-22-23]

This is a repository with the tools developed for predicting energy produced at Open-Source Reservoir (OSR). It includes both simulation data for OSR, as well as Jupyter notebooks for training and evaluating prediction models. OSR was constructed based on the data from Brady Hot Springs reservoir (Nevada, USA) but has a number of sufficiently modified characteristics and does not disclose any sensitive data.

Duplyakin, Dmitry↗

Improving the Accessibility and Usability of Geothermal Information with Data Lakes and Data Pipelines on the Geothermal Data Repository: Preprint

The Geothermal Data Repository (GDR) provides universal access to data and information resulting from research and development activities funded by the Department of Energy (DOE). The GDR has extended this universal access to big data through integration with data lakes developed by the Open Energy Data Initiative (OEDI). Previously, large datasets such as seismic waveform or distributed acoustic sensing (DAS) data could only be accessed by institutions with high performance data storage and compute capabilities, effectively limiting the accessibility of big data to national labs, larger universities, and major corporations. Moreover, the time and resources needed to transport big data and configure them can produce additional barriers to use. Many of the standard formats used for structured data models (also known as content models) are incapable of handling big data and can introduce additional usability problems, often requiring data to be reformatted prior to use. This paper will explore how recent integrations between the GDR and the OEDI data lake have improved the accessibility and usability of geothermal data in a big way, making the data available to a broader audience, and enabling collaborative analysis and innovation across the greater geothermal industry.

access↗

Models, data, and scripts associated with “Prediction of Distributed River Sediment Respiration Rates using Community-Generated Data and Machine Learning”

This data package is associated with the publication “Prediction of Distributed River Sediment Respiration Rates using Community-Generated Data and Machine Learning’’ submitted to the Journal of Geophysical Research: Machine Learning and Computation (Scheibe et al. 2024). River sediment respiration observations are expensive and labor intensive to obtain and there is no physical model for predicting this quantity. The Worldwide Hydrobiogeochemisty Observation Network for Dynamic River Systems (WHONDRS) observational data set (Goldman et al.; 2020) is used to train machine learning (ML) models to predict respiration rates at unsampled sites. This repository archives training data, ML models, predictions, and model evaluation results for the purposes of reproducibility of the results in the associated manuscript and community reuse of the ML models trained in this project. One of the key challenges in this work was to find an optimum configuration for machine learning models to work with this feature-rich (i.e. 100+ possible input variables) data set. Here, we used a two-tiered approach to managing the analysis of this complex data set: 1) a stacked ensemble of ML models that can automatically optimize hyperparameters to accelerate the process of model selection and tuning and 2) feature permutation importance to iteratively select the most important features (i.e. inputs) to the ML models. The major elements of this ML workflow are modular, portable, open, and cloud-based, thus making this implementation a potential template for other applications. This data package is associated with the GitHub repository found at Please see the file level metadata (flmd; “sl-archive-whondrs_flmd.csv”) for a list of all files contained in this data package and descriptions for each. Please see the data dictionary (dd; “sl-archive-whondrs_dd.csv”) for a list of all column headers contained within comma separated value (csv) files in this data package and descriptions for each. The GitHub repository is organized into five top-level directories: (1) “input_data” holds the training data for the ML models; (2) “ml_models” holds machine learning models trained on the data in “input_data”; (3) “scripts” contains data preprocessing and postprocessing scripts and intermediate results specific to this data set that bookend the ML workflow; (4) “examples” contains the visualization of the results in this repository including plotting scripts for the manuscript (e.g., model evaluation, FPI results) and scripts for running predictions with the ML models (i.e., reusing the trained ML models); (5) “output_data” holds the overall results of the ML model on that branch. Each trained ML model resides on its own branch in the repository; this means that inputs and outputs can be different branch-to-branch. Furthermore, depending on the number of features used to train the ML models, the preprocessing and postprocessing scripts, and their intermediate results, can also be different branch-to-branch. The “main-*” branches are meant to be starting points (i.e. trunks) for each model branch (i.e. sprouts). Please see the Branch Navigation section in the top-level README.md in the GitHub repository for more details. There is also one hidden directory “.github/workflows”. This hidden directory contains information for how to run the ML workflow as an end-to-end automated GitHub Action but it is not needed for reusing the ML models archived here. Please the top-level README.md in the GitHub repository for more details on the automation.

13C↗

Code for the manuscript "Mori-Zwanzig Modal Decomposition"

We would like to create an open source repository in LANL's github on code written in Julia, in which we implement and extend the data-driven Mori-Zwanzig method for extracting large-scale spatio-temporal structures from data, which we call MZMD. This method is an extension of Dynamic Mode Decomposition (DMD) in which Mori-Zwanzig memory kernels are included into the associated companion matrix. In the code we would like to release, we apply MZMD to a flow over a cylinder with Reynolds number 100 rather than the much larger data set used in the associated manuscript. DMD is used extensively in the fluid dynamics community mainly for extracting large scale spatio-temporal structures (patters) from flow data. This is useful for understanding the key mechanisms that generate certain complex dynamical process relevant in engineering design. In MZMD, we improve upon DMD by adding the Mori-Zwanzig memory kernels, and show this improvement is especially important in strongly nonlinear regions of the flow.

Woodward, Michael↗