SEARCH · Search NASA

Results for “open datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

All sources NASA NTRS DOE OSTI

DOE OSTI2024

From RNNs to Foundation Models: An Empirical Study on Commercial Building Energy Consumption

Accurate short-term energy consumption forecasting for commercial buildings is crucial for smart grid operations. While smart meters and deep learning models enable forecasting using past data from multiple buildings, data heterogeneity from diverse buildings can reduce model performance. The impact of increasing dataset heterogeneity in time series forecasting, while keeping size and model constant, is understudied. We tackle this issue using the ComStock dataset, which provides synthetic energy consumption data for U.S. commercial buildings. Two curated subsets, identical in size and region but differing in building type diversity, are used to assess the performance of various time series forecasting models, including finetuned open-source foundation models (FMs). The results show that dataset heterogeneity and model architecture have a greater impact on post-training forecasting performance than the parameter count. Moreover, despite the higher computational cost, finetuned FMs demonstrate competitive performance compared to base models trained from scratch.

commercial buildings↗

NASA NTRSDate not supplied

Batch Effect Correction Methods for NASA GeneLab Transcriptomic Datasets

RNA sequencing (RNA-seq) data from space biology experiments promise to yield invaluable insights into the effects of spaceflight on terrestrial biology. However, sample numbers from each study are low due to limited crew availability, hardware, and space. To increase statistical power, spaceflight RNA-seq datasets from different missions are often aggregated together. However, this can introduce technical variation or "batch effects", often due to differences in sample handling, sample processing, and sequencing platforms. Several computational methods have been developed to correct for technical batch effects, thereby reducing their impact on true biological signals. In this study, we combined 7 mouse liver RNA-seq datasets from NASA GeneLab (part of the NASA Open Science Data Repository) to evaluate several common batch effect correction methods (ComBat and ComBat-seq from the sva R package, and Median Polish, Empirical Bayes, and ANOVA from the MBatch R package). We quantitatively evaluated the ability of these methods to correct for technical batch variables in space biology RNA-seq data using the following criteria: BatchQC, principal component analysis, dispersion separability criterion, log fold change correlation, and differential gene expression analysis. Each batch variable / correction method combination was then assessed using a custom scoring approach to identify the optimal correction method for the combined dataset, by geometrically probing the space of all allowable scoring functions to yield an aggregate volume-based scoring measure. Finally, we describe the way in which the GeneLab multi-study analysis and visualization portal will allow users to examine the presence or absence of batch effects using multiple metrics. If the user chooses to perform batch effect correction, the scoring approach described here can be implemented to identify the optimal correction method to use for their specific combined dataset prior to analysis.

Lauren M. Sanders↗

DOE OSTI2025

HexWeather: Hexagonal Spatial Data Aggregation for Weather-Driven Grid Resilience Analysis

Extreme weather accounts for over 8 0 % of major U.S. power outages since 2000, highlighting the need for spatial tools that align weather data with the irregular boundaries of electric infrastructure. This paper introduces HexWeather, a modular, resolution-aware framework for aggregating historical and forecasted weather data using Uber's H3 hexagonal spatial indexing system. Unlike traditional methods that rely on state or county-level grids, HexWeather enables weather analysis across custom geographies such as utility service areas where public datasets are often unavailable or misaligned. Using Open-Meteo data, we evaluate how H3 resolution affects anomaly detection, spatial variability, and forecast uncertainty across three scales: state, county, and utility. Results show that while coarse resolutions suffice for broad trend tracking, finer resolutions are essential for identifying localized variability and operational risks. By applying metrics like Z-score standard deviation and interquartile range, HexWeather quantifies the spatial spread of both historical anomalies and forecasted conditions, allowing users to assess resolution adequacy for each analysis. This framework supports rapid weather data reuse, reproducible anomaly detection, and predictive modeling for infrastructure resilience. By bridging spatial misalignment in traditional datasets and enabling retrospective and forward-looking analysis within the same pipeline, HexWeather lays the groundwork for better post event analysis, outage prediction, and resilience planning.

Morris, Jacob [ORNL]↗

NASA NTRS2016

Managing and Servicing Physical Oceanographic Data at a NASA Distributed Active Archive Center

The NASA Earth Science Data Information Systems Project funds and operates 12 Distributed Active Archive Center(s) (DAAC) throughout the United States. Of these 12 centers, the Physical Oceanography DAAC (PO.DAAC) is committed to providing long term archival, distribution and stewardship for NASA physical oceanographic data, primarily derived from space-born satellite systems, but also including a growing set of recent and future in situ observations from the SPURS-1 and SPURS-2 campaigns. Notable NASA missions supported include: Seasat, TOPEX/Poseidon, NSCAT, QuikSCAT, ISS-RapidScat, Jason-1, Jason-2/OSTM, GRACE, Aquarius, GHRSST, and MODIS. The following interagency and international missions are also supported by PO.DAAC: AVHRR, Coriolis, DMSP, MetOp-A, MetOp-B, Oceansat-2. The PO.DAAC currently holds 525 datasets in public distribution, spanning the following observational parameters: sea surface temperature, sea surface salinity, ocean color, ocean surface currents, ocean surface wind speed, ocean surface wind direction, sea surface height, significant wave height, ocean water mass/thickness, and sea ice age. A hundred of these datasets are available in near-real-time. Datasets are distributed through a variety of open-source access protocols including FTP, OPeNDAP, and THREDDS. FTP will soon be phased out in favor of a recently introduced HTTPS PO.DAAC Drive interface that supports WebDAV and interoperable machine-to-machine communication. OPeNDAP supports remote data/metadata query, subset, and download. THREDDS provides the features of OPeNDAP with the additional feature of temporal aggregation. PO.DAAC also offers proprietary tools and services to further enhance the data discovery, visualization and analysis experience, including but not limited to: State of the Ocean, Web Services (data/metadata discovery and extraction), HiTIDE Level-2 subsetter, Live Access Server (LAS), Webification (w10nsci), and Rich Site Summary (RSS) Datacasting. To assist with provenance of datasets, PO.DAAC has implemented DOIs for the data it distributes so that they can be properly cited. There is a user forum and helpdesk that contains data recipes and via which users can get guidance. In summary, this presentation aims to provide a general overview of PO.DAAC’s web portal and data holdings along with a set of illustrative examples leading prospective data users into the practical utility of its tools and services.

Moroni, David F.↗

NASA NTRSDate not supplied

Using Kamodo for CCMC ITM Output and Beyond

Kamodo is an official NASA open source python software package that functionalizes diverse datasets from models and observations in a consistent way, enabling advanced scientific analysis and visualization with simplistic syntax. Here we demonstrate this ability using several ITM models available through the Community Coordinated Modeling Center (CCMC). Users can now interact directly with model outputs, and satellites can be virtually flown through model output to allow many types of model/model and data/model comparisons. We will also provide information about significant updates and improvements to Kamodo and future plans.

Open Source Software↗

NASA NTRS2011

Improve Data Mining and Knowledge Discovery Through the Use of MatLab

Data mining is widely used to mine business, engineering, and scientific data. Data mining uses pattern based queries, searches, or other analyses of one or more electronic databases/datasets in order to discover or locate a predictive pattern or anomaly indicative of system failure, criminal or terrorist activity, etc. There are various algorithms, techniques and methods used to mine data; including neural networks, genetic algorithms, decision trees, nearest neighbor method, rule induction association analysis, slice and dice, segmentation, and clustering. These algorithms, techniques and methods used to detect patterns in a dataset, have been used in the development of numerous open source and commercially available products and technology for data mining. Data mining is best realized when latent information in a large quantity of data stored is discovered. No one technique solves all data mining problems; challenges are to select algorithms or methods appropriate to strengthen data/text mining and trending within given datasets. In recent years, throughout industry, academia and government agencies, thousands of data systems have been designed and tailored to serve specific engineering and business needs. Many of these systems use databases with relational algebra and structured query language to categorize and retrieve data. In these systems, data analyses are limited and require prior explicit knowledge of metadata and database relations; lacking exploratory data mining and discoveries of latent information. This presentation introduces MatLab(R) (MATrix LABoratory), an engineering and scientific data analyses tool to perform data mining. MatLab was originally intended to perform purely numerical calculations (a glorified calculator). Now, in addition to having hundreds of mathematical functions, it is a programming language with hundreds built in standard functions and numerous available toolboxes. MatLab's ease of data processing, visualization and its enormous availability of built in functionalities and toolboxes make it suitable to perform numerical computations and simulations as well as a data mining tool. Engineers and scientists can take advantage of the readily available functions/toolboxes to gain wider insight in their perspective data mining experiments.

Shaykhian, Gholam Ali↗

NASA NTRS2011

Improve Data Mining and Knowledge Discovery through the use of MatLab

Data mining is widely used to mine business, engineering, and scientific data. Data mining uses pattern based queries, searches, or other analyses of one or more electronic databases/datasets in order to discover or locate a predictive pattern or anomaly indicative of system failure, criminal or terrorist activity, etc. There are various algorithms, techniques and methods used to mine data; including neural networks, genetic algorithms, decision trees, nearest neighbor method, rule induction association analysis, slice and dice, segmentation, and clustering. These algorithms, techniques and methods used to detect patterns in a dataset, have been used in the development of numerous open source and commercially available products and technology for data mining. Data mining is best realized when latent information in a large quantity of data stored is discovered. No one technique solves all data mining problems; challenges are to select algorithms or methods appropriate to strengthen data/text mining and trending within given datasets. In recent years, throughout industry, academia and government agencies, thousands of data systems have been designed and tailored to serve specific engineering and business needs. Many of these systems use databases with relational algebra and structured query language to categorize and retrieve data. In these systems, data analyses are limited and require prior explicit knowledge of metadata and database relations; lacking exploratory data mining and discoveries of latent information. This presentation introduces MatLab(TradeMark)(MATrix LABoratory), an engineering and scientific data analyses tool to perform data mining. MatLab was originally intended to perform purely numerical calculations (a glorified calculator). Now, in addition to having hundreds of mathematical functions, it is a programming language with hundreds built in standard functions and numerous available toolboxes. MatLab's ease of data processing, visualization and its enormous availability of built in functionalities and toolboxes make it suitable to perform numerical computations and simulations as well as a data mining tool. Engineers and scientists can take advantage of the readily available functions/toolboxes to gain wider insight in their perspective data mining experiments.

Shaykahian, Gholan Ali↗

DOE OSTI2025

Hourly Electricity Demand Profiles for Each County in the Contiguous United States

This dataset provides estimated hourly electricity demand for each county in the contiguous United States from 2016-2023. The demand profiles represent the sum of two components: (1) Weighted averages of reported hourly demand profiles for North American Electric Reliability Corporation balancing authority (BA) regions and subregions, scaled to match annual estimates of county-level retail sales and direct use of electricity and weighted by the estimated percentage of county load served by each BA region or subregion. (2) Weighted averages of modeled hourly, county- and sector-level distributed photovoltaic (DPV) capacity factor profiles, scaled to match annual estimates of on-site consumption of DPV-generated electricity for each county and weighted by the percentage of consumption attributable to each sector Annual county-level retail sales are estimated by aggregating utility-reported sales to the state level and allocating the results to counties according to each county's share of state population. Annual county-level direct use is calculated by aggregating power plant-reported direct use values. Annual county-level on-site consumption of DPV-generated electricity is estimated by aggregating utility-reported net metering data to determine the amount of DPV-generated electricity sold back to the grid for each state, subtracting those values from modeled state-level DPV generation estimates, and allocating the results to counties according to each county's share of statewide modeled DPV generation. The open-source Python code used to develop this dataset is available at "Historical Load Data Repository" link below.

14 SOLAR ENERGY↗

DOE OSTI2025

Simulated Microstructures for Laser Powder Bed Fusion Additive Manufacturing Using Myna, AdditiveFOAM, and ExaCA

This dataset provides sample datasets containing voxelized, three-dimensional representations of simulated grain structures and crystallographic orientations that can result from laser powder bed fusion additive manufacturing. The six microstructure files each contain approximately 1 cubic millimeter of material (1 mm x 1 mm cross-section over 26 simulated layers of deposition). Some of the microstructures have columnar grains that extend across nearly the entire simulation domain, while others have more equiaxed or truncated columnar grains. The process conditions to generate these microstructures were from the Peregrine v2023-10 dataset (10.13139/ORNLNCCS/2008021). The codes used are publicly available and released under open-source licenses. Myna (https://github.com/ORNL-MDF/Myna) was used for configuration of the cases from the Peregrine v2023-10 HDF5 dataset and to run the simulation workflow. AdditiveFOAM (https://github.com/ORNL/AdditiveFOAM) was used to simulate the melt pool and generate solidification conditions. And ExaCA (https://github.com/LLNL/ExaCA ) was used to simulate the three-dimensional microstructures.

36 MATERIALS SCIENCE↗

DOE OSTI2024

BindingDB in 2024: a FAIR knowledgebase of protein-small molecule binding data

Abstract BindingDB (bindingdb.org) is a public, web-accessible database of experimentally measured binding affinities between small molecules and proteins, which supports diverse applications including medicinal chemistry, biochemical pathway annotation, training of artificial intelligence models and computational chemistry methods development. This update reports significant growth and enhancements since our last review in 2016. Of note, the database now contains 2.9 million binding measurements spanning 1.3 million compounds and thousands of protein targets. This growth is largely attributable to our unique focus on curating data from US patents, which has yielded a substantial influx of novel binding data. Recent improvements include a remake of the website following responsive web design principles, enhanced search and filtering capabilities, new data download options and webservices and establishment of a long-term data archive replicated across dispersed sites. We also discuss BindingDB’s positioning relative to related resources, its open data sharing policies, insights gleaned from the dataset and plans for future growth and development.

Liu, Tiqing↗

NASA NTRS2017

Static Electric Fields and Lightning Over Land and Ocean in Florida Thunderstorms

Natural cloud-to-ground (CG) lightning and the charge structure of the associated clouds behave differently over land and ocean. Existing literature has raised questions over the years on the behavior of thunderstorms and lightning over oceans, and there are still open scientific questions. We expand on the observational datasets by obtaining identical electric field observations over coastal land, near-shore, and deep ocean regions during both clear air and thunderstorm periods. Oceanic observations were obtained using two 3-meter NOAA buoys that were instrumented with Campbell Scientific electric field mills to measure the static electric fields. These data were compared to selected electric field records from the existing on-shore electric field mill suite of 31 sensors at Kennedy Space Center (KSC). CG lightning occurrence times, locations and peak current values for both on-shore and ocean were provided by the U.S. National Lightning Detection Network. The buoy instruments were first evaluated on-shore at the Florida coast, to calibrate field enhancements and to confirm proper behavior of the system in elevated-field environments. The buoys were then moored 20NM and 120NM off the coast of KSC in February (20NM) and August (120NM) 2014. Statistically larger CG peak currents were reported over the deep ocean for first strokes and for subsequent strokes with new contacts points. Storm-related static fields were significantly larger at both oceanic sites, likely due to decreased screening by nearby space charge. Time-evolution of the static field during storm development and propagation indicated weak or missing lower positive charge regions in most storms that initiated over the deep ocean, supporting one mechanism for the observed high peak currents in negative first strokes over the deep ocean. This project also demonstrated the practicality of off-shore electric field measurements for safety-related decision making at KSC.

Lightning↗

NASA NTRS2017

Static Electric Fields and Lightning Over Land and Ocean in Florida Thunderstorms

Electric Fields↗

NASA NTRS2019

GeneLab: A Systems Biology Platform for Omics Analysis: Disseminate and Reuse Data, Tools, and Samples Post-Project

NASA's GeneLab includes an open-access repository of some 200 plus omics datasets generated by biological experiments relevant to spaceflight (including simulated cosmic radiation and microgravity). In order to maximize the intelligibility of these data, particularly for users with limited bioinformatics knowledge, GeneLab is now transforming the data in the repository into actual biological and physiological knowledge of the genetic and proteomic signatures found in these samples. This processed data is being derived by establishing standard data analysis workflows vetted by 114 scientists who are members of the four GeneLab Analysis Working Groups (Animal AWG, Plant AWG, Microbe AWG, Multi-Omics AWG). AWG members from institutes spanning the U.S. and four other countries participate on a voluntary basis. The AWGs meet monthly to discuss data mining, compare results and interpretations, and test forthcoming releases of the GeneLab Data Systems (GLDS). GLDS version 3.0 has been available to the general public since October 1st 2018, and has been providing a professional state-of-the-art bioinformatics platform for everyone in the space biology community to upload their data into a space biology omics data commons, to process their data with vetted standard workflows and to compare to existing analyses. The user interface for the platform is being designed to be accessible to a broad variety of users including those with limited bioinformatics experience, including high school and college students who can use it to learn about omics data analysis and space biology. As such, Genelab will constitute a powerful general public outreach capability of NASA and the Space Biology community at large. Data mining of the GeneLab database by the AWG has already started generating very interesting findings, including reports linking specific spaceflight conditions such as radiation, microgravity or carbon dioxide levels to molecular changes seen across various species. In this presentation, we will report on the current and future objectives for GeneLab, and review recent studies reported by the various AWGs relating molecular changes observed in various animal models and tissue with microgravity, radiation, circadian rhythm, hydration and carbon dioxide conditions.

Omics↗

NASA NTRS2022

Assimilation of NASA’s Airborne Snow Observatory Snow Measurements for Improved Hydrological Modeling: A Case Study Enabled by the Coupled LIS/WRF-Hydro System

he NASA LIS/WRF-Hydro system is a coupled modeling framework that combines the modeling and data assimilation (DA) capabilities of the NASA Land Information System (LIS) with the multi-scale surface hydrological modeling capabilities of the WRF-Hydro model, both of which are widely used in both operations and research. This coupled modeling framework builds on the linkage between land surface models (LSMs), which simulate surface boundary conditions in atmospheric models, and distributed hydrologic models, which simulate horizontal surface and sub-surface flow, adding new land DA capabilities. In the present study, we employ this modeling framework in the Tuolumne River basin in central California. We demonstrate the added value of the assimilation of NASA Airborne Snow Observatory (ASO) snow water equivalent (SWE) estimates in the Tuolumne basin. This analysis is performed in both LIS as an LSM column model and LIS/WRF-Hydro, with hydrologic routing. Results demonstrate that ASO DA in the basin reduced snow bias by as much as 30% from an open-loop (OL) simulation compared to three independent datasets. It also reduces downstream streamflow runoff biases by as much as 40%, and improves streamflow skill scores in both wet and dry years. Analysis of soil moisture and evapotranspiration (ET) also reveals the impacts of hydrologic routing from WRF-Hydro in the simulations, which would otherwise not be resolved in an LSM column model. By demonstrating the beneficial impact of SWE DA on the improving streamflow forecasts, the article outlines the importance of such observational inputs for reservoir operations and related water management applications.

Hydrologic modeling↗

NASA NTRS2022

Assimilation of NASA's Airborne Snow Observatory Snow Measurements for Improved Hydrological Modeling: A Case Study Enabled by the Coupled LIS/WRF-Hydro System

The NASA LIS/WRF-Hydro system is a coupled modeling framework that combines the modeling and data assimilation (DA) capabilities of the NASA Land Information System (LIS) with the multi-scale surface hydrological modeling capabilities of the WRF-Hydro model, both of which are widely used in both operations and research. This coupled modeling framework builds on the linkage between land surface models (LSMs), which simulate surface boundary conditions in atmospheric models, and distributed hydrologic models, which simulate horizontal surface and sub-surface flow, adding new land DA capabilities. In the present study, we employ this modeling framework in the Tuolumne River basin in central California. We demonstrate the added value of the assimilation of NASA Airborne Snow Observatory (ASO) snow water equivalent (SWE) estimates in the Tuolumne basin. This analysis is performed in both LIS as an LSM column model and LIS/WRF-Hydro, with hydrologic routing. Results demonstrate that ASO DA in the basin reduced snow bias by as much as 30% from an open-loop (OL) simulation compared to three independent datasets. It also reduces downstream streamflow runoff biases by as much as 40%, and improves streamflow skill scores in both wet and dry years. Analysis of soil moisture and evapotranspiration (ET) also reveals the impacts of hydrologic routing from WRF-Hydro in the simulations, which would otherwise not be resolved in an LSM column model. By demonstrating the beneficial impact of SWE DA on the improving streamflow forecasts, the article outlines the importance of such observational inputs for reservoir operations and related water management applications.

Timothy M. Lahmers↗

DOE OSTI2025

Aspen Open Jets: unlocking LHC data for foundation models in particle physics

Foundation models are deep learning models pre-trained on large amounts of data which are capable of generalizing to multiple datasets and/or downstream tasks. This work demonstrates how data collected by the CMS experiment at the Large Hadron Collider can be useful in pre-training foundation models for HEP. Specifically, we introduce the AspenOpenJets (AOJs) dataset, consisting of approximately 178 M high p T jets derived from CMS 2016 Open Data. We show how pre-training the OmniJet-α foundation model on AOJs improves performance on generative tasks with significant domain shift: generating boosted top and QCD jets from the simulated JetClass dataset. In addition to demonstrating the power of pre-training of a jet-based foundation model on actual proton–proton collision data, we provide the ML-ready derived AOJs dataset for further public use.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2026

Web-based Preprocessing and Visualization of 3D FIB Tomography Data for Nuclear Fuel Characterization

Three-dimensional (3D) focused ion beam (FIB) tomography enables reconstruction of internal nuclear fuel features that can't be fully evaluated through surface imaging alone. This capability supports characterization of fuel constituents and defects under thermal and irradiation conditions relevant to microreactor development. However, large tomography datasets can create data-handling, loading, and visualization challenges, especially when image-stack preparation and file conversion must be completed with separate tools. The Computational Ultraspatial Tomography Toolkit for High-Resolution Object Analysis Tools (CUTTRHOAT) is an open-source web application being developed to display FIB tomography datasets available through the Nuclear Research Data System (NRDS). The current alpha version requires prepared HDF5 datasets and has limited integrated data-preparation capabilities. This project improves CUTTHROAT by adding dataset-folder selection, automatic input detection, dataset scanning, missing-slice identification, blank-slice insertion, and image-stack-to-HDF5 conversion. Two applications will be compared: the baseline CUTTHROAT alpha workflow and the updated application containing the integrated data-handling and preprocessing functions. Evaluation will consider dataset detection accuracy, conversion success, loading time, rendering responsiveness, application stability, and user interaction. Preliminary results demonstrate successful loading of existing HDF5 files and converted image stacks, while testing also identified performance reductions caused by excessive blank-slice generation. The updated workflow reduces reliance on external preparation tools and supports more direct movement from image stacks to color-code 3D visualization. Future work includes refining missing-slice handling, integrating additional preprocessing functions, like a denoising feature, parsing TIFF metadata for automatic voxel scaling, and adding manual X, Y, and Z voxel-spacing inputs for PNG and JPEG.

36 - MATERIALS SCIENCE↗

NASA NTRS2017

NASA's Big Earth Data Initiative Accomplishments

The goal of NASA's effort for BEDI is to improve the usability, discoverability, and accessibility of Earth Observation data in support of societal benefit areas. Accomplishments: In support of BEDI goals, datasets have been entered into Common Metadata Repository(CMR), made available via the Open-source Project for a Network Data Access Protocol (OPeNDAP), have a Digital Object Identifier (DOI) registered for the dataset, and to support fast visualization many layers have been added in to the Global Imagery Browse Services (GIBS).

CMR↗