SEARCH · Search NASA

Results for “datasets metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

All sources NASA NTRS DOE OSTI

DOE OSTI2026

COMPASS-FME Synoptic Sites Level 2 Sensor Data v2-1

This is the version 2-1 Level 2 (L2) data release for COMPASS-FME environmental sensors located at our synoptic field sites. COMPASS-FME is studying sites in two distinct regions, the Chesapeake Bay and the Western Lake Erie Basin. We established the network at seven "synoptic" (observational) sites along the Chesapeake Bay and Lake Erie coastlines, collectively generating over three million observations per month, to track and comprehend environmental changes where land and water intersect. Additionally, the two regions provide an interesting contrast of saltwater and freshwater coasts that allow us to differentiate the impacts of inundation and coastal water chemistries in two nationally important coastal systems. Level 2 (L2) data consist of sensor observations from the COMPASS-FME synoptic sites, TEMPEST, and DELUGE. Compared to the L1 data, these are more consistent (always 15-minute timestamps for the entire year); better QA/QC’d (out of bounds, out of service, and extreme outlier values are removed); and more complete, with a gap-filled time series available alongside the main observations, and additional derived (calculated) variables. L2 data are intended to be rapidly and easily usable in analyses and simulations. However, algorithmic outlier identification always carries the risk of removing valid data, and Level 1 data may be more suitable for analyses that focus on variability or extreme events. This dataset includes: - An overall dataset README file that describes the current version, gives citation and contact information, etc. - Site- and year-specific folders, each holding variable-specific Parquet (a high performance, space efficient format; see https://parquet.apache.org) data files for each site and plot in that year. - Metadata files within each site-year folder provide full information on data units, expected ranges, contact information, detailed flood times, as well as a general description of the site. - Environmental sensor types that appear in the data files include weather (ClimaVUE50, CS, RM Young, and LI instruments in the graphs below); soil conditions (TEROS12); soil redox state (Redox); groundwater variables (AquaTROLL200 and AquaTROLL600); open water sondes (Exo); tree sap velocity (Sapflow); and system voltage and state (Datalogger). Data are reported every 15 minutes. Data files are in Apache Parquet, a high performance, space efficient format for tabular data. These files can be read using R's `arrow` package (https://arrow.apache.org/docs/r/), with similar tools available in other languages. Please see v2-1 L2 Sensor Package QStart.pdf for detailed information on data package structure, temporal coverage, and versioning.

EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC TEMPERATU↗

DOE OSTI2022

Leaf gas exchange (BNL-PNNL collaboration), San Lorenzo, Panama, 2020

This data package contains Photosynthetic CO2 response (ACi curves) and Light Response (AQ) of mature sunlit canopy leaves of 11 species from PA-SLZ, February 2020. Branches were cut from the top of the canopy and re-cut under water. Fitted Vcmax and Jmax photosynthetic parameters were derived from measured data as well as the convexity, light use efficiency and apparent quantum yield. This data was collected as a collaboration between Brookhaven National Laboratory (BNL) and Pacific Northwest National Laboratory (PNNL), and is a subset of the 2020 BNL Panama campaign. The species included in this dataset are: Apeiba membranacea, Carapa guianensis, Cespedesia spathulate, Jacaranda copaia, Lonchocarpus heptaphyllus, Pera arboea, Pourouma bicolore, Symphonia globulifera, Tachigali versicolor, Tapirira guianensis, and Virola sebifera. The files included in this data package are in .csv and .pdf format, with two Excel files containing additional metadata.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

A multi-scale time-series dataset of anthropogenic heat from buildings in Los Angeles County

The dataset contains hourly Anthropogenic heat (AH) from buildings in Los Angeles County, based on weather data from 2018. The hourly AH is aggregated at three spatial resolutions: 450m x 450m grid, 12km x 12km grid, and census tract. The AH is broken down into three components: building envelope surface convection, heating, ventilation, and air conditioning (HVAC) system heat release, and zone exfiltration and exhaust air heat loss. The dataset is created with the physics-based EnergyPlus building energy models to calculate individual buildings' AH considering WRF-UCM simulated microclimate conditions. Please refer to the paper "A multi-scale time-series dataset of anthropogenic heat from buildings in Los Angeles County" for more information about the data generation workflow and the data validation procedure. The data set contains two folders: the "output_data" folder holds the simulation results (EP_output and EP_output_csv), building metadata (building_metadata.geojson and building_metadata.csv), aggregated heat emission and energy consumption time-series data (hourly_heat_energy), and geographical data (geo_data) associated with the GEOID referenced in heat and energy consumption data. The "input_data" folder contains the raw data used to generate files in the "output_data" folder as well as data sets used in the validation. The code repository (https://github.com/IMMM-SFA/xu_etal_2022_sdata) holds the processing scripts for data curation, validation, and visualization.

Energy↗

DOE OSTI2025

Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification" Willard et al. (2025).

This data release provides all data and code used in the paper " "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantifications" Willard et al. (2025)" to model stream temperature, evaluate, and assess results. The associated manuscript explores the effect of different ensemble construction techniques across different common machine learning (ML) architectures for predictions in unmonitored basins. Modeling was done using long short-term memory (LSTM), gated recurrent unit (GRU), temporal convolution network (TCN), and extreme gradient boosting (XGBoost) models, and stream site coverage spans 1362 locations across the conterminous United States. The ensemble construction techniques investigated include ensemble by random weight initialization, differing hyperparameters, different random subsets of training data, different subselections of input features, different architectures, and Monte Carlo Dropout. The data is organized into these items items:Code repository and data for the paper " "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantifications" Willard et al. (2025).Code: stream_temp_ml_regionalization.zip contains the code repositoryData to run the code:- data_dir.zip -- contains all files that should be moved to the "DATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- metadata_dir.zip -- contains all files that should be moved to the "METADATA_DIR" variable defined in the "set_env_vars.sh" script in the code repositoryData produced by the code and used in the paper:- outputs_dir.zip - contains model output and results (outputs_dir/results), model weights (outputs_dir/models), and all other outputs used for the paper including feature importances.To cite this code, please use the following BibTeX or MLA entries:bibtex:@misc{willard2025streamensembles,author = {Jared Willard and Charuleka Varadharajan},title = {Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification"},year = {2024},doi = {10.15485/2527393},publisher = {ESS-DIVE Repository},url = {https://data.ess-dive.lbl.gov/datasets/doi:10.15485/2527393}}MLA: Willard, Jared, et al. Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification". 2025. ESS-DIVE Repository, doi:10.15485/2448016.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Soil Temperature and Moisture, Teller Road Mile Marker 27, Seward Peninsula, Alaska, 2016-2023

Daily averages of soil temperature and moisture measured once every hour at different heights located at Intensive Monitoring Stations at Teller Road Mile Marker 27 site. Data are retrieved annually from 2016-2023. This is the final update to this dataset: no changes to previous data files with the addition of 2022-2023 data files. Data files have header rows, NaN fields indicate invalid or missing data, and negative vertical offsets are above ground. UPDATED 2025-07: Seven new data files collected in 2022-2023 were added and no previously archived data were changed. This represents the final collection of data and text in the title was changed to reflect the defined timespan. Other files that were added: file-level metadata, a data dictionary, a site location map, and a data file inventory by year. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2002

Spatial Metadata for Global Change Investigations Using Remote Sensing

Satellite and aircraft-borne remote sensors have gathered petabytes of data over the past 30+ years. These images are an important resource for establishing cause and effect relationships between human-induced land cover changes and alterations in climate and other biophysical patterns at local to global scales. However, the spatial, temporal, and spectral characteristics of these datasets vary, thus complicating long-term studies involving several types of imagery. As the geographical and temporal coverage, the spectral and spatial resolution, and the number of individual sensors increase, the sheer volume and complexity of available data sets will complicate management and use of the rapidly growing archive of earth imagery. Mining this vast data resource for images that provide the necessary information for climate change studies becomes more difficult as more sensors are launched and more imagery is obtained.

Emerson, Charles W.↗

DOE OSTI2024

Dataset for "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models" Willard et al. (2024)

This data release provides all data and code used in the paper " "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models" Willard et al. (2024)" to model stream temperature, evaluate, and assess results. The associated manuscript explores current open questions in prediction in ungauged and unmonitored basins concerning top-down versus bottom-up approaches, tradeoffs between data available and input requirements, and the appropriate representation of catchment attributes as inputs to deep learning models. Modeling was done primarily with long short-term memory (LSTM) models, and stream site coverage spans 1362 locations across the conterminous United States. The data is organized into these items items:Code repository and data for the paper " "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models" Willard et al. (2024)".Code: stream_temp_ml_regionalization.zip contains the code repositoryData to run the code: - data_dir.zip -- contains all files that should be moved to the "DATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- metadata_dir.zip -- contains all files that should be moved to the "METADATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- error_analysis_attribute_and_groundwater_dir.zip - workflows for the extended error analysis by stream attribute and groundwater influenceData produced by the code and used in the paper:- outputs_dir.zip - contains model output and results (outputs_dir/results), model weights (outputs_dir/models), and all other outputs used for the paper including feature importances.To cite this code, please use the following BibTeX or MLA entries:bibtex:@misc{willard2024streamdata, author = {Jared Willard and Fabio Ciulla and Helen Weierbach and Vipin Kumar and Charuleka Varadharajan}, title = {Dataset for "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models"}, year = {2024}, doi = {10.15485/2448016}, publisher = {ESS-DIVE Repository}, url = {https://doi.org/10.15485/2448016}}MLA: Willard, Jared, et al. Dataset for "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models". 2024. ESS-DIVE Repository, doi:10.15485/2448016.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2012

Globally Gridded Satellite (GridSat) Observations for Climate Studies

Geostationary satellites have provided routine, high temporal resolution Earth observations since the 1970s. Despite the long period of record, use of these data in climate studies has been limited for numerous reasons, among them: there is no central archive of geostationary data for all international satellites, full temporal and spatial resolution data are voluminous, and diverse calibration and navigation formats encumber the uniform processing needed for multi-satellite climate studies. The International Satellite Cloud Climatology Project set the stage for overcoming these issues by archiving a subset of the full resolution geostationary data at approx.10 km resolution at 3 hourly intervals since 1983. Recent efforts at NOAA s National Climatic Data Center to provide convenient access to these data include remapping the data to a standard map projection, recalibrating the data to optimize temporal homogeneity, extending the record of observations back to 1980, and reformatting the data for broad public distribution. The Gridded Satellite (GridSat) dataset includes observations from the visible, infrared window, and infrared water vapor channels. Data are stored in the netCDF format using standards that permit a wide variety of tools and libraries to quickly and easily process the data. A novel data layering approach, together with appropriate satellite and file metadata, allows users to access GridSat data at varying levels of complexity based on their needs. The result is a climate data record already in use by the meteorological community. Examples include reanalysis of tropical cyclones, studies of global precipitation, and detection and tracking of the intertropical convergence zone.

Knapp, Kenneth R.↗

DOE OSTI2020

HarDWR - Raw Water Rights Records

A dataset within the Harmonized Database of Western U.S. Water Rights (HarDWR). For a detailed description of the database, please see the meta-record v2.0. Changelog v2.0 - Switched source data from collecting records from each state independently to using the WestDAAT dataset v1.0 - Initial public release Description In order to hold a water right in the western United States, an entity, (e.g., an individual, corporation, municipality, sovereign government, or non-profit) must register a physical document with the state's water regulatory agency. State water agencies each maintain their own database containing all registered water right documents within the state, along with relevant metadata such as the point of diversion and place of use of the water. All western U.S. states have digitized their individual water rights databases, as well as geospatial data defining the areas in which water rights are managed. Each state maintains and provides their own water rights data in accordance with individual state regulations and standards. In addition, while all states make their water rights publicly available, each provides their records in unique formats, meaning that file types, field availability, and terms vary from state to state. This leads to additional challenges to managing resources which cross state lines, or conducting consistent multi-state water analyses. For the first version of HarDWR, we collected the water rights databases from 11 Western States of the United States. In order to preform regional analyses with the collected data, the raw records had to be harmonized into one single format. The Water Data Exchange (WaDE) is a program dedicated to the sharing of water-related data for the Western U.S. in a singular consistent format. Created by the Western States Water Council (WSWC) to facilitate the collection and dissemination of water data among WSWC's member states and the public, WaDE provides an important service for those interested in water resource planning and management in their focus region. Of the services which WaDE provides, the one of the most interesting is the WestDAAT dataset, which is a collection of water rights data provided by the 18 WSWC member states that have been standardized into a single format, much like we had done on a more limited scale with HarDWR v1. For this version of HarDWR we decided to use WestDAAT, specifically a snapshot created in Feburary 2024, as our water rights source data. A full explanation of the benefits gained from this switch can be found in the description of the updated Harmonized Water Rights Records v2.0, but in short it has allowed us to focus more of our efforts on answering research questions and gaining a more realistic understanding of how water rights are allocated. For more information on how the data for WestDAAT was collected, please see the WaDE data summary. Terms of Use While WaDE works directly with the state agencies to collect and standardize the water rights records, the ultimate authority for the water rights data remains the individual states. Each state, and their respective water right authorities, have made their water right records available for non-commercial reference uses. In addition, the states make no guarantees as to the completeness, accuracy, or timeliness of their respective databases, let alone the modifications which we, the authors of this paper, have made to the collected records. None of the states should be held liable for using this data outside of its intended use. As several of the states update their water rights databases daily, the information provided here is not the latest possible, and should not be used for legal purposes. WestDAAT itself has irregular updates. Additional questions about the data the source states provided should be directed to the respective state agencies (see methods.csv and organization.csv files described below). In addition, although data was presented here was not collected directly from the states, several states requested specifically worked disclaimers when sharing their data. These disclaimers are included here as an acknowledgement from where the water rights data is primarily sourced. Colorado: "The data made available here has been modified for use from its original source, which is the State of Colorado. THE STATE OF COLORADO MAKES NO REPRESENTATIONS OR WARRANTY AS TO THE COMPLETENESS, ACCURACY, TIMELINESS, OR CONTENT OF ANY DATA MADE AVAILABLE THROUGH THIS SITE. THE STATE OF COLORADO EXPRESSLY DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED, INCLUDING ANY IMPLIED WARRANTIES OF MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. The data is subject to change as modifications and updates are complete. It is understood that the information contained in the Web feed is being used at one's own risk." Montana: "The Montana State Library provides this product/service for informational purposes only. The Library did not produce it for, nor is it suitable for legal, engineering, or surveying purposes. Consumers of this information should review or consult the primary data and information sources to ascertain the viability of the information for their purposes. The Library provides these data in good faith but does not represent or warrant its accuracy, adequacy, or completeness. In no event shall the Library be liable for any incorrect results or analysis; any direct, indirect, special, or consequential damages to any party; or any lost profits arising out of or in connection with the use or the inability to use the data or the services provided. The Library makes these data and services available as a convenience to the public, and for no other purpose. The Library reserves the right to change or revise published data and/or services at any time." Oregon: "This product is for informational purposes and may not have been prepared for, or be suitable for legal, engineering, or surveying purposes. Users of this information should review or consult the primary data and information sources to ascertain the usability of the information." File Descriptions The unmodified February, 2024 WestDAAT snapshot is composed of nine files. Below is a brief description of each file, as well as how they were utilized for HarDWR. WaDEDataDictionaryTerms.xlsx: As the file's name implies, this is a data dictionary for all of the below named files. This file describes the column names for each of the following files, with the exception of citation.txt which does not have any columns. The descriptions for each file are divided by tab,with the same name as their associated file, within this document. allocationamount.csv: The "main" file of the group, it contains the water right records for each state. Of particular note, each water right is broken down into one or more water allocations. Allocations may be withdrawn from one or more locations, or even multiple allocations associated with a particular location. This is a more subtle and realistic representation of how water is used than what was available in the first version of HarDWR. For the records from some states, this can mean that multiple allocations listed under a single right will appear as rows within this file. citation.txt: A combination of contact information for WaDE personnel, disclaimer about how the data should be used, and guidelines for citing WestDAAT. methods.csv: A file describing the source and method by which WaDE collected water rights data from each state. organization.csv: A file listing the water rights authoritative agencies for each state. sites.csv: This file provides the geographic, and other descriptors, of the physical location of allocations, called 'sites'. To reiterate, it is possible for one allocation to be associated with multiple sites, as well as one site to be associated with multiple allocations. The two descriptors which we were most interested in where the site's coordinates, as well as whether the site was classified as a Point of Diversion (POD) or a Place of Use (POU). As a general rule, PODs are geographic points, while POUs are areas typically represented as property boundaries or irregularly shaped polygons. sites_pouGeometry.csv: For those allocations with a POU site, this file contains the defining points for the associated polygons. variables.csv: A file describing the units in which an allocation's water amount is reported within WestDAAT. This information is essentially a repeat of the 'AllocationFlow_CFS' and 'AllocationVolume_AF' columns within allocationamount.csv, at least for our purposes. watersources: This file describes the source of water from which each site extracts from. For our purposes, this table was used to determine whether the water came from Surface Water, Groundwater, or Unspecified Water.

Lisk, Matthew↗

DOE OSTI2020

Processed data from the Building Management System for the System Engineering Building.

The dataset spans November 2018 to May 2020 and includes time-series measurements corresponding to supply and return temperatures of air and water, air, hot water and cold water flow rates, energy and power consumption, set-points etc. as a single CSV file. In addition to the measurements, a metadata .json file, and a .ttl file to visualize the data as per BRICK schema are also included.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2026

Optimization performance, fidelity, and cost: SIAM VQE

This dataset contains files storing results from classically-simulated quantum subroutines within a dynamical mean-field theory workflow, and jupyter notebooks processing the data in these files to generate plots. The files store: (1) Results from variational quantum eigensolver (VQE) simulations searching for optimal parameters allowing parametrized quantum circuits to prepare approximations to ground states of different Anderson impurity models (AIMs) (2) Results from simulations of a quantum Lanczos algorithm (QLA) estimating the Lanczos coefficients defining the continued-fraction representation of an (AIM) Green’s function Description: Any file named vqe_gs_results* stores approximations to the ground state and energy of a given AIM estimated using three different methods: (1) Numerical diagonalization (2) Ideal VQE simulation (3) VQE simulation with sampling noise For each VQE simulations metadata about the optimization (optimization results plus number of quantum circuits that would have been executed on real hardware) is also stored. Any file named qla_dos_results* estimations for the Lanczos coefficients defining the Green’s function of an AIM. The stored estimations are achieved using different methods: (1) Numerical Lanczos algorithm from initial states obtained from numerical diagonalization (2) Simulated quantum Lanczos algorithm from initial states prepared from parametrized quantum circuits yielded by corresponding ideal and noisy VQE subroutines. The dataset is used and described in M. Karabin et al., "Quantum solver for single-impurity Anderson models with particle-hole symmetry", Phys. Rev. Research 8, 033066 (2026). DOI: https://doi.org/10.1103/7ys3-tl4l

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2024

Refractiveindex.info database of optical constants

Abstract We introduce the refractiveindex.info database, a comprehensive open-source repository containing optical constants for a wide array of materials, and describe in detail the underlying dataset. This collection, derived from a meticulous compilation of data sourced from peer-reviewed publications, manufacturers’ datasheets, and authoritative texts, aims to advance research in optics and photonics. The data is stored using a YAML-based format, ensuring integrity, consistency, and ease of access. Each record is accompanied by detailed metadata, facilitating a comprehensive understanding and efficient utilization of the data. In this descriptor, we outline the data curation protocols and the file format used for data records, and briefly demonstrate how the data can be organized in a user-friendly fashion akin to the books in a traditional library.

43 PARTICLE ACCELERATORS↗

DOE OSTI2025

Electrical Resistivity Tomography data from 2016 to 2018 at the Lower Montane site in the East River Watershed, Colorado

This dataset contains time-lapse Electrical Resistivity Tomography (ERT) data along a transect located on the northeast-facing hillslope at the lower montane site (Pumphouse site) in the upper East River Watershed. The monitoring dataset covers the period from November 2, 2016, to August 6, 2018. In addition, the archive also contains a baseline dataset from October 9, 2016. The ERT transect consisted of 128 electrodes with an electrode spacing of 1.25 m. The acquisition system was located in the middle of the transect, about 50 m on one side, and included an MPT (Multi-Phase Technologies) ERT system, a mini computer, and batteries with solar panels. Acquisition occurred daily under normal circumstances. The first 16 electrodes (from the upper end of the transect) could not be used after the cable was damaged during the 2017–2018 winter. Also, due to multiple failures in the power system, the temporal resolution of the data is much lower in 2018 compared to 2016 and 2017. The data have been processed and used in Dafflon et al., 2023, and the baseline dataset was used in Falco et al., 2019 (see reference list). This archive contains the measurements (ER.zip containing csv files) for each of the 326 acquisition times and a filtered version where only electrodes 17 to 128 are included (ERT_sm.zip containing csv files). The archive also contains the baseline dataset and two acquisitions with full reciprocals (ERT_RB.zip containing csv files), as well as all the raw MPT files (ERT_raw_MTP.zip). The geometry (electrode position and elevation) is provided in Universal Transverse Mercator (UTM) 13N Geoid2012AB in the file named ERT_Location.csv. The archive contains 1 *.csv data files, four *.zip files, and three metadata *.csv files. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Vanderbilt Alumni Hall

This dataset includes processed and raw data from the Vanderbilt Alumni Hall Building (VAH) located on the University of Vanderbilt Campus in Nashville, TN. VAH is a mixed use commercial building (LEED Gold Certified) that consists of classrooms, meeting rooms, office rooms, conference rooms, an exercise room, and others. This dataset spans one year of time-series data from 2019 and includes measurements corresponding to whole building electrical power, thermal power, zone level thermal power, set-point, indoor temperature, humidity, water and air side supply and return temperatures and flow rates as CSV files. In addition to the measurements, a metadata .json file, and a .ttl file to visualize the data as per BRICK schema are also included.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2022

Hvac: Removing I/O Bottleneck for Large-Scale Deep Learning Applications

Scientific communities are increasingly adopting deep learning (DL) models in their applications to accelerate scientific discovery processes. However, with rapid growth in the computing capabilities of HPC supercomputers, large-scale DL applications have to spend a significant portion of training time performing I/O to a parallel storage system. Previous research works have investigated optimization techniques such as prefetching and caching. Unfortunately, there exist non-trivial challenges to adopting the existing solutions on HPC supercomputers for large-scale DL training applications, which include non-performance and/or failures at extreme scale, lack of portability and generality in design, complex deployment methodology, and being limited to a specific application or dataset. To address these challenges, we propose High-Velocity AI Cache (HVAC), a distributed read-cache layer that targets and fully exploits the node-local storage or near node-local storage technology. HVAC seamlessly accelerates read I/O by aggregating node-local or near node-local storage, avoiding metadata lookups and file locking while preserving portability in the application code. We deploy and evaluate HVAC on 1,024 nodes (with over 6000 NVIDIA V100 GPUS) of the Summit supercomputer. In particular, we evaluate the scalability, efficiency, accuracy, and load distribution of HVAC compared to GPFS and XFS-on-NVMe. With four different DL applications, we observe an average 25 % performance improvement atop GPFS and 9% drop against XFS-on-NVMe, which scale linearly and are considered the performance upper bound. We envision HVAC as an important caching library for upcoming HPC supercomputers such as Frontier.

Khan, Awais↗

DOE OSTI2025

High-Resolution Computed Tomography Dataset of Mount Simon Sandstone

The Illinois Basin is a critical structure for subsurface energy related activities and their implementation in the United States. The Mount Simon Sandstone has been identified as a storage target for permanent and transient storage of fluids in the basin. Known for its exceptional thickness, depth, porosity, and sealing properties of overlying formations, this saline reservoir is crucial for long-term subsurface energy efforts. We present an extensive Computed Tomography (CT) dataset on a high porosity and permeability zone in the lower Mount Simon Sandstone available on the Energy Data eXchange® (EDX). This publicly accessible database comprises over 500 GB of high-resolution CT scans of six core samples, with resolutions ranging from 14.8 µm to 0.7 µm per pixel. The scans include both dry sandstone samples and those saturated with multiple fluids, allowing for comparative analyses across different conditions and resolutions. Coarser scans capture the bedding structure of the sandstone, while finer resolutions reveal detailed pore infill and throat characteristics. Metadata on location, depth, and saturation state enhance usability, enabling quick identification and cross-sample comparisons. By providing a robust resource for research and collaboration, the database contributes to domestic energy advancement by supporting continued progress in the use of the subsurface for energy solutions.

characterization↗

DOE OSTI2021

The future low-temperature geochemical data-scape as envisioned by the U.S. geochemical community

Data sharing benefits the researcher, the scientific community, and the public by allowing the impact of data to be generalized beyond one project and by making science more transparent. However, many scientific communities have not developed protocols or standards for publishing, citing, and versioning datasets. One community that lags in data management is that of low-temperature geochemistry (LTG). This paper resulted from an initiative from 2018 through 2020 to convene LTG and data scientists in the U.S. to strategize future management of LTG data. Through webinars, a workshop, a preprint, a townhall, and a community survey, the group of U.S. scientists discussed the landscape of data management for LTG – the data-scape. Currently this data-scape includes a “street bazaar” of data repositories. This was deemed appropriate in the same way that LTG scientists publish articles in many journals. The variety of data repositories and journals reflect that LTG scientists target many different scientific questions, produce data with extremely different structures and volumes, and utilize copious and complex metadata. Nonetheless, the group agreed that publication of LTG science must be accompanied by sharing of data in publicly accessible repositories, and, for sample-based data, registration of samples with globally unique persistent identifiers. LTG scientists should use certified data repositories that are either highly structured databases designed for specialized types of data, or unstructured generalized data systems. Recognizing the need for tools to enable search and cross-referencing across the proliferating data repositories, the group proposed that the overall data informatics paradigm in LTG should shift from “build data repository, data will come” to “publish data online, cybertools will find”. Funding agencies could also provide portals for LTG scientists to register funded projects and datasets, and forge approaches that cross national boundaries. Finally, the needed transformation of the LTG data culture requires emphasis in student education on science and management of data.

58 GEOSCIENCES↗

DOE OSTI2025

DEDUPKV: A Space-Efficient and High-Performance Key-Value Store via Fine-Grained Deduplication

Log-Structured Merge Tree (LSM-tree) based key-value stores excel in write-intensive environments but suffer from data duplication, consuming up to 49% of storage space in LSM-tree-based key-value store deployments. Traditional solutions like compression and coarse-grained file system-level deduplication introduce overhead or have limited effectiveness. In this study, we propose DedupKV, a fine-grained deduplication framework tailored for LSM-tree, maximizing data reduction efficiency while minimizing write stalls and read overheads. DedupKV features three key innovations: (1) FLUSH-integrated inline deduplication, which removes duplicates during memory-to-storage writes; (2) WAL file-based offline deduplication, repurposing write-ahead logs to avoid double writes; and (3) elastic execution, dynamically balancing inline and offline deduplication based on memory pressure and workload intensity. Additionally, dynamic granularity management reduces deduplication metadata overhead. We implemented these four ideas in RocksDB for the first time and conducted experiments in a Linux environment. Our evaluation shows that WAL file-based offline deduplication and DedupKV outperform BlobDB by 33% and 23%, respectively, in write-heavy workloads, while reducing write amplification by 1.2 ×, 2 ×, and 1.6 × for real KV datasets.

Jamil, Safdar [Sogang University]↗