Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Digital twins and deep learning segmentation of defects in monolayer MX 2 phases

Developing methods to understand and control defect formation in nanomaterials offers a promising route for materials discovery. Monolayer MX 2 phases represent a particularly compelling case for defect engineering of nanomaterials due to the large variability in their physical properties as different defects are introduced into their structure. However, effective identification and quantification of defects remain a challenge even as high-throughput scanning transmission electron microscopy methods improve. This study highlights the benefits of employing first principles calculations to produce digital twins for training deep learning segmentation models for defect identification in monolayer MX 2 phases. Around 600 defect structures were obtained using density functional theory calculations, with each monolayer MX 2 structure being subjected to multislice simulations for the purpose of generating the digital twins. Several deep learning segmentation architectures were trained on this dataset, and their performances evaluated under a variety of conditions such as recognizing defects in the presence of unidentified impurities, beam damage, grain boundaries, and with reduced image quality from low electron doses. Further, this digital twin approach allows benchmarking different deep learning architectures on a theory dataset, which enables the study of defect classification under a broad array of finely controlled conditions. It thus opens the door to resolving the underpinning physical reasons for model shortcomings and potentially chart paths forward for automated discovery of materials defect phases in experiments.

36 MATERIALS SCIENCE↗
DOE OSTI2025

SCALE Input and Result Files Supporting SCALE Inventory and Reactivity Analysis of the gFHR

This dataset contains input and result files of computational simulations with the SCALE code system. The simulations cover radionuclide inventory and reactivity analyses of a fluoride salt-cooled high temperature pebble-bed reactor (PB-FHR), specifically the generic FHR benchmark. Users wanting to reproduce results from this dataset are required to obtain a license to the SCALE code system for which details on the distribution can be found here: https://www.ornl.gov/scale/releases

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Crystal Diffraction Prediction and Partiality estimation using Gaussian basis functions

A small dataset with good data quality, useful as a benchmark. There exists a two-fold indexing ambiguity, so when processing with CrystFEL you will need to do: ambigator -o disambiguated.stream -y m-3 -w m-3m -j 72 ambiguous.stream Please check the README file for more information about the dataset. ----- Begin unit cell ----- CrystFEL unit cell file version 1.0 lattice_type = cubic centering = I a = 103.40 A b = 103.40 A c = 103.40 A al = 90.00 deg be = 90.00 deg ga = 90.00 deg ; Please note: this is the target unit cell. ; The actual unit cells produced by indexing depend on many other factors. ----- End unit cell -----

Cydia pomonella granulovirus hull protein↗
DOE OSTI2022

Development of a Annual Air Handling Unit Fault Dataset for FDD Tools: Lessons Learned and Considerations for FDD Developers

As energy management and information systems (e.g., automated fault detection and diagnostics [AFDD] tools) become more prevalent in the commercial building stock, it is important to determine the effectiveness of these technologies by benchmarking their performance. The authors have been working to develop the largest publicly available dataset of HVAC fault data for performance benchmarking applications, covering the most common HVAC systems and designs including chiller plants, rooftop packaged units, dual duct air handling units and single duct air handling units. This study covers the development, modeling, and validation of a synthetic fault dataset for a single duct air handling unit (AHU), one of the most common HVAC configurations found in the commercial building stock. Despite this being a common system, real-world time series data are scarce and usually do not span a wide range of weather conditions. Due to this limitation, a detailed AHU model was employed to carry out annual simulations of numerous common sensor and mechanical faults, which were then validated by comparing their effects on system performance to expected symptoms. We summarize the nature of each fault and their impacts under different weather and operation conditions. Finally, we highlight considerations for FDD developers that may want to use this dataset to assess their algorithms’ performance and their improvement over time.

Casillas, Armando↗
DOE OSTI2026

QPatLib v1.0 — Measurement-based quantum simulation Pauli string unitary pattern collections

This Zenodo record accompanies the paper “Scalable Measurement-Based Quantum Simulation Patterns for Benchmarking” arXiv.2605.12502 and provides QPatLib v1.0 measurement-pattern datasets in human-readable JSONL together with a ZIP archive of OpenQASM 3.0 circuits used for validation and reproducibility. The patterns and circuits implement Pauli string unitaries for benchmark cases. Cases include all possible string combinations for less than 6 qubits and strings used in Hamiltonians for certain diatomic molecules for 6 or more qubits. Format: Each pattern_*.jsonl file is containins measurement patterns for all subsets for a given model/instance and subset strategy: it begins with a preamble containing model metadata, subset definitions, provenance, and (when feasible) full-pattern test results, followed by one pattern entry per subset. Each subset entry includes a required pattern_ascii field storing the measurement pattern in the measurement-calculus/Graphix standard with signal shifting, written left-to-right in the canonical order nodes → edges → measurements (with signal dependencies) → byproduct corrections (X/Z). The circuits are included as circuit_files.zip. Patterns in this record were validated against the corresponding circuits and checked for causal flow. Codes for generating these patterns can be found at QPatLib repository on Github

Graphix↗
DOE OSTI2026

Thermal property characterization of phase change materials in building applications: A systematic review of fundamentals, recent progress, and future directions

Phase change materials (PCMs) can reduce building peak loads and enable demand-responsive thermal energy storage (TES), but their deployment depends on reliable measurement and interpretation of thermal properties across laboratory, intermediate, and application scales. Here, this review systematically examines characterization methods, testing protocols, and recent advances for neat PCMs and PCM composites, emphasizing thermal conductivity, enthalpy-related properties (phase change temperature, latent heat, specific heat), and cycling stability. For thermal conductivity, we compare steady-state and transient techniques and note limitations when phase transition and contact resistance affect measurements. For enthalpy–temperature characterization, we discuss differential scanning calorimetry together with intermediate- and bulk-scale methods, including T-history, heat flow meter testing, and three-layer calorimetry (3LC), to generate application-relevant enthalpy–temperature profiles. Cycling stability is organized into four experimental families: thermoelectric–air, fully thermoelectric, water-bath, and in situ chamber approaches, with attention to separating reversible supercooling from true degradation such as phase segregation. We highlight emerging noncontact diagnostics, including infrared thermography and embedded sensing, for spatially resolved validation and multiscale interpretation. Finally, we review the growing use of AI and machine learning for property prediction, inverse characterization from experimental signals, and real-time state estimation in building-integrated TES. Key needs include harmonized protocols, interlaboratory benchmarking, uncertainty reporting, and metadata-rich datasets to accelerate reproducible PCM qualification for grid-flexible buildings.

AI↗
DOE OSTI2022

Using artificial intelligence to detect human errors in nuclear power plants: A case in operation and maintenance

Human error (HE) is an important concern in safety-critical systems such as nuclear power plants (NPPs). HE has played a role in many accidents and outage incidents in NPPs. Despite the increased automation in NPPs, HE remains unavoidable. Hence, the need for HE detection is as important as HE prevention efforts. In NPPs, HE is rather rare. Hence, anomaly detection, a widely used machine learning technique for detecting rare anomalous instances, can be repurposed to detect potential HE. In this study, we develop an unsupervised anomaly detection technique based on generative adversarial networks (GANs) to detect anomalies in manually collected surveillance data in NPPs. More specifically, our GAN is trained to detect mismatches between automatically recorded sensor data and manually collected surveillance data, and hence, identify anomalous instances that can be attributed to HE. We test our GAN on both a real-world dataset and an external dataset obtained from a testbed, and we benchmark our results against state-of-the-art unsupervised anomaly detection algorithms, including one-class support vector machine and isolation forest. Our results show that the proposed GAN provides improved anomaly detection performance. Our study is promising for the future development of artificial intelligence based HE detection systems.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

$\mathrm{COBRA}$ improves the completeness and contiguity of viral genomes assembled from metagenomes

Viruses are often studied using metagenome-assembled sequences, but genome incompleteness hampers comprehensive and accurate analyses. Contig Overlap Based Re-Assembly (COBRA) resolves assembly breakpoints based on the de Bruijn graph and joins contigs. Here we benchmarked COBRA using ocean and soil viral datasets. COBRA accurately joined the assembled sequences and achieved notably higher genome accuracy than binning tools. From 231 published freshwater metagenomes, we obtained 7,334 bacteriophage clusters, ~83% of which represent new phage species. Notably, ~70% of these were circular, compared with 34% before COBRA analyses. We expanded sampling of huge phages (≥200 kbp), the largest of which was curated to completion (717 kbp). Improved phage genomes from Rotsee Lake provided context for metatranscriptomic data and indicated the in situ activity of huge phages, whiB-encoding phages and cysC- and cysH-encoding phages. COBRA improves viral genome assembly contiguity and completeness, thus the accuracy and reliability of analyses of gene content, diversity and evolution.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Explainable machine learning for incipient anomaly detection in compact molten salt heat exchanger with overlapping feature distributions

High-temperature molten salt-cooled reactors (MSCRs) are a promising next-generation nuclear technology option, offering efficient power conversion and inherent safety features. However, the reliability of these systems depends on the robust operation of heat exchangers (HXs), which are susceptible to failure due to temperature gradients and channel plugging caused by fluid freezing. Conventional monitoring methods, relying on inlet and outlet measurements, lack the spatial resolution needed to detect early-stage faults. We propose a novel design of a compact salt-to-salt matrix-type HX design consisting of interleaved arrays of parallel tubes, with integrated synthetic fiber optic distributed temperature sensing (DTS) to enable localized detection of incipient faults. To evaluate performance of this design, we generate high-fidelity synthetic data using heat transfer computational modeling to simulate channel plugging, and introduce sensor noise for realistic modeling of measurements. The dataset comprises of 97% normal operation and 3% anomaly cases, with each anomaly class representing 1% of the data. These early anomalies result in overlapping temperature profiles between normal and faulty channels, producing a non-separable dataset that challenges traditional classification techniques. We benchmark eight supervised machine learning (ML) models and demonstrate that XGBoost achieves the highest performance. To improve transparency, we develop an explainability framework combining Shapley values and partially ordered sets (POSETs) to quantify and structurally analyze feature importance. This approach identifies both dominant predictors and ambiguous feature relationships, enhancing trust and interpretability. Our results highlight the potential of combining DTS and explainable ML with intelligent feature selection to improve predictive maintenance and ensure operational resilience in advanced nuclear systems.

Prantikos, Konstantinos [Argonne National Laborato↗
DOE OSTI2024

GradDFT. A software library for machine learning enhanced density functional theory

Density functional theory (DFT) stands as a cornerstone method in computational quantum chemistry and materials science due to its remarkable versatility and scalability. Yet, it suffers from limitations in accuracy, particularly when dealing with strongly correlated systems. To address these shortcomings, recent work has begun to explore how machine learning can expand the capabilities of DFT: an endeavor with many open questions and technical challenges. In this work, we present GradDFT a fully differentiable JAX-based DFT library, enabling quick prototyping and experimentation with machine learning-enhanced exchange–correlation energy functionals. GradDFT employs a pioneering parametrization of exchange–correlation functionals constructed using a weighted sum of energy densities, where the weights are determined using neural networks. Moreover, GradDFT encompasses a comprehensive suite of auxiliary functions, notably featuring a just-in-time compilable and fully differentiable self-consistent iterative procedure. To support training and benchmarking efforts, we additionally compile a curated dataset of experimental dissociation energies of dimers, half of which contain transition metal atoms characterized by strong electronic correlations. The software library is tested against experimental results to study the generalization capabilities of a neural functional across potential energy surfaces and atomic species, as well as the effect of training data noise on the resulting model accuracy.

Chemistry↗
DOE OSTI2025

GRinding Automated Classification Engine

This work is an ML-driven framework for automated surface analysis of microscopy images. We create a training dataset by imaging stainless steel samples to benchmark four developed deep neural network architectures. These models, based on a YOLOv8n-cls backend, integrate image features and process metadata using various fusion methods to distinguish between acceptable and unacceptable surface finishes. This code is associated with publication "Classifying Alloy Surface Preparation Quality with Metadata-Infused Machine Learning for Rapid Alloy Discovery" for project APEX LDRD-ER (25-ERD-039)

Gongora, AldairE [Lawrence Livermore National Labo↗
DOE OSTI2021

Compaction of crushed salt for safe containment – overview of the KOMPASS project

Abstract. In Germany, rock salt formations are possible host rock candidates for a repository for heat-emitting radioactive waste. The safety concept of a repository in salt bases on a multibarrier system consisting mainly of the geological barrier salt and geotechnical seals ensuring safe containment. Crushed salt will be used for backfilling of cavities and sealing measures in drifts and shafts due to its favourable properties and its easy availability (mined-off material). The creep of the rock salt leads to crushed salt compaction with time. Thereby, the crushed salts' porosity is reduced from the initial porosity of 30 %–40 % to a value comparable to the porosity of undisturbed rock salt (≤1 %). In such low porosity ranges, technical impermeability is assumed. The compaction behaviour of crushed salt is rather complex and involves several coupled THM processes (Kröhn et al., 2017; Hansen et al., 2014). It is influenced by internal properties like humidity and grain size distribution, as well as boundary conditions such as temperature, compaction rate or stress state. However, the current process understanding has some important gaps referring to the material behaviour, experimental database and numerical modelling. It needs to be extended and validated, especially in the low porosity range. The objective of the KOMPASS project was development of methods and strategies for the reduction of deficits in the prediction of crushed salt compaction leading to an improvement of the prognosis quality. Key results are as follows (KOMPASS Phase 1, 2020): selection of an easily available and permanently producible synthetic crushed salt mixture, acting as a reference material for generic investigations; development and proof of different techniques for producing pre-compacted samples for further investigations; establishment of a tool of microstructure investigation methods to demonstrate the comparability of grain structures of pre-compacted samples with in-situ compacted material for future investigations; execution of various laboratory experiments using pre-compacted samples, e.g. long-term creep tests which deliver reliable information about time- and stress-dependent compaction behaviour; development of a complex experimental investigation strategy to derive necessary model parameters considering individual functional dependencies. Its technical feasibility was successfully verified; benchmarking with various existing numerical models using datasets from three different triaxial long-term tests. The result was not entirely satisfactory; however, the number of influencing factors is small and further validation work has to be done. Overall, the KOMPASS project has made significant progress in the approaches to solving the outstanding question, building the basis for further investigations.

Friedenberg, Larissa↗
NASA NTRSDate not supplied

A14H-04 Analysis of Simulated and Observed Trends in Global Surface PM2.5 and Aerosol Optical Properties from 1958 to 2018 Using the NASA GEOSCCM

Modeling of long-term trends of aerosols and their properties is important for constraining aerosol-climate forcing, and for characterizing changes in particulate matter pollution speciation and exposure. Here we study global and regional long-term trends in surface fine particulate matter (PM ) and aerosol optical properties for 60 years from 1958 to 2018, using simulations performed with the NASA Goddard Earth Observing System Chemistry Climate Model (GEOSCCM), and evaluate the model hindcast with observations for the last three decades. Comparing the modeled aerosols with a diverse set of observations helps interpret observed and simulated trends, and serves as a benchmark for future GEOSCCM developments and input datasets improvements. We first characterize modeled global and regional temporal changes in surface PM and its components, aerosol optical depth (AOD) and single scattering albedo (SSA) and we interpret their link with emissions drivers. We then compare modeled surface PM with ground-based observations from monitoring networks and with global reconstructed PM datasets from observations-model data fusion. Total AOD is compared with long-term satellite measurements from the Moderate Resolution Imaging Spectroradiometer (MODIS) and measurements from the ground-based Aerosol Robotic Network (AERONET). Additional aerosol optical properties, such as absorption and scattering coefficients, are evaluated with ground-based observations from the Global Atmosphere Watch (GAW) records.

aerosols↗
DOE OSTI2021

Expanding standards in viromics: in silico evaluation of dsDNA viral genome identification, classification, and auxiliary metabolic gene curation

Viruses influence global patterns of microbial diversity and nutrient cycles. Though viral metagenomics (viromics), specifically targeting dsDNA viruses, has been critical for revealing viral roles across diverse ecosystems, its analyses differ in many ways from those used for microbes. To date, viromics benchmarking has covered read pre-processing, assembly, relative abundance, read mapping thresholds and diversity estimation, but other steps would benefit from benchmarking and standardization. Here we use in silico-generated datasets and an extensive literature survey to evaluate and highlight how dataset composition (i.e., viromes vs bulk metagenomes) and assembly fragmentation impact (i) viral contig identification tool, (ii) virus taxonomic classification, and (iii) identification and curation of auxiliary metabolic genes (AMGs). The in silico benchmarking of five commonly used virus identification tools show that gene-content-based tools consistently performed well for long (≥3 kbp) contigs, while k -mer- and blast-based tools were uniquely able to detect viruses from short (≤3 kbp) contigs. Notably, however, the performance increase of k -mer- and blast-based tools for short contigs was obtained at the cost of increased false positives (sometimes up to ~5% for virome and ~75% bulk samples), particularly when eukaryotic or mobile genetic element sequences were included in the test datasets. Furthermore, for viral classification, variously sized genome fragments were assessed using gene-sharing network analytics to quantify drop-offs in taxonomic assignments, which revealed correct assignations ranging from ~95% (whole genomes) down to ~80% (3 kbp sized genome fragments). A similar trend was also observed for other viral classification tools such as VPF-class, ViPTree and VIRIDIC, suggesting that caution is warranted when classifying short genome fragments and not full genomes. Finally, we highlight how fragmented assemblies can lead to erroneous identification of AMGs and outline a best-practices workflow to curate candidate AMGs in viral genomes assembled from metagenomes. Together, these benchmarking experiments and annotation guidelines should aid researchers seeking to best detect, classify, and characterize the myriad viruses ‘hidden’ in diverse sequence datasets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

ORBIT-2 Dataset for Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling

This dataset release corresponds to the work conducted in ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling, where large-scale AI methods were applied to improve climate and weather resolution. The collection integrates four widely used, publicly available datasets: ERA5, PRISM, DAYMET, and IMERG. To prepare the data for ORBIT-2 model training and evaluation, we applied a preprocessing pipeline that generates paired low-resolution and high-resolution samples, enabling supervised downscaling experiments. The transformation from coarse to fine scales was performed using bilinear regridding, consistent with the procedures described in WeatherBench2, a community benchmark for weather and climate AI models. This dataset supports the development and evaluation of foundation models designed for weather and climate downscaling at exascale. Additional details on methodology and applications can be found in Wang et al., ORBIT-2 (arXiv:2505.04802, 2025).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multiple efforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680,000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin [Fermilab] (ORCID:0000000157000288↗
DOE OSTI2025

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multipleefforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of synthesized ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680 000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin G. [Fermilab]↗
DOE OSTI2022

Time-Based CAN IDS Paper Results Code

Modern vehicles are complex cyber-physical systems made of hundreds of electronic control units (ECUs) that communicate over controller area networks (CANs). This inherited complexity has expanded the CAN attack surface which is vulnerable to message injection attacks. These injections change the overall timing characteristics of messages on the bus, and thus, to detect these malicious messages, time-based intrusion detection systems (IDSs) have been proposed. However, time-based IDSs are usually trained and tested on low-fidelity datasets with unrealistic, labeled attacks. This makes difficult the task of evaluating, comparing, and validating IDSs. Here we detail and benchmark four time-based IDSs against the newly published ROAD dataset, the first open CAN IDS dataset with real (non-simulated) stealthy attacks with physically verified effects. We found that methods that perform hypothesis testing by explicitly estimating message timing distributions have lower performance than methods that seek anomalies in a distribution related statistic. In particular, these “distribution-agnostic” based methods outperform “distribution-based” methods by at least 55% in area under the precision-recall curve (AUC-PR). Our results expand the body of knowledge of CAN time-based IDSs by providing details of these methods and reporting their results when tested on datasets with real advanced attacks. Finally, we develop an after-market plug-in detector using lightweight hardware, which can be used to deploy the best performing IDS method on nearly any vehicle.

Moriano, Pablo [Oak Ridge National Lab. (ORNL), Oa↗