Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Development of a Benchmark Eddy Flux Evapotranspiration Dataset for Evaluation of Satellite-Driven Evapotranspiration Models Over the CONUS

A large sample of ground-based evapotranspiration (ET) measurements made in the United States, primarily from eddy covariance systems, were post-processed to produce a benchmark ET dataset. The dataset was produced primarily to support the intercomparison and evaluation of the OpenET satellite-based remote sensing ET (RSET) models and could also be used to evaluate ET data from other models and approaches. OpenET is a web-based service that makes field-delineated and pixel-level ET estimates from well-established RSET models readily available to water managers, agricultural producers, and the public. The benchmark dataset is composed of flux and meteorological data from a variety of providers covering native vegetation and agricultural settings. Flux footprint predictions were developed for each station and included static flux footprints developed based on average wind direction and speed, as well as dynamic hourly footprints that were generated with a physically based model of upwind source area. The two footprint prediction methods were rigorously compared to evaluate their relative spatial coverage. Data from all sources were post-processed in a consistent and reproducible manner including data handling, gap-filling, temporal aggregation, and energy balance closure correction. The resulting dataset included 243,048 daily and 5,284 monthly ET values from 194 stations, with all data falling between 1995 and 2021. We assessed average daily energy imbalance using 172 flux sites with a total of 193,021 days of data, finding that overall turbulent fluxes were understated by about 12% on average relative to available energy. Multiple linear regression analyses indicated that daily average latent energy flux may be typically understated slightly more than sensible heat flux. This dataset was developed to provide a consistent reference to support evaluation of RSET data being developed for a wide range of applications related to water accounting and water resources management at field to watershed scales.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Stable parallel training of Wasserstein conditional generative adversarial neural networks

In this work, we propose a stable, parallel approach to train Wasserstein conditional generative adversarial neural networks (W-CGANs) under the constraint of a fixed computational budget. Differently from previous distributed GANs training techniques, our approach avoids inter-process communications, reduces the risk of mode collapse and enhances scalability by using multiple generators, each one of them concurrently trained on a single data label. The use of the Wasserstein metric also reduces the risk of cycling by stabilizing the training of each generator. We illustrate the approach on the CIFAR10, CIFAR100, and ImageNet1k datasets, three standard benchmark image datasets, maintaining the original resolution of the images for each dataset. Performance is assessed in terms of scalability and final accuracy within a limited fixed computational time and computational resources. To measure accuracy, we use the inception score, the Fréchet inception distance, and image quality. An improvement in inception score and Fréchet inception distance is shown in comparison to previous results obtained by performing the parallel approach on deep convolutional conditional generative adversarial neural networks as well as an improvement of image quality of the new images created by the GANs approach. Weak scaling is attained on both datasets using up to 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Ecological Benchmark for Chemicals

The Ecological Benchmark Tool dataset serves as a comprehensive repository of benchmarks designed to assess ecological risks at contaminated sites. This tool facilitates the evaluation of various environmental media and contaminants, supporting regulatory compliance and ecological protection. Benchmarks are available for air, biota, sediment, soil, and surface water. The dataset also provides species-specific benchmarks for fish, plants, birds, mammals, and invertebrates. Users can select benchmark sources, media, individual chemicals, and retrieve results in tabular or spreadsheet formats for analysis. Benchmarks are derived from authoritative sources, including government agencies, scientific councils, and academic publications. The dataset supports ecological risk assessments, regulatory decision-making, and environmental planning, with tools for benchmarking against chemical thresholds, sensitive species protection, and habitat impact evaluations. This structured approach ensures a robust evaluation of ecological risks tailored to site-specific and regulatory needs.

Stewart, Debra [Oak Ridge National Laboratory (ORN↗
DOE OSTI2026

SpaceNet 9—Cross-Sensor Alignment of Optical and SAR Imagery

Precise registration of high-resolution synthetic aperture radar (SAR) and optical imagery is necessary for realizing the full potential and benefits of multimodal image analysis. However, two significant challenges presently exist. First, there is a lack of annotated datasets and benchmarks available for high-resolution SAR–optical image registration. Second, an assessment of efficient and reliable image registration methods that can precisely align these modalities is lacking. Here, we present a holistic description of the SpaceNet 9 Challenge and its results. We present a description of the dataset and baseline algorithm along with the results of the challenge, including a description of the winning algorithms. We release the SpaceNet 9 dataset along with open-sourcing the winning algorithms and baseline. The objective of SpaceNet 9 was to compute a dense displacement map that indicates the shift needed to align pixels in an optical image to the pixels in a SAR image. The challenge launched in April 2025 and was active for approximately two months. The top five solutions reduced image alignment error from approximately 34 m to under 13 m for public and private test data, with the best results obtaining a registration error of only 8.5 and 6.7 m on the public testing and private testing dataset, respectively. Usage of pretrained image matching models, robust outlier rejection with RANSAC, and estimating local displacement were common among the top solutions. The results of this challenge provide insight into high-resolution SAR–optical image registration and offer opportunities for future benchmarking in this domain. The baseline algorithm, winning solutions, and datasets are available at https://spacenet.ai/sn9-challenge/.

benchmark datasets↗
DOE OSTI2026

SCALE HTR-PROTEUS Benchmark Model

This dataset contains input and result files of computational simulations of HTR-PROTEUS benchmark with the latest version of SCALE code system. The simulations cover criticality control rod worth calculations as well as sensitivity analysis and uncertainty quantification. Users wanting to reproduce results from this dataset are required to obtain a license to the SCALE code system for which details on the distribution can be found here: https://www.ornl.gov/scale/releases

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRSDate not supplied

Creating Benchmark Data for Artificial Intelligence and Machine Learning Space Biology Research

To identify an appropriate AI/ML approach for a specific problem, the best practice is to measure algorithm performance through the benchmarking process. A scientific benchmark consists of an AI-ready dataset and a reference implementation on a specific scientific question. The NASA Science Mission Directorate (SMD) has started the “Benchmark Initiative for AI/ML to create scientific benchmark datasets in three applications: 1) scientific benchmarking, which finds the best algorithm for a specific problem; 2) application benchmarking, which measures algorithm performance against a set of parameters; and 3) system benchmarking, which evaluates performance of hardware and software architecture. Currently, there are no standardized datasets available to benchmark AI/ML algorithms in the domain of space biology. In this work, we constructed two AI/ML-ready biological datasets from experiments in space-flown mice: cellular imaging and RNA-seq. First, radiation-exposed immune cells harbor DNA damage foci that can be fluorescently marked to visualize the amount of damage following exposure to ionizing radiation. However, such large datasets are difficult to analyze visually, due to imaging inconsistencies and human bias, and classical image processing approaches can fail on imaging artifacts. AI/ML are therefore exciting alternative, providing the speed of machines and the accuracy of humans. We have made this dataset available at https://registry.opendata.aws/bps_microscopy/. Second, high-throughput nucleic acid sequencing (DNA-seq, RNA-seq) has become widespread in biomedical research due to the growing availability and affordability of these assays. However, most sequencing datasets suffer from high dimensionality and low sample count. In this work, we used a generative adversarial network to synthesize a standardized, AI-ready, publicly available benchmark dataset for space biology RNA-seq data with sufficient space-flown and ground control mouse liver samples from NASA GeneLab. This dataset is available at https://registry.opendata.aws/bps_rnaseq/. These datasets are now fully open the Space Biology community to test their favorite AI/ML approaches.

James Casaletto↗
DOE OSTI2026

VECTOR Phase 1 Dataset: CAV Trajectory and Energy Consumption Records

This dataset contains benchmark experimental data from Phase 1 of the VECTOR project, focusing on the energy impact of CAV hardware components. The dataset includes vehicle trajectory data (speed and position) and corresponding energy consumption records collected from a CAV platform equipped with lidar, cameras, onboard computation units, and communication modules. The primary objective is to quantify the baseline energy consumption attributable to sensing and computing systems, independent of any advanced cooperative control strategies. During experiments, the leading vehicle followed a predetermined velocity profile, and the following CAV mirrored this trajectory using a basic car-following control to ensure consistent driving behavior. This setup enables a reliable benchmark for assessing the energy cost introduced by onboard CDA hardware (e.g., lidar and GPU-based processing). The dataset is essential for evaluating energy baselines and supports future comparative studies involving additional cooperative strategies. ![system img](system.png) ![vector img](vector.png)

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

When does global attention help: a unified empirical study on atomistic graph learning

Graph neural networks (GNNs) are widely used as surrogates for costly experiments and first-principles simulations to study the behavior of compounds at atomistic scale, and their architectural complexity is constantly increasing to enable the modeling of complex physics. While most recent GNNs combine more traditional message passing neural networks (MPNNs) layers to model short-range interactions with more advanced graph transformers (GTs) with global attention mechanisms to model long-range interactions, it is still unclear when global attention mechanisms provide real benefits over well-tuned MPNN layers due to inconsistent implementations, features, or hyperparameter tuning. We introduce the first unified, reproducible benchmarking framework–built on HydraGNN–that enables seamless switching among four controlled model classes: MPNN, MPNN with chemistry/topology encoders, GPS-style hybrids of MPNN with global attention, and fully fused localglobal models with encoders. Using seven diverse open-source datasets for benchmarking across regression and classification tasks, we systematically isolate the contributions of message passing, global attention, and encoder-based feature augmentation. Our study shows that encoder-augmented MPNNs form a robust baseline, while fused localglobal models yield the clearest benefits for properties governed by long-range interaction effects. We further quantify the accuracycompute trade-offs of attention, reporting its overhead in memory. Together, these results establish the first controlled evaluation of global attention in atomistic graph learning and provide a reproducible testbed for future model development.

Equivariant graph neural networks↗
NASA NTRS2022

Microstructure Segmentation With Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation intersection over union accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures were trained on over 100,000 labeled microscopy images from 54 material classes. These pre-trained encoders were then embedded into multiple segmentation architectures including UNet and DeepLabV3+ to evaluate segmentation performance on created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2% reduction in relative intersection over union error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗
NASA NTRS2022

Microstructure Segmentation with Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures, including VGG, Inception, and ResNet, were trained on over 100,000 labelled microscopy images from 54 classes. These pre-trained encoders were then embedded into multiple segmentation architectures including U-Net and DeepLabV3+ to evaluate segmentation performance on newly created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2 percent reduction in relative segmentation error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗
DOE OSTI2025

Data-Driven Recommendation of Optimal Tuning Scheme for Range-Separated Hybrid Functionals in Solution-Phase UV/Vis Absorption Energy Prediction

Time-dependent density functional theory (TDDFT) combined with range-separated hybrid (RSH) functionals and a tuned range-separation parameter γ offers a computationally economical approach for high-throughput excited- state property predictions. The γ-tuning procedure in the gas phase is well established. However, no agreement on the best γ- tuning procedure has been made when considering the solvent effect with implicit solvent models like the polarizable continuum model (PCM). To answer that question, this study created a diverse dataset with 937 molecules with experimental solutionphase UV/vis absorption spectra. Three γ-tuning methods, the gasphase γ-tuning (GPγT), the partial vertical γ-tuning (PVγT), and the strict vertical γ-tuning (SVγT), were evaluated for the ωPBEh functional over the entire dataset. Additional benchmarks are done for the optimally tuned screened range-separated hybrid combined with the PCM approach (SRSH-PCM) and the solvation-mediated tuning procedure (sol-med-OT). Our findings revealed that the optimal γ-values obtained by the PVγT and the SVγT are significantly smaller than the GPγT. This trend holds consistently across all molecules in our dataset, and we explained the origin of this phenomenon. TDDFT calculations with PVγTand SVγT-tuned γ-values and default global Fock exchange fraction achieve superior performance compared to those using GPγTtuned or default γ and slightly outperform SRSH-PCM and sol-med-OT with similar or lesser computational cost. Furthermore, we found that the smaller γ-values from SVγT captured the expected 1/(εR) asymptotic behavior in the solution phase, resulting in accurate prediction of solution-phase CT excitations, consistent with the screened asymptote behavior encoded in SRSH-PCM. These results show that SVγT is the best scheme for high-throughput UV/vis absorption spectrum calculations using the ωPBEh functional from a data-driven perspective.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Strictly Enforcing Invertibility and Conservation in CNN-Based Super Resolution for Scientific Datasets

Abstract Recently, deep convolutional neural networks (CNNs) have revolutionized image “super resolution” (SR), dramatically outperforming past methods for enhancing image resolution. They could be a boon for the many scientific fields that involve imaging or any regularly gridded datasets: satellite remote sensing, radar meteorology, medical imaging, numerical modeling, and so on. Unfortunately, while SR-CNNs produce visually compelling results, they do not necessarily conserve physical quantities between their low-resolution inputs and high-resolution outputs when applied to scientific datasets. Here, a method for “downsampling enforcement” in SR-CNNs is proposed. A differentiable operator is derived that, when applied as the final transfer function of a CNN, ensures the high-resolution outputs exactly reproduce the low-resolution inputs under 2D-average downsampling while improving performance of the SR schemes. The method is demonstrated across seven modern CNN-based SR schemes on several benchmark image datasets, and applications to weather radar, satellite imager, and climate model data are shown. The approach improves training time and performance while ensuring physical consistency between the super-resolved and low-resolution data. Significance Statement Recent advancements in using deep learning to increase the resolution of images have substantial potential across the many scientific fields that use images and image-like data. Most image super-resolution research has focused on the visual quality of outputs, however, and is not necessarily well suited for use with scientific data where known physics constraints may need to be enforced. Here, we introduce a method to modify existing deep neural network architectures so that they strictly conserve physical quantities in the input field when “super resolving” scientific data and find that the method can improve performance across a wide range of datasets and neural networks. Integration of known physics and adherence to established physical constraints into deep neural networks will be a critical step before their potential can be fully realized in the physical sciences.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Coupled cluster benchmarks of large noncovalent complexes: The L7 dataset as well as DNA–ellipticine and buckycatcher–fullerene

In this work, benchmark binding energies for dispersion-bound complexes in the L7 dataset, the DNA–ellipticine intercalation complex, and the buckycatcher–C60 complex with 120 heavy atoms using a focal-point method based on the canonical form of second-order Møller−Plesset theory (MP2) and the domain based local pair natural orbital scheme for the coupled cluster with single, double, and perturbative triple excitations [CCSD(T)] extrapolated to the complete basis set (CBS) limit are reported. This work allows for increased confidence given the agreement with respect to values recently obtained using the local natural orbital CCSD(T) for L7 and the canonical CCSD(T)/CBS result for the coronene dimer (C2C2PD). Therefore, these results can be considered pushing the CCSD(T)/CBS binding benchmark to the hundred-atom scale. The disagreements between the two state-of-the-art methods, CCSD(T) and fixed-node diffusion Monte Carlo, are substantial with at least 2.0 (∼10%), 1.9 (∼5%), and 10.3 kcal/mol (∼25%) differences for C2C2PD in L7, DNA–ellipticine, and buckycatcher–C60, respectively. Such sizable discrepancy above “chemical accuracy” for large noncovalent complexes indicates how challenging it is to obtain benchmark binding interactions for systems beyond small molecules, although the three up-to-date density functionals, PBE0+D4, ωB97M-V, and B97M-V, agree better with CCSD(T) for these large systems. In addition to reporting these values, different basis sets and various CBS extrapolation parameters for Hartree–Fock and MP2 correlation energies were tested for the first time in large noncovalent complexes with the goal of providing some indications toward optimal cost effective routes to approach the CBS limit without substantial loss in quality.

Ballesteros, Francisco (ORCID:0000000325732307)↗
DOE OSTI2022

Tracking blobs in the turbulent edge plasma of a tokamak fusion device

Abstract The analysis of turbulence in plasmas is fundamental in fusion research. Despite extensive progress in theoretical modeling in the past 15 years, we still lack a complete and consistent understanding of turbulence in magnetic confinement devices, such as tokamaks. Experimental studies are challenging due to the diverse processes that drive the high-speed dynamics of turbulent phenomena. This work presents a novel application of motion tracking to identify and track turbulent filaments in fusion plasmas, called blobs, in a high-frequency video obtained from Gas Puff Imaging diagnostics. We compare four baseline methods (RAFT, Mask R-CNN, GMA, and Flow Walk) trained on synthetic data and then test on synthetic and real-world data obtained from plasmas in the Tokamak à Configuration Variable (TCV). The blob regime identified from an analysis of blob trajectories agrees with state-of-the-art conditional averaging methods for each of the baseline methods employed, giving confidence in the accuracy of these techniques. By making a dataset and benchmark publicly available, we aim to lower the entry barrier to tokamak plasma research, thereby greatly broadening the community of scientists and engineers who might apply their talents to this endeavor.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Resolving ECRH deposition broadening due to edge turbulence in DIII-D

Microwave heat pulse propagation experiments have demonstrated a correlation between millimeter-scale turbulence and deposition profile broadening of electron cyclotron (EC) waves on the DIII-D tokamak. In a set of discharges in DIII-D, a variation in edge density fluctuations on the mm-scale is associated with 40%–150% broader deposition profiles, expressed in terms of normalized minor radius, as compared with equilibrium ray tracing. The 1D power profile is determined from transport analysis of the electron temperature response to EC power modulation using perturbative analysis with a square wave power modulation at 20–70 Hz, producing a series of Fourier harmonics that are fit collectively to resolve transport. Fitting an integrated heat flux expressed in the Fourier basis of the modulation to diffusive, convective, and coupled transport terms in a linear model can resolve the broadened EC deposition width from the power perturbation to resolve a broadening in each case. The best fit degree of beam broadening observed scales approximately linearly with the Doppler backscattering measured fluctuation level in the steep gradient region. Quantifying the effect of edge fluctuation broadening on EC current drive power needs of future devices will require 3D full-wave codes that can be validated on the current generation of machines. These DIII-D experiments provide a quantitative measure of fluctuation effects and a dataset to benchmark full-wave simulations that can model and eventually predict nonlinear effects neglected by 1D equilibrium beam and ray tracing.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Enzyme Engineering Database (EnzEngDB): a platform for sharing and interpreting sequence–function relationships across protein engineering campaigns

The discovery and engineering of new enzymes is important across the bioeconomy, with diverse applications from foods to pharmaceuticals, sensors to agriculture. However, enzyme engineering, in particular machine learning-guided engineering, is hampered by a lack of data. Currently there exists no database designed to capture and interpret datasets created in this domain, nor are there easy analysis and visualisation tools. We developed the Enzyme Engineering Database to provide a centralized resource and an online analysis tool to consolidate sequence-function data from enzyme engineering campaigns, thereby making three contributions: (i) a database into which researchers can deposit public data, (ii) visualisation and analysis tools for protein engineers to analyse their own data or compare enzyme variants to other engineering campaigns, and (iii) a gold-standard dataset for benchmarking automated extraction along with the first large language model extraction pipeline specific for enzyme engineering campaigns. The Enzyme Engineering Database is accessible at http://enzengdb.org/.

Long, Yueming [California Institute of Technology ↗
DOE OSTI2023

RX-ADS: Interpretable Anomaly Detection Using Adversarial ML for Electric Vehicle CAN Data

Recent year has brought considerable advancements in Electric Vehicles (EVs) and associated infrastructures/ communications. Intrusion Detection Systems (IDS) are widely deployed for anomaly detection in such critical infrastructures. This paper presents an Interpretable Anomaly Detection System (RX-ADS) for intrusion detection in CAN protocol communication in EVs. Contributions include: 1) window based feature extraction method; 2) deep Autoencoder based anomaly detection method; and 3) adversarial machine learning based explanation generation methodology. The presented approach was tested on two benchmark CAN datasets: OTIDS and Car Hacking. The anomaly detection performance of RX-ADS was compared against the state-of-the-art approaches on these datasets: HIDS and GIDS. The RX-ADS approach presented performance comparable to the HIDS approach (OTIDS dataset) and has outperformed HIDS and GIDS approaches (Car Hacking dataset). Further, the proposed approach was able to generate explanations for detected abnormal behaviors arising from various intrusions. Furthermore, these explanations were later validated by information used by domain experts to detect anomalies. Other advantages of RX-ADS include: 1) the method can be trained on unlabeled data; 2) explanations help experts in understanding anomalies and root course analysis, and also help with AI model debugging and diagnostics, ultimately improving user trust in AI systems.

42 ENGINEERING↗