Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Buildings Sector Scenarios: Demand-side data to support energy system planning in the United States

The US energy system is in a period of high uncertainty about load growth, its implications for the energy generation mix, and downstream impacts on customer energy costs. In this context, there is a need for comprehensive, credible, and readily-customized projections of energy demand to ensure that planning decisions account for end-use management opportunities to improve system reliability and affordability. Here we introduce the Buildings Sector Scenarios (BSS) dataset, which includes a benchmark suite of such projections for the buildings sector — a key source of energy consumption, peak electricity demand, and consumer energy expenditures. The dataset contains projections through 2050 covering the contiguous United States (CONUS) resolved down to the county, hourly level by sector and end use for electricity demand and to the state, annual level by sector and end use for non-electric fuels. We summarize the BSS analysis workflow and the tools and datasets that support it, document key BSS scenario inputs and modeling assumptions, and outline BSS scenario outputs. We assess the technical quality of the dataset against historical surveys and projected estimates of buildings sector demand. Finally, we provide guidance on how stakeholders can access, use, and reproduce the dataset, and/or create new scenarios to explore their own analysis questions.

Langevin, Jared↗
DOE OSTI2021

Robust implementation of generative modeling with parametrized quantum circuits

Although the performance of hybrid quantum-classical algorithms is highly dependent on the selection of the classical optimizer and the circuit ansätze, a robust and thorough assessment on-hardware of such features has been missing to date. From the optimizer perspective, the primary challenge lies in the solver’s stochastic nature, and their significant variance over the random initialization. Therefore, a robust comparison requires one to perform several training curves for each solver before one can reach conclusions about their typical performance. Since each of the training curves requires the execution of thousands of quantum circuits in the quantum computer, such a robust study remained a steep challenge for most hybrid platforms available today. In this work, we leverage on Rigetti’s Quantum Cloud Services (QCS™) to overcome this implementation barrier, and we study the on-hardware performance of the data-driven quantum circuit learning (DDQCL) for three different state-of-the-art classical solvers, and on two-different circuit ansätze associated to different entangling connectivity graphs for the same task. Additionally, we assess the gains in performance from varying circuit depths. To evaluate the typical performance associated with each of these settings in this benchmark study, we use at least five independent runs of DDQCL towards the generation of quantum generative models capable of capturing the patterns of the canonical Bars and Stripes dataset. In this experimental benchmarking, the gradient-free optimization algorithms show an outstanding performance compared to the gradient-based solver. In particular, one of them had better performance when handling the unavoidable noisy objective function to be minimized under experimental conditions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Building fault detection data to aid diagnostic algorithm creation and performance testing

It is estimated that approximately 4-5% of national energy consumption can be saved through corrections to existing commercial building controls infrastructure and resulting improvements to efficiency. Correspondingly, automated fault detection and diagnostics (FDD) algorithms are designed to identify the presence of operational faults and their root causes. A diversity of techniques is used for FDD spanning physical models, black box, and rule-based approaches. A persistent challenge has been the lack of common datasets and test methods to benchmark their performance accuracy. This article presents a first of its kind public dataset with ground-truth data on the presence and absence of building faults. This dataset spans a range of seasons and operational conditions and encompasses multiple building system types. It contains information on fault severity, as well as data points reflective of the measurements in building control systems that FDD algorithms typically have access to. The data were created using simulation models as well as experimental test facilities, and will be expanded over time.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

A robust synthetic data generation framework for machine learning in high-resolution transmission electron microscopy (HRTEM)

Machine learning techniques are attractive options for developing highly-accurate analysis tools for nanomaterials characterization, including high-resolution transmission electron microscopy (HRTEM). However, successfully implementing such machine learning tools can be difficult due to the challenges in procuring sufficiently large, high-quality training datasets from experiments. In this work, we introduce Construction Zone, a Python package for rapid generation of complex nanoscale atomic structures which enables fast, systematic sampling of realistic nanomaterial structures and can be used as a random structure generator for large, diverse synthetic datasets. Using Construction Zone, we develop an end-to-end machine learning workflow for training neural network models to analyze experimental atomic resolution HRTEM images on the task of nanoparticle image segmentation purely with simulated databases. Further, we study the data curation process to understand how various aspects of the curated simulated data—including simulation fidelity, the distribution of atomic structures, and the distribution of imaging conditions—affect model performance across three benchmark experimental HRTEM image datasets. Using our workflow, we are able to achieve state-of-the-art segmentation performance on these experimental benchmarks and, further, we discuss robust strategies for consistently achieving high performance with machine learning in experimental settings using purely synthetic data. Construction Zone and its documentation are available at https://github.com/lerandc/construction_zone.

36 MATERIALS SCIENCE↗
DOE OSTI2022

BioADAPT-MRC: adversarial learning-based domain adaptation improves biomedical machine reading comprehension task

ABSTRACT Motivation Biomedical machine reading comprehension (biomedical-MRC) aims to comprehend complex biomedical narratives and assist healthcare professionals in retrieving information from them. The high performance of modern neural network-based MRC systems depends on high-quality, large-scale, human-annotated training datasets. In the biomedical domain, a crucial challenge in creating such datasets is the requirement for domain knowledge, inducing the scarcity of labeled data and the need for transfer learning from the labeled general-purpose (source) domain to the biomedical (target) domain. However, there is a discrepancy in marginal distributions between the general-purpose and biomedical domains due to the variances in topics. Therefore, direct-transferring of learned representations from a model trained on a general-purpose domain to the biomedical domain can hurt the model’s performance. Results We present an adversarial learning-based domain adaptation framework for the biomedical machine reading comprehension task (BioADAPT-MRC), a neural network-based method to address the discrepancies in the marginal distributions between the general and biomedical domain datasets. BioADAPT-MRC relaxes the need for generating pseudo labels for training a well-performing biomedical-MRC model. We extensively evaluate the performance of BioADAPT-MRC by comparing it with the best existing methods on three widely used benchmark biomedical-MRC datasets—BioASQ-7b, BioASQ-8b and BioASQ-9b. Our results suggest that without using any synthetic or human-annotated data from the biomedical domain, BioADAPT-MRC can achieve state-of-the-art performance on these datasets. Availability and implementation BioADAPT-MRC is freely available as an open-source project at https://github.com/mmahbub/BioADAPT-MRC. Supplementary information Supplementary data are available at Bioinformatics online.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Bragg Spot Finder (BSF): a new machine-learning-aided approach to deal with spot finding for rapidly filtering diffraction pattern images

Macromolecular crystallography contributes significantly to understanding diseases and, more importantly, how to treat them by providing atomic resolution 3D structures of proteins. This is achieved by collecting X-ray diffraction images of protein crystals from important biological pathways. Spotfinders are used to detect the presence of crystals with usable data, and the spots from such crystals are the primary data used to solve the relevant structures. Having fast and accurate spot finding is essential, but recent advances in synchrotron beamlines used to generate X-ray diffraction images have brought us to the limits of what the best existing spotfinders can do. This bottleneck must be removed so spotfinder software can keep pace with the X-ray beamline hardware improvements and be able to see the weak or diffuse spots required to solve the most challenging problems encountered when working with diffraction images. In this paper, we first present Bragg Spot Detection (BSD), a large benchmark Bragg spot image dataset that contains 304 images with more than 66 000 spots. We then discuss the open source extensible U-Net-based spotfinder Bragg Spot Finder (BSF), with image pre-processing, a U-Net segmentation backbone, and post-processing that includes artifact removal and watershed segmentation. Finally, we perform experiments on the BSD benchmark and obtain results that are (in terms of accuracy) comparable to or better than those obtained with two popular spotfinder software packages ( Dozor and DIALS ), demonstrating that this is an appropriate framework to support future extensions and improvements.

36 MATERIALS SCIENCE↗
DOE OSTI2023

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. Furthermore, we observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Flow reversal benchmark of a one-sided heated narrow rectangular channel with CATHARE and RELAP5

Flow reversal in narrow coolant channels can be a crucial phenomenon for the safety of research reactors with a downward nominal flow direction. During a loss of forced flow accident, the downward flow stagnates briefly before transitioning into an upward natural circulation flow. The fuel may be damaged if dryout occurs and threshold fuel and/or cladding temperatures are exceeded. A comprehensive study is provided for flow reversal in narrow rectangular channels by examining experimental data and conducting software model analyses. The literature on flow reversal was reviewed, and selected experimental datasets were used to benchmark against CATHARE and RELAP5 models and also compare the code calculations with each other. The experimental data comes from flow reversal tests conducted with a narrow rectangular channel with one-sided heating. The results were compared with experimental data for successful flow reversal tests and predicted dryout power for dryout conditions. Also, the study examined the effects of the pump coastdown period, inlet liquid temperature, system pressure, and localized pressure drops. The experimental results showed that shorter coastdown periods, reduced pressure drops, and lower coolant inlet temperatures increased the dryout power. However, the system pressure did not noticeably affect the results. The simulation results showed that both CATHARE and RELAP5 agreed with experimental data, capturing the trends of the experimental results. Slight differences between each code calculation, as well as the predicted and measured dryout powers, were attributed to experimental uncertainties and the modeling of physical phenomena such as wall nucleation, interfacial heat transfer, drag coefficients, and critical heat flux. Overall, this study provides an understanding of flow reversal and the prediction capabilities of thermal-hydraulics software models. In conclusion, a future study of the flow reversal benchmark of a narrow rectangular channel with two-sided heating may provide additional valuable insights.

CATHARE↗
DOE OSTI2025

Enhancing ChatPORT with CUDA-to-SYCL Kernel Translation Capability

Large Language Models (LLMs) have shown strong capabilities in general code translation. However, code translation involving parallel programming models remains largely unexplored. This work enhances the capabilities of code LLMs in CUDA-to-SYCL kernel translation with parameter-efficient fine-tuning. The resultant fine-tuned LLM, called ChatPORT, is an effort to provide high-fidelity translations from one programming model to another. We describe the preparation of datasets from heterogeneous computing benchmarks for model fine-tuning and testing, the parameter-efficient fine-tuning of 19 open-source code models ranging in size from 0.5 to 34 billion parameters and evaluate the correctness rates of the SYCL kernels by the fine-tuned models. The experimental results show that most code models fail to translate CUDA codes to SYCL correctly. However, fine-tuning these models using a small set of CUDA and SYCL kernels can enhance the capabilities of these models in kernel translation. Depending on the sizes of the models, the correctness rate ranges from 19.9% to 81.7% for a test dataset of 62 CUDA kernels.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2025

Towards robust surrogate models: Benchmarking machine learning approaches to expediting phase field simulations of brittle fracture

Data-driven approaches have the potential to make modeling complex, nonlinear physical phenomena significantly more computationally tractable. For example, computational modeling of fracture is a core challenge where machine learning techniques have the potential to provide a much needed speedup that would enable progress in areas such as multi-scale modeling and uncertainty quantification. Currently, phase field modeling (PFM) of fracture is one such approach that offers a convenient variational formulation to model crack nucleation, branching and propagation. To date, machine learning techniques have shown promise in approximating PFM simulations. While standard fracture benchmarks represent realistic scenarios frequently observed in practice, they typically do not provide sufficiently challenging tests for data-driven methods. Here, to address this gap, we introduce a challenging dataset based on PFM simulations designed to benchmark and advance ML methods for fracture modeling. This dataset includes three energy decomposition methods, two boundary conditions, and 1000 random initial crack configurations for a total of 6000 simulations. Each sample contains 100 time steps capturing the temporal evolution of the crack field. Alongside this dataset, we also implement and evaluate Physics Informed Neural Networks (PINN), Fourier Neural Operators (FNO), and UNet models as baselines, and explore the impact of ensembling strategies on prediction accuracy. With this combination of our dataset and baseline models drawn from the literature we aim to provide a standardized and challenging benchmark for evaluating machine learning approaches to solid mechanics. Our results highlight both the promise and limitations of popular current models, and demonstrate the utility of this dataset as a testbed for advancing machine learning in fracture mechanics research.

Benchmark dataset↗
DOE OSTI2022

Measuring binary fluidization of nonspherical and spherical particles using machine learning aided image processing

Abstract The binary fluidization of Geldart D type nonspherical wood particles and spherical low density polyethylene (LDPE) particles was investigated in a laboratory‐scale bed. The experiment was performed for varying static bed height, wood particles count, as well as superficial gas velocity. The LDPE velocity field were quantified using particle image velocimetry (PIV). The wood particles orientation and velocity are measured using particle tracking velocimetry (PTV). A machine learning pixel‐wise classification model was trained and applied to acquire wood and LDPE particle masks for PIV and PTV processing, respectively. The results show significant differences in the fluidization behavior between LDPE only case and binary fluidization case. The effects of wood particles on the slugging frequency, mean, and variation of bed height, and characteristics of the particle velocities/orientations were quantified and compared. This comprehensive experimental dataset serves as a benchmark for validating numerical models.

Li, Cheng↗
DOE OSTI2026

Hybrid Quantum–Classical Graph Transformers for Efficient Sentiment Analysis

Quantum Machine Learning (QML) offers a promising paradigm that leverages quantum computing principles to develop efficient and expressive models for learning from complex and structured data. Recent advances in natural language processing (NLP) and artificial intelligence (AI) have demonstrated capabilities in understanding, generating, and reasoning over linguistic and multimodal information. In this work, we present the Quantum Graph Transformer (QGT), a hybrid quantum–classical architecture that extends graph transformer capabilities through quantum self-attention. The QGT models variable-length sentences as token graphs, where both the embedding encoding and the self-attention mechanisms are implemented using parameterized quantum circuits (PQCs), enabling efficient contextual learning with significantly fewer trainable parameters. We train QGT using both fully connected and 𝑘 -nearest-neighbor graph structures and evaluate it on five benchmark sentiment-classification datasets. Experimental results show that QGT consistently achieves higher or comparable accuracy to existing quantum NLP models and outperforms a Classical Graph Transformer (CGT) baseline with identical architecture, achieving 29.4 × fewer parameters while requiring 3–5 × fewer samples to reach comparable performance. These findings highlight the potential of graph-based quantum models as scalable and data-efficient architectures for natural language understanding.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Data on Cu- and Ni-Si-Mn-rich solute clustering in a neutron irradiated austenitic stainless steel

The data presented in this article is supplementary to the research article “Phase instabilities in austenitic steels during particle bombardment at high and low dose rates” (Levine et al.). Needle-shaped samples were prepared with focused ion beam milling from a 304L stainless steel that was irradiated with fast neutrons (E 0.1 MeV) in the BOR-60 reactor at 318 °C to 47.5 dpa. Atom probe tomography (APT) experiments in voltage mode were then conducted on a Cameca LEAP 5000X HR. Atom position, range, and mass spectrum files after reconstruction with Cameca’s IVAS software are included. Cu- and Ni-Si-Mn-rich solute nanoclusters were identified and analyzed using the Open Source Characterization of APT Reconstructions (OSCAR) program. Python code for OSCAR, information on the program’s underlying algorithm, and sample output files are provided. A proximity histogram of a Ni-Si-Mn-rich cluster and a 1D density/solute concentration profile of a Cu-rich cluster are given to demonstrate OSCAR’s analytical functionalities. The provided APT dataset is valuable for benchmarking phase instabilities in neutron-irradiated austenitic stainless steels that occur at high doses. The OSCAR program can be reused to process other APT data sets where solute nanoclustering is of interest.

42 ENGINEERING↗
DOE OSTI2022

A classical model for three-body interactions in aqueous ionic systems

We present a classical induction model to evaluate the three-body ion–water–water (I–W–W) and water–water–water (W–W–W) interactions in aqueous ionic systems. The classical description of the induction energy is based on electrostatic distributed multipoles up to hexadecapole and distributed polarizabilities up to quadrupole–quadrupole on the O and H atoms of water. The monatomic ions were described by a point charge and a dipole–dipole polarizability, while for the polyatomic ions, distributed multipoles up to hexadecapole and distributed polarizabilities up to quadrupole–quadrupole were used. The accuracy of the classical model is benchmarked against an accurate dataset of 936 (I–W–W) and 2184 (W–W–W) three-body terms for 13 different monatomic and polyatomic cation and anion systems. The classical model shows excellent agreement with the reference second order Moller–Plesset and coupled-cluster single double and perturbative triple [CCSD(T)] three-body energies. The Root-Mean-Square-Errors (RMSEs) for monatomic cations, monatomic anions, and polyatomic ions were 0.29, 0.25, and 0.12 kcal/mol, respectively. The corresponding RMSE for 1744 CCSD(T)/aVTZ three-body (W–W–W) energies, used to train MB-pol, was 0.12 kcal/mol. The accuracy of the proposed classical model demonstrates that the three-body term for aqueous ionic systems can be accurately modeled classically. This approach provides a fast, efficient, and as-accurate path toward modeling the three-body term in aqueous ionic systems that is fully transferable across systems with different ions without the need to fit to tens of thousands of ab initio calculations for each ion to extend existing many-body force fields to interactions between water and ions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Advanced Signal Decomposition Analysis and Anomaly Detection in Photovoltaic Systems

With the rapid expansion of large-scale photovoltaic (PV) plants, it is paramount for solar stakeholders to understand the reliability and efficiency of their plants to inform maintenance decisions, increase production, and understand the design factors that impact performance. Diagnosing underperformance in PV plants is challenging due to the relatively few monitoring points with respect to the large geographic footprint of the plant. This work introduces a cutting-edge method that transforms the analysis and management of key factors influencing PV plant performance, including performance loss rate (PLR), recoverable soiling, and major system changes. Identifying these factors is critical for deriving actionable insights. Leveraging advanced analytical techniques such as wavelet transformation, robust regression, and extreme point analysis, this approach provides a nuanced understanding of these factors. This method has been tested across two synthetic datasets and one real dataset, consistently surpassing existing benchmarks by achieving a lower median mean absolute error and reduced error variability across all comparable components.

14 SOLAR ENERGY↗
DOE OSTI2024

U.S. Average Corn Ethanol Production Baseline for 45Q Life Cycle Analysis Version 1.0 (2018-2022)

U.S. Average Benchmark life cycle inventory documentation for ethanol production for calendar years 2018 - 2022. The documentation includes a technical report describing data sources and statistical methods used and an Excel file showing calculations. The calculations performed are used in the 45Q benchmark json-ld dataset, compatible with the NETL CO2U LCA Guidance Toolkit. Summary impact results will be included in the NETL CO2U LCA Documentation Spreadsheet within the Toolkit. To access the model referenced in the report, please visit https://www.netl.doe.gov/energy-analysis/details?id=55ab21f1-9238-4b63-9a7a-ae1a4cfa7b05.

09 BIOMASS FUELS↗
DOE OSTI2024

U.S. Average Market Carbon Dioxide Production Baseline Documentation For 45Q Life Cycle Analysis: Version 1.0 (2018-2022)

U.S. Average Benchmark life cycle inventory documentation for market carbon dioxide production for calendar years 2018 - 2022. The documentation includes a technical report describing data sources and statistical methods used and an Excel file showing calculations. The calculations performed are used in the 45Q benchmark json-ld dataset, compatible with the NETL CO2U LCA Guidance Toolkit. Summary impact results will be included in the NETL CO2U LCA Documentation Spreadsheet within the Toolkit. To access the model referenced in the report, please visit https://www.netl.doe.gov/energy-analysis/details?id=21915ba8-d2cf-43ea-b9b2-1d76363a46a7

29 ENERGY PLANNING, POLICY, AND ECONOMY↗