Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Nuclear Dependence of Charged Current Inclusive $\bar{\nu}_\mu -$A Cross Section Measurements at 6 GeV at MINERvA

The main advantage of the MINERvA detector is that it uses differently sized nuclear targets, in the same neutrino beam, allowing for a direct study of nuclear medium effects, with reduced flux and detector uncertainties. This thesis presents the first high statistics charged current antineutrino-nucleus inclusive two-dimensional cross section measurements using an antineutrino beam with $\langle E_{\bar\nu}\rangle\sim$ 6 GeV, on iron, lead, carbon and hydrocarbon targets, as a function of bjorken $x$ and four-momentum transfer squared $Q^2$, invariant mass $W$ and $Q^2$ and muon longitudinal and transverse momenta, $p_t$ and $p_z$. These nuclei of different sizes were utilized to measure the nuclear dependence of the cross section. The antineutrino interactions that were utilized for the analysis presented in this thesis, were having an antimuon in the final state with energy 2-20GeV and with an antimuon scattering angle less than $17^o$. This analysis represents one of the largest antineutrino datasets analyzed to date in the energy region of $\langle E_{\bar\nu} \rangle \sim$ 6 GeV. This direct evidence for the nuclear medium effects, can help benchmarking models, which can be used by future neutrino experiments like DUNE, by extrapolating these results to Argon target.

Gaur, Prameet Kumar↗

An Open Combinatorial Diffraction Dataset Including Consensus Human and Machine Learning Labels with Quantified Uncertainty for Training New Machine Learning Models

Modern machine learning and autonomous experimentation schemes in materials science rely on accurate analysis of the data ingested by these models. Unfortunately, accurate analysis of the underlying data can be difficult, even for domain experts, complicating the training of the models intended to drive experiments. This is especially true when the goal is to identify the presence of weak signatures in diffraction or spectroscopic datasets. In this work, we examine a set of as-obtained diffraction data that track the phase transition from monoclinic to tetragonal in a Nb-doped VO2 film as a function of temperature and dopant concentration. We then task a set of domain experts and a set of machine learning experts with identifying which phase is present in each diffraction pattern manually and algorithmically, respectively; in both cases, the labels can vary dramatically, especially at the phase boundaries. We use the mode of the labels and the Shannon entropy as a method to capture, preserve and propagate consensus labels and their variance. Further we use the expert labels as a benchmark and demonstrate the use of Shannon entropy weighted scoring to test the performance of machine learning generated labels. Finally, we propose a material data challenge centered around generating improved labeling algorithms. This real-world dataset curated with expert labels can act as test bed for new algorithms. The raw data, annotations and code used in this study are all available online at data.gov and the interested reader is encouraged to replicate and improve the existing models

97 MATHEMATICS AND COMPUTING↗

Advances in building data management for building performance standards using the SEED platform

Reducing energy consumption and greenhouse gas emissions in the built environment is a critical step in achieving emission goals to mitigate climate change impacts. Local, federal, and international jurisdictions are deploying several methods to reduce energy and emissions such as voluntary and mandatory benchmarking and building performance standards, requiring building owners to reach energy and emission targets. Jurisdictions leveraging benchmarking and building performance standards require knowledge of the buildings covered; which is a large task due to staffing constraints, limited information on building characteristics and tax parcel data, and the need for advanced data management techniques to align datasets. This paper describes an open-source platform's recent advances to create consistent taxonomies, identify erroneous data, enable auditability, and track building performance. The paper concludes with two use cases on how the platform has been used by jurisdictions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Systematic softening in universal machine learning interatomic potentials

Machine learning interatomic potentials (MLIPs) have introduced a new paradigm for atomic simulations. Recent advancements have led to universal MLIPs (uMLIPs) that are pre-trained on diverse datasets, providing opportunities for universal force fields and foundational machine learning models. However, their performance in extrapolating to out-of-distribution complex atomic environments remains unclear. In this study, we highlight a consistent potential energy surface (PES) softening effect in three uMLIPs: M3GNet, CHGNet, and MACE-MP-0, which is characterized by energy and force underprediction in atomic-modeling benchmarks including surfaces, defects, solid-solution energetics, ion migration barriers, phonon vibration modes, and general high-energy states. The PES softening behavior originates primarily from the systematically underpredicted PES curvature, which derives from the biased sampling of near-equilibrium atomic arrangements in uMLIP pre-training datasets. Our findings suggest that a considerable fraction of uMLIP errors are highly systematic, and can therefore be efficiently corrected. We argue for the importance of a comprehensive materials dataset with improved PES sampling for next-generation foundational MLIPs.

36 MATERIALS SCIENCE↗

Improving the Concrete Crack Detection Process via a Hybrid Visual Transformer Algorithm

Inspections of concrete bridges across the United States represent a significant commitment of resources, given their biannual mandate for many structures. With a notable number of aging bridges, there is an imperative need to enhance the efficiency of these inspections. This study harnessed the power of computer vision to streamline the inspection process. Our experiment examined the efficacy of a state-of-the-art Visual Transformer (ViT) model combined with distinct image enhancement detector algorithms. We benchmarked against a deep learning Convolutional Neural Network (CNN) model. These models were applied to over 20,000 high-quality images from the Concrete Images for Classification dataset. Traditional crack detection methods often fall short due to their heavy reliance on time and resources. This research pioneers bridge inspection by integrating ViT with diverse image enhancement detectors, significantly improving concrete crack detection accuracy. Notably, a custom-built CNN achieves over 99% accuracy with substantially lower training time than ViT, making it an efficient solution for enhancing safety and resource conservation in infrastructure management. These advancements enhance safety by enabling reliable detection and timely maintenance, but they also align with Industry 4.0 objectives, automating manual inspections, reducing costs, and advancing technological integration in public infrastructure management.

42 ENGINEERING↗

Machine learning and ligand binding predictions: A review of data, methods, and obstacles

We report that computational predictions of ligand binding is a difficult problem, with more accurate methods being extremely computationally expensive. The use of machine learning for drug binding predictions could possibly leverage the use of biomedical big data in exchange for time-intensive simulations. This paper reviews current trends in the use of machine learning for drug binding predictions, data sources to develop machine learning algorithms, and potential problems that may lead to overfitting and ungeneralizable models. A few popular datasets that can be used to develop virtual high-throughput screening models are characterized using spatial statistics to quantify potential biases. We can see from evaluating some common benchmarks that good performance correlates with models with high-predicted bias scores and models with low bias scores do not have much predictive power. A better understanding of the limits of available data sources and how to fix them will lead to more generalizable models that will lead to novel drug discovery.

59 BASIC BIOLOGICAL SCIENCES↗

Modeling Oxidative Dehydrogenation of Propane with Supported Vanadia Catalysts Using Multireference Methods

The oxidative dehydrogenation of propane over supported vanadium oxide catalysts poses significant computational challenges due to complex electronic structure changes along the reaction coordinate, driven primarily by changes in the oxidation states of vanadium. To address these challenges, we systematically test quantum chemical methods, including multireference (MR) approaches, domain-based local pair natural orbital coupled cluster theory (DLPNO-CCSD(T)), and density functional theory (DFT). The initial C–H bond-breaking transition state requires MR treatment due to its multireference character, while subsequent steps permit efficient single-reference calculations. For the rate-limiting C–H activation step mediated by the vanadyl moiety, complete 1 active space second-order perturbation theory (CASPT2) yields an apparent activation barrier (E app 600K ) of 138 kJ/mol, consistent with experimental values (134 ± 4 kJ/mol; Gruene et al. Catal. Today 2010, 157, 137). In contrast, DLPNO-CCSD(T) overestimates this barrier (198 kJ/mol), whereas DFT predictions span 125–150 kJ/mol, depending on the functional. Our multireference investigation of this transition metal oxide-catalyzed process demonstrates that an active space that incorporates the C–H σ and V=O σ/π bonding orbitals, oxygen lone pairs, and their antibonding counterparts adequately captures electronic structure changes along the chemical transformation. Furthermore, these findings provide a general strategy for active space selection in transition metal oxide-catalyzed C/O–H bond activation reactions. The reference dataset from this work, which includes MR calculations with manually selected active spaces for all intermediates and transition states in the propane ODH reaction network, will serve as a benchmark for automating active space selection in similar systems.

Catalysts↗

LandScan Mosaic Time Series Version 1.0: 1975-2024

The LandScan Mosaic Annual Global Ambient Population Time Series, Version 1.0, provides annual high-resolution global ambient population estimates from 1975 to 2024. The dataset is distributed as one Cloud-Optimized GeoTIFF per year on a globally aligned 3 arc-second WGS 84 grid. The 2024 layer is the LandScan Mosaic benchmark reference year, while the 1975–2023 layers are historical estimates generated using the LandCast backcasting framework.

Zimmer, Andrew [ORNL] (ORCID:0000000176838713)↗

DeepBench: A simulation package for physical benchmarking data

We introduce **DeepBench**, a python library that generates simple simulated image data from first principles, such as basic geometric shapes and astronomical objects. These data are highly valuable for developing (calibration, testing, and benchmarking) statistical and machine learning models because they make it possible to connect the final data product to physically interpretable inputs. This software includes tools to curate and store the datasets to maximize reproducibility.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Ice Accretions and Full-Scale Iced Aerodynamic Performance Data for a Two-Dimensional NACA 23012 Airfoil

This report documents the data collected during the large wind tunnel campaigns conducted as part of the SUNSET project (StUdies oN Scaling EffecTs due to ice) also known as the Ice-Accretion Aerodynamics Simulation study: a joint effort by NASA, the Office National d'Etudes et Recherches Aérospatiales (ONERA), and the University of Illinois. These data form a benchmark database of full-scale ice accretions and corresponding ice-contaminated aerodynamic performance data for a two-dimensional (2D) NACA 23012 airfoil. The wider research effort also included an analysis of ice-contaminated aerodynamics that categorized ice accretions by aerodynamic effects and an investigation of subscale, low- Reynolds-number ice-contaminated aerodynamics for the NACA 23012 airfoil. The low-Reynolds-number investigation included an analysis of the geometric fidelity needed to reliably assess aerodynamic effects of airfoil icing using artificial ice shapes. Included herein are records of the ice accreted during campaigns in NASA Glenn Research Center's Icing Research Tunnel (IRT). Two different 2D NACA 23012 airfoil models were used during these campaigns; an 18-in. (45.7-cm) chord (subscale) model and a 72-in. (182.9-cm) chord (full-scale) model. The aircraft icing conditions used during these campaigns were selected from the Federal Aviation Administration's (FAA's) Code of Federal Regulations (CFR) Part 25 Appendix C icing envelopes. The records include the test conditions, photographs of the ice accreted, tracings of the ice, and ice depth measurements. Model coordinates and pressure tap locations are also presented. Also included herein are the data recorded during a wind tunnel campaign conducted in the F1 Subsonic Pressurized Wind Tunnel of ONERA. The F1 tunnel is a pressured, high- Reynolds-number facility that could accommodate the full-scale (72-in. (182.9-cm) chord) 2D NACA 23012 model. Molds were made of the ice accreted during selected test runs of the full-scale model in the IRT. From these molds, castings were made that closely replicated the features of the accreted ice. The castings were then mounted on the full-scale model in the F1 tunnel, and aerodynamic performance measurements were made using model surface pressure taps, the facility force balance system, and a large wake rake designed specifically for these tests. Tests were run over a range of Reynolds and Mach numbers. For each run, the model was rotated over a range of angles-of-attack that included airfoil stall. The benchmark data collected during these campaigns were, and continue to be, used for various purposes. The full-scale data form a unique, ice-accretion and associated aerodynamic performance dataset that can be used as a reference when addressing concerns regarding the use of subscale ice-accretion data to assess full-scale icing effects. Further, the data may be used in the development or enhancement of both ice-accretion prediction codes and computational fluid dynamic codes when applied to study the effects of icing. Finally, as was done in the wider study, the data may be used to help determine the level of geometric fidelity needed for artificial ice used to assess aerodynamic degradation due to aircraft icing. The structured, multifaceted approach used in this research effort provides a unique perspective on the aerodynamic effects of aircraft icing. The data presented in this report are available in electronic form upon formal approval by proper NASA and ONERA authorities.

Addy, Harold E., Jr.↗

NASA POWER: Providing Analysis-Ready, Cloud-Optimized Data for AI /ML Training and Applications in Earth Science

As global demand for sustainable development grows, the integration of Earth Observation (EO) data into decision making frameworks has become a primary objective for the scientific community. The NASA Prediction of Worldwide Energy Resources (POWER) project serves as a bridge between NASA EO data and the specialized needs of the renewable energy, sustainable infrastructure and agroclimatology communities. In this poster presentation we will present an overview of POWER data products and services along with its use in diverse research to decision-making workflows. By providing over 40 years of high-resolution historical, hourly and daily solar and meteorological data, POWER transforms satellite observations and global model reanalysis into actionable, Analysis-Ready Dataset (ARD). Currently, the project delivers over 250 industry-friendly parameters to the users from different NASA datasets like CERES SYN1Deg, MERRA-2, and IMERG alongside downscaled CMIP6 climate model data, fulfilling over 16 million requests from 50,000 unique users monthly. To ensure data quality and traceability, these parameters are rigorously validated against the ground-based observations from the Baseline Surface Radiation Network (BSRN) and the Global Surface Summary of the Day (GSOD) – these results will be discussed in the presentation. A newly introduced web-based PaRameter Uncertainty ViEwer (PRUVE) tool will be presented that provides an online validation platform to the users that benchmarks satellite-based and assimilation data products against these surface measurements. To reduce technical barriers to data adoption, POWER data is accessible through RESTful APIs, ESRI ArcGIS Image Services, a web-based Data Access Viewer tool, allowing users to visualize, validate and apply the dataset. For efficient data delivery POWER data is cloud-optimized into Zarr datastore accessible through NASA managed Amazon S3 ensures high-performance allowing users to integrate EO directly into operational pipelines. These customized services will be presented. Use cases from application will be presented from the energy sector - such as for design of generation systems, performance monitoring of solar power plants, in infrastructure sector- optimizing building energy efficiency and thermal comfort, in agriculture – such as driving crop simulation and yield forecasting models to enable climate resilient farming. Furthermore, the shift toward machine learning (ML) in EO research that has positioned POWER as a key provider for training datasets which will be discussed. Use-cases will be presented to showcase how NASA data is enabling the development of predictive tools for climate variability and resource management. The poster will present POWER’s future plans including technology development to enhance data traceability and reproducibility and improving I/O performance to support the rapid integration of new EO products, ensuring that POWER remains a robust scalable backend for the evolving landscape of AI-driven Earth Science. Additionally, POWER is developing an AI Agent and an MCP-Server to enable industry AI-Agentic workflows.

Neha Khadka↗

DeepPatent2: A Large-Scale Benchmarking Corpus for Technical Drawing Understanding

Abstract Recent advances in computer vision (CV) and natural language processing have been driven by exploiting big data on practical applications. However, these research fields are still limited by the sheer volume, versatility, and diversity of the available datasets. CV tasks, such as image captioning, which has primarily been carried out on natural images, still struggle to produce accurate and meaningful captions on sketched images often included in scientific and technical documents. The advancement of other tasks such as 3D reconstruction from 2D images requires larger datasets with multiple viewpoints. We introduce DeepPatent2, a large-scale dataset, providing more than 2.7 million technical drawings with 132,890 object names and 22,394 viewpoints extracted from 14 years of US design patent documents. We demonstrate the usefulness of DeepPatent2 with conceptual captioning. We further provide the potential usefulness of our dataset to facilitate other research areas such as 3D image reconstruction and image retrieval.

97 MATHEMATICS AND COMPUTING↗

Resilience Assessment for Distribution Systems during Hurricanes: A Learning-Based Framework

This paper presents a proactive strategy for hurricane-resilient distribution systems. It proposes a Bayesian Neural Network-based outage prediction model considering various parameters, including electrical components, and weather and environmental factors. Addressing challenges in imbalanced outage datasets, a Bias-Variance Tradeoff method is proposed. A resilience assessment model quantifies resilience indices, providing insights into system weaknesses. The approach identifies weak points and serves as a planning benchmark. Numerical results on the modified IEEE 123-node test system demonstrate effectiveness in realistic hurricane scenarios.

Vahedi, Soroush↗

Benchmarking blockchain-based gene-drug interaction data sharing methods: A case study from the iDASH 2019 secure genome analysis competition blockchain track

Blockchain distributed ledger technology is just starting to be adopted in genomics and healthcare applications. Despite its increased prevalence in biomedical research applications, skepticism regarding the practicality of blockchain technology for real-world problems is still strong and there are few implementations beyond proof-of-concept. We focus on benchmarking blockchain strategies applied to distributed methods for sharing records of gene-drug interactions. We expect this type of sharing will expedite personalized medicine. We generated gene-drug interaction test datasets using the Clinical Pharmacogenetics Implementation Consortium (CPIC) resource. We developed three blockchain-based methods to share patient records on gene-drug interactions: Query Index, Index Everything, and Dual-Scenario Indexing. We achieved a runtime of about 60 s for importing 4,000 gene-drug interaction records from four sites, and about 0.5 s for a data retrieval query. Our results demonstrated that it is feasible to leverage blockchain as a new platform to share data among institutions.

60 APPLIED LIFE SCIENCES↗

Assessment of fine-tuned large language models for real-world chemistry and material science applications

The current generation of large language models (LLMs) has limited chemical knowledge. Recently, it has been shown that these LLMs can learn and predict chemical properties through fine-tuning. Using natural language to train machine learning models opens doors to a wider chemical audience, as field-specific featurization techniques can be omitted. In this work, we explore the potential and limitations of this approach. We studied the performance of fine-tuning three open-source LLMs (GPT-J-6B, Llama-3.1-8B, and Mistral-7B) for a range of different chemical questions. We benchmark their performances against “traditional” machine learning models and find that, in most cases, the fine-tuning approach is superior for a simple classification problem. Depending on the size of the dataset and the type of questions, we also successfully address more sophisticated problems. The most important conclusions of this work are that, for all datasets considered, their conversion into an LLM fine-tuning training set is straightforward and that fine-tuning with even relatively small datasets leads to predictive models. These results suggest that the systematic use of LLMs to guide experiments and simulations will be a powerful technique in any research study, significantly reducing unnecessary experiments or computations.

Van Herck, Joren↗

The PDF4LHC21 combination of global PDF fits for the LHC Run III

A precise knowledge of the quark and gluon structure of the proton, encoded by the parton distribution functions (PDFs), is of paramount importance for the interpretation of high-energy processes at present and future lepton–hadron and hadron–hadron colliders. Motivated by recent progress in the PDF determinations carried out by the CT, MSHT, and NNPDF groups, we present an updated combination of global PDF fits: PDF4LHC21. It is based on the Monte Carlo combination of the CT18, MSHT20, and NNPDF3.1 sets followed by either its Hessian reduction or its replica compression. Extensive benchmark studies are carried out in order to disentangle the origin of the differences between the three global PDF sets. In particular, dedicated fits based on almost identical theory settings and input datasets are performed by the three groups, highlighting the role played by the respective fitting methodologies. We compare the new PDF4LHC21 combination with its predecessor, PDF4LHC15, demonstrating their good overall consistency and a modest reduction of PDF uncertainties for key LHC processes such as electroweak gauge boson production and Higgs boson production in gluon fusion. We study the phenomenological implications of PDF4LHC21 for a representative selection of inclusive, fiducial, and differential cross sections at the LHC. The PDF4LHC21 combination is made available via the LHAPDF library and provides a robust, user-friendly, and efficient method to estimate the PDF uncertainties associated to theoretical calculations for the upcoming Run III of the LHC and beyond.

quantum chromodynamics↗

Extreme flood return levels in a U.S. mid-Atlantic estuary using 40-year fluvial-coastal model simulations

Using an integrated watershed-coastal modeling framework, we conducted long-term historical simulations (1980-2019) of fluvial and coastal flooding in the Delaware Bay and River, a vulnerable estuarine system in the U.S., at high spatial resolutions. By applying process-based models, we generated physically consistent and spatially detailed estimates of estuarine, riverine, and surge-driven extreme water level compared to previous studies that used field data only. We then evaluated changes in the magnitude of flood events using the 40-year simulations and detrended Floodwater Depth values with stationary extreme value analysis. Our detailed assessment of spatial-varying extreme values revealed how different flood-generation mechanisms can dominate various zones in the estuary. The datasets produced through this work will be valuable for long-term flood hazard mitigation planning in coastal communities in the Delaware Bay and River region. Additionally, this work will serve as a benchmark for other coastal flood hazard modeling communities worldwide, aiding them in systematically modeling long-term and continuous extreme flood events.

Coastal and Riverine Flooding↗

Latent Twins

Over the past decade, scientific machine learning has transformed the development of mathematical and computational frameworks for analyzing, modeling, and predicting complex systems. From inverse problems to numerical partial differential equations (PDEs), dynamical systems, and model reduction, these advances have pushed the boundaries of what can be simulated. Yet they have often progressed in parallel, with representation learning and algorithmic solution methods evolving largely as separate pipelines. With Latent Twins, we propose a unifying mathematical framework that creates a hidden surrogate in latent space for the underlying equations. Whereas digital twins mirror physical systems in the digital world, Latent Twins mirror mathematical systems in a learned latent space governed by operators. Through this lens, classical modeling, inversion, model reduction, and operator approximation all emerge as special cases of a single principle. We establish the fundamental approximation properties of Latent Twins for both ordinary differential equations (ODEs) and PDEs and demonstrate the framework across three representative settings: (i) canonical ODEs, capturing diverse dynamical regimes; (ii) a PDE benchmark using the shallow-water equations, contrasting Latent Twin simulations with deep operator network and forecasts with a four-dimensional variational method baseline; and (iii) a challenging real-data geopotential reanalysis dataset, reconstructing and forecasting from sparse, noisy observations. Latent Twins provide a compact, interpretable surrogate for solution operators that evaluate across arbitrary time gaps in a single-shot, while remaining compatible with scientific pipelines such as assimilation, control, and uncertainty quantification. Looking forward, this framework offers scalable, theory-grounded surrogates that bridge data-driven representation learning and classical scientific modeling across disciplines.

Latent Twins↗