Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Graph neural networks for CO 2 solubility predictions in Deep Eutectic Solvents

Deep Eutectic Solvents (DESs) are a promising class of solvents for CO 2 capture. DESs are complex mixtures that can be designed to optimize CO solubility and overall capture process efficiency. However, the vast design landscape of DES mixtures makes experimental investigation prohibitive; as such, there is a need for computational models that can quickly and efficiently navigate the design space and inform data collection efforts. In this work, we propose Graph Neural Network (GNN) models for predicting CO 2 solubility for DESs; the GNN leverages a mixture graph representation that captures the molecular structure of the DES components as well as their intermolecular interactions. Here, we compare the GNN framework against alternative architectures (neural networks, graph convolution networks, and random forests) and data representations (molecular fingerprints, sigma profiles, and graphs). We show that the proposed approach offers superior predictive performance; specifically, we show that solubility can be predicted reliably directly from molecular structure (without the need of using sigma profiles as proposed in previous studies). This result is important, as obtaining sigma profiles requires expensive density functional theory computations. We also explored the ability of GNNs to predict solubility for new DES mixtures and operating conditions. We found that the model extrapolates across temperature reliably. However, we also found deficiencies in the ability of the model to predict solubility for DES mixtures, pressures, and molar ratio not included in the training sets; we show that this is due to an inherent lack of chemical diversity in datasets available in the literature. The proposed computational capabilities can thus help navigate the design space of DES and inform data collection efforts. Our models, data, and benchmarks are shared as Python code implemented in Jupyter notebooks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Vehicle Tracking with Crop-based Detection

End-to-end production of vehicle tracking data from video in real-time and with high accuracy remains a challenging problem due to the computational cost of object detection on each frame. In this work we present Tracking with Crop-based Detection, a method for speeding object tracking in constrained contexts (with stable cameras and relatively-predictable object motion) such as vehicle traffic monitoring. We leverage this context to provide a strong prior for object locations, which we use to 1.) boost detection speed by detecting objects only in regions corresponding to object priors on most frames and 2.) inform the selection of the detector output for each object. We evaluate Crop-based Detection as an extension to the KIOU object tracker (Crop-KIOU) on the UA-DETRAC dataset. The proposed tracker outperforms all other reported algorithms in terms of PR-MOTA, PR-MOTP, and mostly tracked objects on the UA-DETRAC benchmark, establishing a new state-of-the-art. Relative to tracking by detection with KIOU, Crop-KIOU achieves a 26% higher frame-rate and increases accuracy. Furthermore, Tracking with Crop-based Detection can be combined with frame skipping; we show a 149% increase in framerate relative to KIOU with no decrease in accuracy using this combination of methods.

Gloudemans, Derek↗

A Summary of Validation Studies for the Integrated TIGER Series Performed on ACORN Plus-up 218468/99

The Integrated TIGER Series (ITS) transport code is a valuable tool for photon-electron transport. A seven-problem validation suite exists to make sure that the ITS transport code works as intended. It is important to ensure that data from benchmark problems is correctly compared to simulated data. Additionally, the validation suite did not previously make use of a consistent quantitative metric for comparing experimental and simulated datasets. To this end, the goal of this long-term project was to expand the validation suite both in problem type and in the quality of the error assessment. To accomplish that, the seven validation problems in the suite were examined for potential drawbacks. When a drawback was identified, the problems were ranked based on severity of the drawback and approachability of a solution. We determined that meaningful improvements could be made to the validation suite by improving the analysis for the Lockwood Albedo problem and by introducing the Ross dataset as an eighth problem to the suite. The Lockwood error analysis has been completed and will be integrated in the future. The Ross data is unfinished, but significant progress has been made towards analysis.

97 MATHEMATICS AND COMPUTING↗

On Final Results From the Muon $g\mathrm{-}2$ Experiment at Fermilab

The Muon g-2 Experiment at Fermilab has measured the muon anomalous magnetic moment, a_mu, with unprecedented precision, leveraging a dataset from Runs 1 6 that is 21 times larger than its Brookhaven predecessor. This talk will present the experiment s final result, which serves as a benchmark for testing the Standard Model with high precision. The measurement was performed in a storage ring with a highly uniform magnetic field, where precise beam dynamics understanding is vital to determine the muon anomalous precession frequency. Key to this effort were advanced simulation tools including COSY INFINITY, precise fringe field modeling, and calculations of beam dynamics characteristics like tunes and chromaticity. We will conclude by exploring how the experiment's findings reshape our understanding of particle physics.

Valetov, Eremey [Michigan State U.]↗

Nuclear Dependence of Charged Current Inclusive $\bar{\nu}_\mu -$A Cross Section Measurements at 6 GeV at MINERvA

The main advantage of the MINERvA detector is that it uses differently sized nuclear targets, in the same neutrino beam, allowing for a direct study of nuclear medium effects, with reduced flux and detector uncertainties. This thesis presents the first high statistics charged current antineutrino-nucleus inclusive two-dimensional cross section measurements using an antineutrino beam with $\langle E_{\bar\nu}\rangle\sim$ 6 GeV, on iron, lead, carbon and hydrocarbon targets, as a function of bjorken $x$ and four-momentum transfer squared $Q^2$, invariant mass $W$ and $Q^2$ and muon longitudinal and transverse momenta, $p_t$ and $p_z$. These nuclei of different sizes were utilized to measure the nuclear dependence of the cross section. The antineutrino interactions that were utilized for the analysis presented in this thesis, were having an antimuon in the final state with energy 2-20GeV and with an antimuon scattering angle less than $17^o$. This analysis represents one of the largest antineutrino datasets analyzed to date in the energy region of $\langle E_{\bar\nu} \rangle \sim$ 6 GeV. This direct evidence for the nuclear medium effects, can help benchmarking models, which can be used by future neutrino experiments like DUNE, by extrapolating these results to Argon target.

Gaur, Prameet Kumar↗

An Open Combinatorial Diffraction Dataset Including Consensus Human and Machine Learning Labels with Quantified Uncertainty for Training New Machine Learning Models

Modern machine learning and autonomous experimentation schemes in materials science rely on accurate analysis of the data ingested by these models. Unfortunately, accurate analysis of the underlying data can be difficult, even for domain experts, complicating the training of the models intended to drive experiments. This is especially true when the goal is to identify the presence of weak signatures in diffraction or spectroscopic datasets. In this work, we examine a set of as-obtained diffraction data that track the phase transition from monoclinic to tetragonal in a Nb-doped VO2 film as a function of temperature and dopant concentration. We then task a set of domain experts and a set of machine learning experts with identifying which phase is present in each diffraction pattern manually and algorithmically, respectively; in both cases, the labels can vary dramatically, especially at the phase boundaries. We use the mode of the labels and the Shannon entropy as a method to capture, preserve and propagate consensus labels and their variance. Further we use the expert labels as a benchmark and demonstrate the use of Shannon entropy weighted scoring to test the performance of machine learning generated labels. Finally, we propose a material data challenge centered around generating improved labeling algorithms. This real-world dataset curated with expert labels can act as test bed for new algorithms. The raw data, annotations and code used in this study are all available online at data.gov and the interested reader is encouraged to replicate and improve the existing models

97 MATHEMATICS AND COMPUTING↗

Advances in building data management for building performance standards using the SEED platform

Reducing energy consumption and greenhouse gas emissions in the built environment is a critical step in achieving emission goals to mitigate climate change impacts. Local, federal, and international jurisdictions are deploying several methods to reduce energy and emissions such as voluntary and mandatory benchmarking and building performance standards, requiring building owners to reach energy and emission targets. Jurisdictions leveraging benchmarking and building performance standards require knowledge of the buildings covered; which is a large task due to staffing constraints, limited information on building characteristics and tax parcel data, and the need for advanced data management techniques to align datasets. This paper describes an open-source platform's recent advances to create consistent taxonomies, identify erroneous data, enable auditability, and track building performance. The paper concludes with two use cases on how the platform has been used by jurisdictions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Systematic softening in universal machine learning interatomic potentials

Machine learning interatomic potentials (MLIPs) have introduced a new paradigm for atomic simulations. Recent advancements have led to universal MLIPs (uMLIPs) that are pre-trained on diverse datasets, providing opportunities for universal force fields and foundational machine learning models. However, their performance in extrapolating to out-of-distribution complex atomic environments remains unclear. In this study, we highlight a consistent potential energy surface (PES) softening effect in three uMLIPs: M3GNet, CHGNet, and MACE-MP-0, which is characterized by energy and force underprediction in atomic-modeling benchmarks including surfaces, defects, solid-solution energetics, ion migration barriers, phonon vibration modes, and general high-energy states. The PES softening behavior originates primarily from the systematically underpredicted PES curvature, which derives from the biased sampling of near-equilibrium atomic arrangements in uMLIP pre-training datasets. Our findings suggest that a considerable fraction of uMLIP errors are highly systematic, and can therefore be efficiently corrected. We argue for the importance of a comprehensive materials dataset with improved PES sampling for next-generation foundational MLIPs.

36 MATERIALS SCIENCE↗

Improving the Concrete Crack Detection Process via a Hybrid Visual Transformer Algorithm

Inspections of concrete bridges across the United States represent a significant commitment of resources, given their biannual mandate for many structures. With a notable number of aging bridges, there is an imperative need to enhance the efficiency of these inspections. This study harnessed the power of computer vision to streamline the inspection process. Our experiment examined the efficacy of a state-of-the-art Visual Transformer (ViT) model combined with distinct image enhancement detector algorithms. We benchmarked against a deep learning Convolutional Neural Network (CNN) model. These models were applied to over 20,000 high-quality images from the Concrete Images for Classification dataset. Traditional crack detection methods often fall short due to their heavy reliance on time and resources. This research pioneers bridge inspection by integrating ViT with diverse image enhancement detectors, significantly improving concrete crack detection accuracy. Notably, a custom-built CNN achieves over 99% accuracy with substantially lower training time than ViT, making it an efficient solution for enhancing safety and resource conservation in infrastructure management. These advancements enhance safety by enabling reliable detection and timely maintenance, but they also align with Industry 4.0 objectives, automating manual inspections, reducing costs, and advancing technological integration in public infrastructure management.

42 ENGINEERING↗

Machine learning and ligand binding predictions: A review of data, methods, and obstacles

We report that computational predictions of ligand binding is a difficult problem, with more accurate methods being extremely computationally expensive. The use of machine learning for drug binding predictions could possibly leverage the use of biomedical big data in exchange for time-intensive simulations. This paper reviews current trends in the use of machine learning for drug binding predictions, data sources to develop machine learning algorithms, and potential problems that may lead to overfitting and ungeneralizable models. A few popular datasets that can be used to develop virtual high-throughput screening models are characterized using spatial statistics to quantify potential biases. We can see from evaluating some common benchmarks that good performance correlates with models with high-predicted bias scores and models with low bias scores do not have much predictive power. A better understanding of the limits of available data sources and how to fix them will lead to more generalizable models that will lead to novel drug discovery.

59 BASIC BIOLOGICAL SCIENCES↗

Modeling Oxidative Dehydrogenation of Propane with Supported Vanadia Catalysts Using Multireference Methods

The oxidative dehydrogenation of propane over supported vanadium oxide catalysts poses significant computational challenges due to complex electronic structure changes along the reaction coordinate, driven primarily by changes in the oxidation states of vanadium. To address these challenges, we systematically test quantum chemical methods, including multireference (MR) approaches, domain-based local pair natural orbital coupled cluster theory (DLPNO-CCSD(T)), and density functional theory (DFT). The initial C–H bond-breaking transition state requires MR treatment due to its multireference character, while subsequent steps permit efficient single-reference calculations. For the rate-limiting C–H activation step mediated by the vanadyl moiety, complete 1 active space second-order perturbation theory (CASPT2) yields an apparent activation barrier (E app 600K ) of 138 kJ/mol, consistent with experimental values (134 ± 4 kJ/mol; Gruene et al. Catal. Today 2010, 157, 137). In contrast, DLPNO-CCSD(T) overestimates this barrier (198 kJ/mol), whereas DFT predictions span 125–150 kJ/mol, depending on the functional. Our multireference investigation of this transition metal oxide-catalyzed process demonstrates that an active space that incorporates the C–H σ and V=O σ/π bonding orbitals, oxygen lone pairs, and their antibonding counterparts adequately captures electronic structure changes along the chemical transformation. Furthermore, these findings provide a general strategy for active space selection in transition metal oxide-catalyzed C/O–H bond activation reactions. The reference dataset from this work, which includes MR calculations with manually selected active spaces for all intermediates and transition states in the propane ODH reaction network, will serve as a benchmark for automating active space selection in similar systems.

Catalysts↗

LandScan Mosaic Time Series Version 1.0: 1975-2024

The LandScan Mosaic Annual Global Ambient Population Time Series, Version 1.0, provides annual high-resolution global ambient population estimates from 1975 to 2024. The dataset is distributed as one Cloud-Optimized GeoTIFF per year on a globally aligned 3 arc-second WGS 84 grid. The 2024 layer is the LandScan Mosaic benchmark reference year, while the 1975–2023 layers are historical estimates generated using the LandCast backcasting framework.

Zimmer, Andrew [ORNL] (ORCID:0000000176838713)↗

DeepBench: A simulation package for physical benchmarking data

We introduce **DeepBench**, a python library that generates simple simulated image data from first principles, such as basic geometric shapes and astronomical objects. These data are highly valuable for developing (calibration, testing, and benchmarking) statistical and machine learning models because they make it possible to connect the final data product to physically interpretable inputs. This software includes tools to curate and store the datasets to maximize reproducibility.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DeepPatent2: A Large-Scale Benchmarking Corpus for Technical Drawing Understanding

Abstract Recent advances in computer vision (CV) and natural language processing have been driven by exploiting big data on practical applications. However, these research fields are still limited by the sheer volume, versatility, and diversity of the available datasets. CV tasks, such as image captioning, which has primarily been carried out on natural images, still struggle to produce accurate and meaningful captions on sketched images often included in scientific and technical documents. The advancement of other tasks such as 3D reconstruction from 2D images requires larger datasets with multiple viewpoints. We introduce DeepPatent2, a large-scale dataset, providing more than 2.7 million technical drawings with 132,890 object names and 22,394 viewpoints extracted from 14 years of US design patent documents. We demonstrate the usefulness of DeepPatent2 with conceptual captioning. We further provide the potential usefulness of our dataset to facilitate other research areas such as 3D image reconstruction and image retrieval.

97 MATHEMATICS AND COMPUTING↗

Resilience Assessment for Distribution Systems during Hurricanes: A Learning-Based Framework

This paper presents a proactive strategy for hurricane-resilient distribution systems. It proposes a Bayesian Neural Network-based outage prediction model considering various parameters, including electrical components, and weather and environmental factors. Addressing challenges in imbalanced outage datasets, a Bias-Variance Tradeoff method is proposed. A resilience assessment model quantifies resilience indices, providing insights into system weaknesses. The approach identifies weak points and serves as a planning benchmark. Numerical results on the modified IEEE 123-node test system demonstrate effectiveness in realistic hurricane scenarios.

Vahedi, Soroush↗

Benchmarking blockchain-based gene-drug interaction data sharing methods: A case study from the iDASH 2019 secure genome analysis competition blockchain track

Blockchain distributed ledger technology is just starting to be adopted in genomics and healthcare applications. Despite its increased prevalence in biomedical research applications, skepticism regarding the practicality of blockchain technology for real-world problems is still strong and there are few implementations beyond proof-of-concept. We focus on benchmarking blockchain strategies applied to distributed methods for sharing records of gene-drug interactions. We expect this type of sharing will expedite personalized medicine. We generated gene-drug interaction test datasets using the Clinical Pharmacogenetics Implementation Consortium (CPIC) resource. We developed three blockchain-based methods to share patient records on gene-drug interactions: Query Index, Index Everything, and Dual-Scenario Indexing. We achieved a runtime of about 60 s for importing 4,000 gene-drug interaction records from four sites, and about 0.5 s for a data retrieval query. Our results demonstrated that it is feasible to leverage blockchain as a new platform to share data among institutions.

60 APPLIED LIFE SCIENCES↗

Assessment of fine-tuned large language models for real-world chemistry and material science applications

The current generation of large language models (LLMs) has limited chemical knowledge. Recently, it has been shown that these LLMs can learn and predict chemical properties through fine-tuning. Using natural language to train machine learning models opens doors to a wider chemical audience, as field-specific featurization techniques can be omitted. In this work, we explore the potential and limitations of this approach. We studied the performance of fine-tuning three open-source LLMs (GPT-J-6B, Llama-3.1-8B, and Mistral-7B) for a range of different chemical questions. We benchmark their performances against “traditional” machine learning models and find that, in most cases, the fine-tuning approach is superior for a simple classification problem. Depending on the size of the dataset and the type of questions, we also successfully address more sophisticated problems. The most important conclusions of this work are that, for all datasets considered, their conversion into an LLM fine-tuning training set is straightforward and that fine-tuning with even relatively small datasets leads to predictive models. These results suggest that the systematic use of LLMs to guide experiments and simulations will be a powerful technique in any research study, significantly reducing unnecessary experiments or computations.

Van Herck, Joren↗

The PDF4LHC21 combination of global PDF fits for the LHC Run III

A precise knowledge of the quark and gluon structure of the proton, encoded by the parton distribution functions (PDFs), is of paramount importance for the interpretation of high-energy processes at present and future lepton–hadron and hadron–hadron colliders. Motivated by recent progress in the PDF determinations carried out by the CT, MSHT, and NNPDF groups, we present an updated combination of global PDF fits: PDF4LHC21. It is based on the Monte Carlo combination of the CT18, MSHT20, and NNPDF3.1 sets followed by either its Hessian reduction or its replica compression. Extensive benchmark studies are carried out in order to disentangle the origin of the differences between the three global PDF sets. In particular, dedicated fits based on almost identical theory settings and input datasets are performed by the three groups, highlighting the role played by the respective fitting methodologies. We compare the new PDF4LHC21 combination with its predecessor, PDF4LHC15, demonstrating their good overall consistency and a modest reduction of PDF uncertainties for key LHC processes such as electroweak gauge boson production and Higgs boson production in gluon fusion. We study the phenomenological implications of PDF4LHC21 for a representative selection of inclusive, fiducial, and differential cross sections at the LHC. The PDF4LHC21 combination is made available via the LHAPDF library and provides a robust, user-friendly, and efficient method to estimate the PDF uncertainties associated to theoretical calculations for the upcoming Run III of the LHC and beyond.

quantum chromodynamics↗