Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

The LHC Olympics 2020 a community challenge for anomaly detection in high energy physics

A new paradigm for data-driven, model-agnostic new physics searches at colliders is emerging, and aims to leverage recent breakthroughs in anomaly detection and machine learning. In order to develop and benchmark new anomaly detection methods within this framework, it is essential to have standard datasets. To this end, we have created the LHC Olympics 2020, a community challenge accompanied by a set of simulated collider events. Participants in these Olympics have developed their methods using an R&D dataset and then tested them on black boxes: datasets with an unknown anomaly (or not). Furthermore, methods made use of modern machine learning tools and were based on unsupervised learning (autoencoders, generative adversarial networks, normalizing flows), weakly supervised learning, and semi-supervised learning. This paper will review the LHC Olympics 2020 challenge, including an overview of the competition, a description of methods deployed in the competition, lessons learned from the experience, and implications for data analyses with future datasets as well as future colliders.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Bay Area Regional Energy: Network Integrated Commercial Retrofits (BRICR) Project. Final Report

The BRICR project applied large-scale building energy modeling concepts with the aim of reducing the cost of energy efficiency targeting, design, and project development, and measurement of energy savings for energy efficiency programs implemented by local governments that serve small and medium commercial buildings (SMB). The project leveraged the services and resources of existing local government energy programs serving disadvantaged and hard-to-reach SMB customers. In contrast to programs run by utilities, local government programs generally do not have direct access to energy billing records for an entire class of customers in a geographic area, which prior research demonstrated useful for large-scale building energy model baseline development and calibration. , However, local governments are rich in public records that offer important clues about physical attributes and uses that, along with behavior, determine energy use. Relying only on public records, BRICR demonstrated development of credible baseline energy models for 3,792 office, retail, and hotel buildings. Publicly disclosed annual energy use data from a local energy benchmarking program and anonymized data from the Building Performance Database, the nation’s largest dataset about energy-related characteristics of buildings, were utilized to validate and calibrate energy models via an innovative method comparing distributions of energy intensity by fuel type for portfolios of buildings of similar size, vintage, and use. Portfolio calibration does not provide certainty that an energy model fits an individual building; the method is useful when billing data is not accessible – a common situation for researchers, energy service providers and ESCOs, local governments, and any party other than a utility. A software component was developed, the BRICR gem, which automates simulation when relevant data is added or edited by the user to a file saved in the standardized BuildingSync XML schema for energy audit data. The component was demonstrated as a simplified means to generate a mass of energy models corresponding to public records containing basic attributes such as building scale, location, use, year built, and aspect ratio in combination with building energy code prototype data corresponding to use and vintage. The component was also demonstrated as a simplified means to automate energy simulation when attributes are revised; the intention was to enable iterative improvement of the baseline model and energy savings estimates for common energy conservation measures as users revise relevant attributes based on their observations. In the context of institutional change and uncertainty for the participating local government energy programs, 13 whole building retrofits were completed. Impacts were measured by applying the CalTRACK2.0 methods to standardize measurement of normalized metered energy consumption. The GRIDMeter methods of stratified sampling and individual load shape analysis were applied to adjust for impacts of the effect of COVID-19 on retrofitted buildings in the context of all local buildings of similar size and use. Excluding impacts of the pandemic, retrofitted buildings demonstrated between 1.6% and 25.1% reduction in energy use. The project contributed use cases and feedback that helped inform evolution of the software tools and data formats that were combined for the first time in the BRICR project.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Structural- and Functional-Informed Machine Learning for Protein Function Prediction

In this project we aimed to extend methods for protein function prediction to include structural prediction data, and benchmark methods against existing tools. We proposed to apply the method to large metagenome datasets, and develop approaches to examine activity-based protein profiling results for protein function-structure patterns. Nitrogen cycle protein families were previously identified and are used here to provide a proof-of-principle for use of structure prediction in protein function classification.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

SFCOMPO Database of Spent Nuclear Fuel Assay Data – the Next Frontier

SFCOMPO is the world’s largest database for measured spent nuclear fuel assay data. An international effort coordinated by the Nuclear Energy Agency (NEA) resulted in a significant expansion of the database and its release online in 2017 as a downloadable application. The SFCOMPO Technical Review Group (TRG) was recently formed under the direction of NEA’s Nuclear Science Committee/Working Party on Nuclear Criticality Safety and was mandated to maintain and further coordinate the development of SFCOMPO. This TRG is currently focused on (1) critical evaluation of the experimental assay data by independent experts and (2) development of benchmarks and benchmark models that can be applied to validate burnup codes. This will improve the quality and documentation of the experimental datasets and enable their use by the international community to support code validation for design and safety analysis of spent nuclear fuel transportation, storage, and repository applications. It follows the precedent and draws on the experience gained from similar NEA efforts in the International Reactor Physics Experiment Evaluation Project and the International Criticality Safety Benchmark Experiment Project. Ongoing SFCOMPO evaluations have served as a test bed to develop templates for documenting evaluations, develop review guidance, improve approaches for a global uncertainty analysis, and devise a strategy focused on providing practical information of highest value to the user community. The current effort, status, and associated challenges are discussed.

Ilas, Germina↗
DOE OSTI2024

I/O performance studies of analysis workloads on production and dedicated resources at CERN

The recent evolutions of the analysis frameworks and physics data formats of the LHC experiments provide the opportunity of using central analysis facilities with a strong focus on interactivity and short turnaround times, to complement the more common distributed analysis on the Grid. In order to plan for such facilities, it is essential to know in detail the performance of the combination of a given analysis framework, of a specific analysis and of the installed computing and storage resources. This contribution describes performance studies performed at CERN, using the EOS disk-based storage, either directly or through an XCache instance, from both batch resources and highperformance compute nodes which could be used to build an analysis facility. A variety of benchmarks, both synthetic and based on real-world physics analyses and their corresponding input datasets, are utilized. In particular, the RNTuple format from the ROOT project is put to the test and compared to the latest version of the TTree format, and the impact of caches is assessed. In addition, we assessed the difference in performance between the use of storage system specific protocols, like XRootd, and FUSE. The results of this study are intended to be a valuable input in the design of analysis facilities, at CERN and elsewhere.

Sciabà, Andrea↗
DOE OSTI2021

Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

Source code for Brain Attend and Decode paper. Functional magnetic resonance imaging (fMRI) is a neuroimaging modality that captures the blood oxygen level in a subject's brain while the subject either rests or performs a variety of functional tasks under different conditions. Given fMRI data, the problem of inferring the task, known as task state decoding, is challenging due to the high dimensionality (hundreds of million sampling points per datum) and complex spatio-temporal blood flow patterns inherent in the data. In this work, we propose to tackle the fMRI task state decoding problem by casting it as a 4D spatiotemporal classification problem. We present a novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project-Young Adult (HCP-YA) dataset. We also investigate the transferability of BAnD's extracted features on unseen HCP tasks, either by freezing the spatial feature extraction layers and retraining the temporal model, or finetuning the entire model. The pre-trained features from BAnD are useful on similar tasks while finetuning them yields competitive results on unseen tasks/conditions.

Ng, BrendaM.↗
DOE OSTI2024

Established Regulatory Limits for Surface Water and Groundwater

The regulatory limits for surface water and groundwater contained in this database comprise chemical-specific preliminary remediation goals (PRGs) based on potential Applicable or Relevant and Appropriate Requirements (ARARs). These values should be used in conjunction with PRGs based on risk assessment to ensure that the PRGs for a site meet the residual risk requirements for protection of human health and the environment in the National Oil and Hazardous Substance Pollution Contingency Plan (NCP). In some cases, the values presented in this database may be To-Be-Considered (TBC) Guidance because these values are superseded by site-specific state requirements. ARARs are available for the Federal government, California, Georgia, Illinois, Kentucky, New Jersey, New Mexico, Nevada, New York, Ohio, South Carolina, Tennessee, Washington, and Wisconsin. The dataset provides maximum contaminant levels for drinking water and water quality criteria for fresh and marine environments based on acute and chronic exposures. Users can select ARAR sources, drinking water standards and water quality standards, individual chemicals, and retrieve results in tabular or spreadsheet formats for analysis. ARARs are derived from authoritative sources, including federal and state government agencies. The dataset supports environmental risk assessments, regulatory decision-making, and environmental planning, with tools for benchmarking against promulgated standards. This structured approach ensures a robust evaluation of environmental risks tailored to regulatory needs.

Stewart, Debra [Oak Ridge National Laboratory (ORN↗
DOE OSTI2024

Chemical-specific Parameters Dataset

The chemical-specific parameters dataset is searchable for physicochemical information for multiple chemicals simultaneously. After selecting chemicals of interest and the desired parameters, the RAIS will generate a table containing the values, chosen according to an established hierarchy. Results can be downloaded in Excel format. Over 40 parameters are available, including melting point, boiling point, density, density, vapor pressure, water solubility, and Henry’s Law constants. Thirteen primary sources are used to populate the dataset of chemical-specific parameters. These values should be used in cancer risk and noncancer hazard assessments for the calculation of preliminary remediation goals (PRGs), hazard characterization, and transport modeling. Users can select up to 1000 chemicals per query. The dataset supports environmental risk assessments, regulatory decision-making, and environmental planning with tools for benchmarking against risk-based standards. This structured approach ensures a robust evaluation of environmental risks tailored to regulatory needs.

Dolislager, Fred [Oak Ridge National Laboratory (O↗
DOE OSTI2024

Toxicity Values for Chemicals

The toxicity values for chemicals contained in this dataset comprise acute, subchronic, and chronic exposure durations. Cancer slope factors, inhalation unit risk, reference dose, and reference concentrations are available. These values should be used in cancer risk and noncancer hazard assessments for the calculation of preliminary remediation goals (PRGs) and hazard characterization. Users can select toxicity values from 10 combinations of exposure durations and cancer/noncancer toxicity values and select up to 1000 chemicals per query. The dataset supports environmental risk assessments, regulatory decision-making, and environmental planning with tools for benchmarking against risk-based standards. This structured approach ensures a robust evaluation of environmental risks tailored to regulatory needs.

Stewart, Debra [Oak Ridge National Laboratory (ORN↗
DOE OSTI2024

Radionuclide-specific Parameters Dataset

The radionuclide-specific parameters dataset is searchable for radiological information for multiple isotopes simultaneously. After selecting radionuclides of interest and the desired parameters, the RAIS will generate a table containing the values, chosen according to an established hierarchy. Results can be downloaded in Excel format. 50 parameters are available, including atomic number, soil to animal transfer coefficients, plant uptake coefficients, half-life, specific activity, and water solubility. Seven primary sources are used to populate the dataset of radiological-specific parameters. These values should be used in cancer risk assessments for the calculation of preliminary remediation goals (PRGs), hazard characterization, and transport modeling. Users can select up to 1000 radionuclides per query. The dataset supports environmental risk assessments, regulatory decision-making, and environmental planning with tools for benchmarking against risk-based standards. This structured approach ensures a robust evaluation of environmental risks tailored to regulatory needs.

Manning, Karessa [Oak Ridge National Laboratory (O↗
DOE OSTI2025

Analyzing the Impact of Future Weather Data on Energy Consumption in Weatherization Assistant

This study supports the mission of the U.S. Department of Energy’s Weatherization Assistance Program (WAP), which aims to increase the energy efficiency of dwellings and reduce their total residential expenditures. Specifically, we examine how projected future climate conditions may affect residential building energy performance by integrating future weather data into the National Energy Audit Tool (NEAT). Since WAP evaluates the cost-effectiveness of retrofit measures over lifespans of up to 30 years, accounting for evolving climate conditions is increasingly important. To reflect future household energy demands, this study replaces historically based Typical Meteorological Year (TMY3) weather inputs with Future Typical Meteorological Year (fTMY) datasets derived from global climate model (GCM) projections. A simulation-based framework was established to enable NEAT analysis under future weather conditions. This workflow involves converting EPW-format weather files into JSON inputs compatible with NEAT and generating degree-hour metrics needed for load calculations. The fTMY dataset used in this study was developed by Oak Ridge National Laboratory through downscaling of six GCMs under different emission scenarios and covers the period from 2020 to 2100. In contrast, the TMY3 dataset is based on historical weather data from 1961 to 1990. Simulations were conducted for benchmark single-family prototype buildings across ASHRAE climate zones 1–7, which cover all regions of the U.S. except the subarctic Zone 8 in northern Alaska, evaluating both heating and cooling loads under TMY3 and fTMY conditions. Four foundation types were tested, while heating systems were standardized, as NEAT does not differentiate thermal energy load by HVAC system type in its load calculations. Results show that fTMY weather input consistently yield lower heating loads and higher cooling loads across most locations, aligning with expected climate warming trends. Notably, colder regions such as zones 6A, 6B, and 7 experience marked reductions in heating load, while warmer and transitional zones, such as 2A (Lufkin, TX) and 3C (San Francisco, CA), have substantial increases in cooling loads. Although this study does not directly assess the performance of retrofit measures under future climate conditions, it provides a critical foundation for doing so. By quantifying shifts in baseline (i.e., pre-retrofit case) energy loads between historical and future weather files, the study highlights the importance of integrating climate-responsive data into audit tools. These findings will inform future efforts to evaluate the long-term effectiveness and cost-effectiveness of weatherization measures under changing climate conditions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Multicode benchmark on simulated Ti K-edge x-ray absorption spectra of Ti-O compounds

X-ray absorption spectroscopy (XAS) is an element-specific materials characterization technique that is sensitive to structural and electronic properties. First-principles simulated XAS has been widely used as a powerful tool to interpret experimental spectra and draw physical insights. Recently, there has also been growing interest in building computational XAS databases to enable data analytics and machine learning applications. However, there are non-trivial differences among commonly used XAS simulation codes, both in underlying theoretical formalism and in technical implementation. Reliable and reproducible computational XAS databases require systematic benchmark studies. In this work, we benchmarked Ti K-edge XAS simulations of ten representative Ti-O binary compounds, which we refer to as the Ti-O-10 dataset, using three state-of-the-art codes: XSPECTRA, OCEAN and exciting. We systematically studied the convergence behavior with respect to the input parameters and developed a workflow to automate and standardize the calculations to ensure converged spectra. Our benchmark comparison considers a 35 eV spectral range starting from the K-edge onset, representative of widely used near-edge spectra. Quantitative comparison over this range is based on Spearman’s rank correlation score (r sp ). Our results show: (1) the two Bethe-Salpeter equation (BSE) codes (OCEAN and exciting) have excellent agreement with an average $r$ $sp$ of 0.998; (2) good agreement is obtained between the core-hole potential code (XSPECTRA) and BSE codes (OCEAN and exciting) with an average $r$ $sp$ of 0.990, and this smaller $r$ $sp$ reflects the noticeable differences in the main edge spectral shape that can be primarily attributed to the difference in the strength of the screened core-hole potential; (3) simulations from both methods overall reproduce well the main experimental spectral features of rutile and anatase, and the different treatments of the screened core-hole potential have visible impact on pre-edge intensities and the peak ratio of the main edge; (4) there exist moderate differences in the relative edge alignment of the three codes with a standard deviation of about 0.2 eV, which arise from multiple contributions including the frozen core approximation, final state effects, and different approximations used for the self-energy correction. In conclusion, our benchmark study provides important standards for first-principles XAS simulations with broad impact in data-driven XAS analysis.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Implementation of a Binary Neural Network on a Passive Array of Magnetic Tunnel Junctions

The increasing scale of neural networks and their growing application space have produced demand for more energy- and memory-efficient artificial-intelligence-specific hardware. Avenues to mitigate the main issue, the von Neumann bottleneck, include in-memory and near-memory architectures, as well as algorithmic approaches. In this report we leverage the low-power and the inherently binary operation of magnetic tunnel junctions (MTJs) to demonstrate neural network hardware inference based on passive arrays of MTJs. In general, transferring a trained network model to hardware for inference is confronted by degradation in performance due to device-to-device variations, write errors, parasitic resistance, and nonidealities in the substrate. To quantify the effect of these hardware realities, we benchmark 300 unique weight matrix solutions of a two-layer perceptron to classify the Wine dataset for both classification accuracy and write fidelity. Despite device imperfections, we achieve software-equivalent accuracy of up to 95.3% with proper tuning of network parameters in 15 x 15 MTJ arrays having a range of device sizes. The success of this tuning process shows that new metrics are needed to characterize the performance and quality of networks reproduced in mixed signal hardware.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

classLog: Logistic regression for the classification of genetic sequences

Introduction Sequencing and phylogenetic classification have become a common task in human and animal diagnostic laboratories. It is routine to sequence pathogens to identify genetic variations of diagnostic significance and to use these data in realtime genomic contact tracing and surveillance. Under this paradigm, unprecedented volumes of data are generated that require rapid analysis to provide meaningful inference. Methods We present a machine learning logistic regression pipeline that can assign classifications to genetic sequence data. The pipeline implements an intuitive and customizable approach to developing a trained prediction model that runs in linear time complexity, generating accurate output rapidly, even with incomplete data. Our approach was benchmarked against porcine respiratory and reproductive syndrome virus (PRRSv) and swine H1 influenza A virus (IAV) datasets. Trained classifiers were tested against sequences and simulated datasets that artificially degraded sequence quality at 0, 10, 20, 30, and 40%. Results When applied to a poor-quality sequence data, the classifier achieved between >85% to 95% accuracy for the PRRSv and the swine H1 IAV HA dataset and this increased to near perfect accuracy when using the full dataset. The model also identifies amino acid positions used to determine genetic clade identity through a feature selection ranking within the model. These positions can be mapped onto a maximum-likelihood phylogenetic tree, allowing for the inference of clade defining mutations. Discussion Our approach is implemented as a python package with code available at https://github.com/flu-crew/classLog .

Zeller, Michael A.↗
DOE OSTI2025

The ePIC Simulation Campaign Workflow on the Open Science Grid

The ePIC collaboration is realizing the first experiment of the future Electron-Ion Collider (EIC) at the Brookhaven National Laboratory that will allow for a precision study of the nucleons and the nucleus at the scale of sea quarks and gluons through the study of electron-proton/ion collisions. This paper will discuss the current workflow for running centralized simulation campaigns for ePIC on the Open Science Grid (OSG) infrastructure. This involves monthly releases of ePIC software and container deployments to CVMFS, generation of input datasets in HepMC format according to collaboration-defined policy, using Snakemake in CI/CD for validation and benchmarking, and submitting jobs to the OSG condor scheduler for opportunistic running on available resources. File transfers utilize XrootD, and Rucio is used for data management. The workflow is continuously refined to improve daily throughput (currently 50-100k core hours per day) and minimize job failures. Since May 2023, monthly simulation campaigns employing the workflow have cumulatively used over 20 million core hours on the OSG and produced over 350 TB of simulation data. The campaigns incorporate simulations for the broad science program of the EIC and are actively used for the detector and physics studies in preparation of the Technical Design Report (TDR).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Ptychographic phase retrieval by proximal algorithms

We derive a set of ptychography phase-retrieval iterative engines based on proximal algorithms originally developed in convex optimization theory, and discuss their connections with existing ones. The use of proximal operator creates a simple frame work that allows us to incorporate the effect of noise from a maximum-likelihood (ML) principle. We focus on three particular algorithms, namely proximal minimization, alternating direction method of multiplier and accelerated proximal gradient (APG). We benchmark their performance with numerical simulations, and discuss their optimal conditions for convergence and accuracy. An experimental dataset is used to demonstrate their effectiveness as well, in which case an array of cubic Au nanoparticles with a size of 50 nm is imaged. We show that with the presence of Poisson noise, a dataset with photon counts up to 10 4 at one detector pixel already requires ML-based methods to achieve a stable convergence. Among the three algorithms derived in this work, APG method is reported first time for its application in ptychographic reconstruction and shows superior performance in terms of both accuracy and convergence rate with a noisy dataset.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Transformative Materials for High-Efficiency Thermochemical Production of Solar Fuels (Final Report)

Metal oxide-based two-step solar thermochemical (STC) H 2 O and CO 2 -splitting cycles are a promising route to convert solar thermal energy into fuels. The metal oxide materials are reduced at high temperatures (Step 1), and then at low (but still elevated) temperatures, the reduced oxide is used to split H 2 O or CO 2 (Step 2). However, current applications of these cycles are limited by the efficiency of the metal oxide materials. A lower temperature for reduction is desirable, but that brings a concomitant reduction in the driving force for gas splitting. So, designing novel, high-efficiency materials is challenging. Here, we devised and performed a joint computational-experimental project, combined with materials design strategies and high-throughput approaches with the goal to quickly discover and demonstrate novel thermochemical materials with superior properties. Our approach involves an active feedback loop between experimental and computational research. We have use our previously developed materials design map, to efficiently screen results from high-throughput first- principles computation to predict new compositions and subsequently to experimentally study the properties of novel, predicted materials. Experimental measurements of the thermodynamic properties of selected materials serve as a critical benchmark, not only in evaluating the STCH properties of synthesized compounds, but also serving as a validation dataset for computational approaches. We experimentally explored a set of predicted ABO3 perovskites and focus on obtaining high quality thermodynamic properties for validation of computational prediction of enthalpy and entropy of reduction. These thermodynamic quantities play a major role in designing materials with reduced temperatures of reduction but sufficient gas-splitting rates. Finding good validation between experimental and computational data for enthalpies of reduction, we turn to promising ABO3 materials modified by A and B site substitutions, opening an enormous combinatorial space of materials. We use our high-throughput approach to “tune in” the desired solar thermochemical (STC) properties for STC. This vast composition space can only be reasonably explored using the high-throughput approaches, both computational and experimental, of this proposal. Several promising novel oxide materials have been predicted, synthesized, and verified using our materials design approach.

08 HYDROGEN↗