Search NASA⌕ Search

SEARCH · Search NASA

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Evaluation of Seismic Artificial Intelligence with Uncertainty

Artificial intelligence has transformed the seismic community with deep learning models (DLMs) that are trained to complete specific tasks within workflows. However, there is still a lack of robust evaluation frameworks for evaluating and comparing DLMs. Here, we address this gap by designing an evaluation framework that jointly incorporates two crucial aspects: performance uncertainty and learning efficiency. To target these aspects, we meticulously construct the training, validation, and test splits using a clustering method tailored to seismic data and enact an expansive training design to segregate performance uncertainty arising from stochastic training processes and random data sampling. The framework’s ability to guard against misleading declarations of model superiority is demonstrated through the evaluation of PhaseNet (Zhu and Beroza, 2018), a popular seismic phase picking DLM, under three training approaches. Our framework helps practitioners choose the best model for their problem and set performance expectations by explicitly analyzing model performance with uncertainty at varying budgets of training data.

58 GEOSCIENCES↗
DOE OSTI2020

ECP Report: Update on Proxy Applications and Vendor Interactions

The ExaLearn miniGAN team (Ellis and Rajamanickam) have released miniGAN, a generative adversarial network(GAN) proxy application, through the ECP proxy application suite. miniGAN is the first machine learning proxy application in the suite (note: the ECP CANDLE project did previously release some benchmarks) and models the performance for training generator and discriminator networks. The GAN's generator and discriminator generate plausible 2D/3D maps and identify fake maps, respectively. miniGAN aims to be a proxy application for related applications in cosmology (CosmoFlow, ExaGAN) and wind energy (ExaWind). miniGAN has been developed so that optimized mathematical kernels (e.g., kernels provided by Kokkos Kernels) can be plugged into to the proxy application to explore potential performance improvements. miniGAN has been released as open source software and is available through the ECP proxy application website (https://proxyapps.exascaleproject.ordecp-proxy-appssuite/) and on GitHub (https://github.com/SandiaMLMiniApps/miniGAN). As part of this release, a generator is provided to generate a data set (series of images) that are inputs to the proxy application.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Development of Gamma Background Radiation Digital Twin with Machine Learning Algorithms: Application of Unsupervised Machine Learning to Detection of Anomalies and Nuisances in Gamma Background Radiation Environmental Screening Data

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore unsupervised machine learning (ML) algorithms for development of a digital twin of gamma radiation background, and for detection and identification of weak nuisances and anomalies events in the presence of highly fluctuating background. In one segment of work, we developed a gamma background estimation model using a Longshort term memory (LSTM) network for one-step CPS time series prediction. The LSTM model was validated with two data sets of measurements from two independent NaI detectors positioned on a mobile platform. The data sets contained background radiation only and no orphan isotope sources. The LSTM model was constructed and tested using data from one of the detectors. Performance of the LSTM model was validate through one-step prediction of CPS time series of another NaI detector without re-training. This approach allows to create a digital twin for nuclear background estimation. Using LSTM, it could be possible to detect a source through subtraction of the estimated counts from the measured background. In another segment of work, we investigated detection of gamma emitting sources in the presence of complex background using unsupervised machine learning. Spectral lines of isotopes are difficult to observe in one-second measurements. Averaging over the entire measurement campaign data set reveals spectral lines of most common background isotopes. Spectral lines of orphan sources, which might appear only in a few measurements during the campaign, will be washed out if averaging is performed over the entire measurement data set. The approach we have explored consists of extracting one-second measurements containing weak spectral features through data clustering. Averaging one-second spectra in a cluster should reveal the presence of anomaly sources. We created two ML models using K-means clustering and Neural Network Self-organizing Map (SOM). Performance of these ML models was benchmarked using search data. One data set contained 137 Cs source, and another dataset contained 131 I source.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Closures and Simulation for Thermal Radiation Transport in Stochastic Media with Nonlinear Temperature Dependence

Because of the practical challenge of rendering very complex realistic spatial structures for numerical work, it is common practice to resort to characterizing such media as stochastic mixtures of materials, ideally parametrized with low order statistics such as the mean, variance, and correlation functions of the now random material properties. This enables realizations of the medium to be repeatedly generated and radiation transport computations to, in principle, be performed for a large ensemble of these realizations to obtain a statistically well-characterized radiation field. Statistical post-processing yields desired quantities such as conditional and unconditional mean radiation flux and probability distributions of transmitted radiation. However, such computations prove expensive for all but the simplest stochastic geometries and are most suited for benchmarking approximate models. The most common approximations lead to homogenized media so that transport computations are required only on a single medium realization but by construct provide only limited statistical information on the radiation field. Almost all approximate approaches to this problem attempt to develop equations for low order moments of the radiation intensity (mean, second moment, correlation function) but inevitably encounter a closure problem: the equation for any statistical moment will contain terms depending on unknown higher-order moments. Thus, the challenge shifts to one of developing closure relations that relate the unknown moments to the lower order moments. Under very special conditions, an exact closure can be derived but in general closures are heuristically stated constitutive relations. Also, closure approaches depend on whether the mixing statistics are spatially and/or temporally continuous as in fluctuating turbulent fields, or discontinuous as in randomly mixed solid chunks of material. Thus, unconditional averaging is generally applied in the former case but conditional averaging is more appropriate when the mixing is discontinuous. In this work, the emphasis is on binary statistical mixtures of immiscible fluids as as such the quantities of interest are averages (flux, temperature) conditioned on the material type.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Detection of Anomalies in Gamma Background Radiation Data with K-Means and Self-Organizing Map Clustering Algorithms (Consortium on Nuclear Security Technologies (CONNECT) Q1 Report)

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore unsupervised machine learning (ML) algorithms for detection and identification of weak nuisances and anomalies events in the presence of highly fluctuating background. The challenge is that spectral lines of isotopes are difficult to observe in one-second measurements. Averaging over the entire measurement campaign data set reveals spectral lines of most common background isotopes. Spectral lines of orphan sources, which might appear only in a few measurements during the campaign, will be washed out if averaging is performed over the entire measurement data set. The approach we have explored consists of extracting one-second measurements containing weak spectral features through data clustering. Averaging one-second spectra in a cluster should reveal the presence of anomaly sources. We created two ML models using K-means clustering and Neural Network Self-organizing Map (SOM). Performance of these ML models was benchmarked using search data. One data set contained 137 Cs source, and another dataset contained 131 I source.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2020

Griffin/Pronghorn PBMR-400 Benchmark Results [Slides]

Content Overview: 1) Pronghorn & Griffin, MOOSE based applications overview: Codes capabilities description & MOOSE ecosystem overiew; Current research activities and synergies with other projects; Different multiphysics coupling scheme description. 2) PBMR-400 Benchmark: OECD NEA Benchmark and Pronghorn/Griffin models description; Standalone NK Ex1 and TH Ex2 Results description; Coupled Steady state Ex3; Results analysis for Transiet Ex4a (100-40-100 Load Following) & Ex3 (Protected PLOFC). 3) Conclusions $ Future steps.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Brine Availability Test in Salt (BATS) FY24 Update

This report summarizes fiscal year 2024 (FY24) activities centered around a series of field tests in bedded salt at the Waste Isolation Pilot Plant (WIPP) funded by the Office of Spent Fuel and Waste Science and Technology in the Spent Fuel and Waste Disposition (SFWD) program of the US Department of Energy’s Office of Nuclear Energy (DOE-NE). High-level Purpose of Experiments: The Brine Availability Test in Salt (BATS) field tests are revealing both brine occurrence (i.e., where, and how much) and brine migration (i.e., how easily it moves) in the excavation damaged zone (EDZ). This understanding is foundational to develop a safety case for a future heat-generating waste repository in salt, and to starting up a generic repository program in salt to buy down risk. BATS seeks to predict how much brine can flow into both ambient and heated excavations (e.g., boreholes or rooms) in salt. This work is educating and empowering new repository scientists on two fronts: “design and execution of field tests” and “prediction and modeling of coupled processes.” DOE-NE capabilities in salt have grown and been tested through international modeling and benchmarking exercises (e.g., DECOVALEX, RANGERS, KOMPASS, and MEASURES; see Mills et al., 2024). The hands-on expertise we are building is a necessary step towards large-scale disposal demonstrations and eventual implementation.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2024

DECOVALEX-2023: Task G, SAFENET Final Report

DECOVALEX Task G deals with fracture mechanics at several scales using a combined approach of experimental work, related model development, benchmarking and experimental analysis. Fig. 0.1 provides a graphical abstract for Task G. The experimental basis for the related mechanical (M), hydro-mechanical (HM), and thermo-mechanical (TM) processes comes from the rock mechanics laboratories of Universities of Freiberg (TUBAF) and Edinburgh, as well as the Korea Institute of Civil Engineering and Building Technology (KICT). The experimental work in the rock laboratories is closely linked to the underground research laboratories (URLs) Reiche Zeche (Germany), KURT (Korea), and Mont Terri (Switzerland). The scientific key questions are related to fracture permeability evolution under THM conditions, anisotropy effects on fracturing processes, and thermal fracture slip, which are being addressed in specific steps of the task. A large variety of numerical methods have been developed and applied for experimental analysis, ranging from continuum to discontinuum approaches. A detailed comparison of mechanical and hydro-mechanical processes is given in the section 3 via the benchmarking exercises. As a result of DECOVALEX-2023 Task G we further improved our understanding and predictability of fracturing processes under THM conditions. This was based on robust numerical simulation methods and in-depth experimental analysis.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

The Fluid Dynamics Uncertainty Quantification Challenge Problem: XFOIL vs. MFOIL

Uncertainty quantification (UQ) has become more critical in aerospace engineering due to the growing dependence on computational tools for design optimization and performance analyses of aerospace vehicles. Even though the significance of UQ in assessing the credibility of computational analyses is well recognized, its costs and complexity impede its integration into standard practices, particularly in computational fluid dynamics (CFD) and other fluid analyses. This paper presents a UQ study for low-fidelity computational aerodynamics analyses with XFOIL and mfoil (i.e., the MATLAB version of XFOIL with several implementation modifications); these tools are utilized widely in both research and education. The main contributions of this paper are as follows: 1) improved precision in quantifying the uncertainty of the baseline Monte Carlo results used to benchmark surrogate modeling techniques for UQ, 2) quantification of the effect of the implementation differences between XFOIL and mfoil on solution quantities of interest (QoIs), such as lift and pitching moment coefficients, and 3) development of an open-source UQ library for use with XFOIL and mfoil, which has educational values and helps promote UQ for fluid analyses with aerospace applications. Results and discussions revolve around cases 1-4 of the challenge problem posed by the AIAA Fluid Dynamics Technical Committee’s Uncertainty Quantification Discussion Group (UQDG). In case 3, this work employs CFDverify, an open-source solution verification software, to quantify the discretization error and evaluate the extrapolated QoIs based on the grid convergence index (GCI). This UQ study differentiates itself from previous studies in the rigor of handling baseline Monte Carlo uncertainty and in including mfoil, which is a more accessible alternative to XFOIL. Finally, despite the growing computing power, low-fidelity computational tools remain valuable, such as for aerodynamic shape optimization at Mach numbers below 0.65 and low-to-mid Reynolds numbers.

Lay, Aidan S [University of Tennessee, Knoxville (↗
DOE OSTI2020

AlphaBuilding - Synthetic Buildings Operation Dataset

This is a synthetic building operation dataset which includes HVAC, lighting, miscellaneous electric loads (MELs) system operating conditions, occupant counts, environmental parameters, end-use and whole-building energy consumptions at 10-minute intervals. The data is created with 1395 annual simulations using the U.S. DOE detailed medium-sized reference office building, and 30 years' historical weather data in three typical climates including Miami, San Francisco, and Chicago. Three energy efficiency levels of the building and systems are considered. Assumptions regarding occupant movements, occupants' diverse temperature preferences, lighting, and MELs are adopted to reflect realistic building operations. A semantic building metadata schema - BRICK, is used to store the building metadata. The dataset is saved in a 1.2 TB of compressed HDF5 file. This dataset can be used in various applications, including building energy and load shape benchmarking, energy model calibration, evaluation of occupant and weather variability and their influences on building performance, algorithm development and testing for thermal and energy load prediction, model predictive control, policy development for reinforcement learning based building controls.

AlphaBuilding↗
DOE OSTI2020

Challenging a Global Land Surface Model in a Local Socio-Environmental System

Land surface models (LSMs) predict how terrestrial fluxes of carbon, water, and energy change with abiotic drivers to inform the other components of Earth system models. Here, we focus on a single human-dominated watershed in southwestern Michigan, USA. We compare multiple processes in a commonly used LSM, the Community Land Model (CLM), to observational data at the single grid cell scale. For model inputs, we show correlations (Pearson’s R) ranging from 0.46 to 0.81 for annual temperature and precipitation, but a substantial mismatch between land cover distributions and their changes over time, with CLM correctly representing total agricultural area, but assuming large areas of natural grasslands where forests grow in reality. For CLM processes (outputs), seasonal changes in leaf area index (LAI; phenology) do not track satellite estimates well, and peak LAI in CLM is nearly double the satellite record (5.1 versus 2.8). Estimates of greenness and productivity, however, are more similar between CLM and observations. Summer soil moisture tracks in timing but not magnitude. Land surface reflectance (albedo) shows significant positive correlations in the winter, but not in the summer. Looking forward, key areas for model improvement include land cover distribution estimates, phenology algorithms, summertime radiative transfer modelling, and plant stress responses.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Design of inertial fusion implosions reaching the burning plasma regime

One of the last remaining milestones in fusion research before reaching ignition is creating a burning plasma state, where alpha particles from deuterium-tritium (DT) fusion reactions redeposit their energy as the dominant source of heating in the plasma. The indirect-drive inertial confinement fusion approach at the National Ignition Facility (NIF) uses a laser-generated radiation cavity (hohlraum) to spherically implode DT fuel to high temperatures and densities in a central ”hot spot”. Here, we deliver more energy to the hot spot than ever before, while maintaining the extreme pressures required for inertial confinement, by increasing the size of the implosion compared to previous experiments. We develop more efficient hohlraums, to drive these larger implosions within NIF’s current laser energy and power capability and control symmetry by moving energy between laser beams and by changing the shape of the hohlraum. These designs resulted in record fusion powers of 1.5 petawatts, greater than the input power of the laser, and 170 kJ of fusion energy. Radiation hydrodynamics simulations show alpha particle heating as the dominant term in the hot spot energy balance, e.g. a burning plasma state. This work is expected to motivate future studies of burning plasmas and improve predictive capability by providing a benchmark for modeling used to understand the proximity to ignition.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Building the Financial Business Case for Resilience

This fact sheet is designed to help owners, operators, occupants, and investors in commercial buildings and other sectors to understand and address emerging issues in resilience. It includes resources from DOE’s Better Buildings Financing and Resilience Roadmap to identify key issues in measuring, managing, and mitigating resilience risk.

Resilience, Resiliency, Commercial building, comme↗
DOE OSTI2022

Benchmarking intra-nuclear cascade models for neutrino scattering with relativistic optical potentials

The description of final-state interactions (FSI) in the large phase space probed in neutrino experiments poses a great challenge. In neutrino experiments, which operate under semi-inclusive conditions, cascade models are commonly used for this task, while under exclusive conditions FSI can be treated with relativistic optical potentials (ROP). We formulate conditions under which the ROP approach and cascade model can be directly compared. We feed the NEUT cascade with events from a relativistic distorted-wave impulse approximation calculation that uses the real part of an optical potential. Cuts on the missing energy of the resulting events are applied to define a set of events that can be directly compared to RDWIA calculations with the full optical potential. The NEUT cascade and ROP agree for proton kinetic energies $T_p > 150$ MeV for carbon, oxygen and calcium nuclei when a realistic nuclear density is used to introduce events in the cascade. For $T_p < 100$ MeV the ROP and NEUT cross sections differ in shape and differences in magnitude are larger than 50 %. Single transverse variables allow to distinguish different approaches to FSI, but due to a large non-QE contribution the comparison to T2K data does not give an unambiguous view of FSI. We discuss electron scattering and argue that with a cut in missing energy FSI can be studied with minimal confounding factors in e.g. $e4\nu$. The agreement of the ROP and NEUT for T2K conditions lends confidence to these models as a tool in oscillation analyses for sufficiently large nucleon kinetic energies. These results urge for caution when a cascade model is applied for small nucleon energies. The assessment of model assumptions relevant to this region are strongly encouraged. This paper provides novel constraints on cascade models from proton-nucleus scattering that can be easily applied to other neutrino event generators.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Verification of Special Nuclear Material (SNM) using Active Well Coincidence Counter (AWCC) and a High Purity Germanium (HPGe) Detector

SNM verification can be accomplished via two methods i.e. destructive analysis (DA) or nondestructive analysis (NDA). The DA modes are much more accurate compared to NDA modes; nevertheless, the sample's physical integrity is destroyed in the process. Techniques such as ICP-MS, ID-ICP-MS, TI-MS are few examples of DA, where a bulk material is ground and mixed with a solution (some form of acid) to acquire a homogenous sample. In contrast, NDA modes utilize signals emitted by the sample such as gamma, neutrons, light, or heat to analyze samples under investigation. HPGe detectors are some of the most frequently used gamma detectors to assay SNM. However, in a high activity scenario it provides a large uncertainty and sometimes becomes unfeasible to measure low energy gamma rays. Meanwhile, neutron detectors are not affected by the gamma background, therefore they can be used in addition to the gamma detectors to verify total fissile mass ({sup 235}U + {sup 239}Pu) of SNM. The AWCC is one of the thermal neutron NDA systems developed at the Los Alamos National Laboratory (LANL). It consists of 42 {sup 3}He detectors to measure gross neutrons (singles), coincidence neutrons (doubles), higher order multiplicity counting (triples, quadruples, etc.). It can effectively operate in both active and passive neutron interrogation modes. It can utilize AmLi, AmBe, or {sup 252}Cf active interrogation neutron sources. It is used to verify declared fissile mass ({sup 235}U or {sup 239}Pu) present in SNM. Objectives: Calibrate HPGe and AWCC for lightly irradiated (with enough {sup 137}Cs gamma background) ∼93% enriched HEU fuels; Verify fissile mass ({sup 235}U +{sup 239}Pu) content using AWCC and HPGe; Model and benchmark AWCC in MCNP6. Neutron coincidence and multiplicity counting: Two or more neutrons are coincident if they are detected by the system within the specified gate window; In AWCC: Gate window is 128 μsec; Induced fission in fissile material such as {sup 235}U and {sup 239}Pu - emission of time correlated neutrons - Singles, Doubles, Triples, or Quadruples; No induced fission in non-fissile materials- no correlated neutrons - only singles. SNM can be verified with high accuracy and precision using gamma and neutron NDA instruments such as HPGe and AWCC. Interaction with thermal neutrons. Non-fissile atoms - do not emit correlated neutrons. Fissile atoms - emit correlated neutrons during induced fission events. The time correlated neutrons can be detected by using coincidence or multiplicity counters such as AWCC. Likewise, {sup 235}U produces 186 keV gamma peak that can be measured by a HPGe gamma detector. Counts under 186 keV gamma peak provide information about {sup 235}U content. Combining AWCC and HPGe results can provide better understanding of special nuclear material under investigation.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

IP Protection in TinyML

Tiny machine learning (TinyML) is an essential component of emerging smart microcontrollers (MCUs). How- ever, the protection of the intellectual property (IP) of the model is an increasing concern due to the lack of desktop/server-grade resources on these power-constrained devices. In this paper, we propose STML, a system and algorithm co-design to Secure IP of TinyML on MCUs with ARM TrustZone. Our design jointly optimizes memory utilization and latency while ensuring the security and accuracy of emerging models. We implemented a prototype and benchmarked with 7 models, demonstrating STML reduces 40% of model protection runtime overhead on average.

42 ENGINEERING↗
DOE OSTI2024

Measurement of the Neutron Magnetic Form Factor at Large Momentum Transfer Using the Super-Bigbite Apparatus in Jefferson Lab Hall-A

The nucleon elastic electromagnetic form factors help us study the electromagnetic structure of the nucleon, benchmark theoretical models, and improve our understanding of non-perturbative quantum chromodynamics and confinement. The Nobel Prize-winning electron-nucleon scattering experiments by Robert Hofstadter and collaborators in the 1950s at Stanford High Energy Physics Lab were the first nucleon form factor measurements performed using leptonic probes. The Super Bigbite Spectrometer (SBS) program at Hall-A of Jefferson Lab represents the latest efforts to measure nucleon form factors. This ambitious program aims to significantly extend the current data set in terms of square momentum transfer (Q2) with high precision. The advent of novel detector technologies, like Gas Electron Multipliers (GEM), which provide excellent position resolution (< 100 ?m) while withstanding high background particle rates (several hundred MHz/cm2) over a large active area, has paved the way for open-geometry, moderate solid angle spectrometers, which are central to all form factor experiments in the SBS program. The first experimental run group in SBS ran successfully between September 2021 and February 2022, collecting data for the measurement of the magnetic form factor of the neutron Gn M at five squared momentum-transfer values: 3.0, 4.5, 7.5, 9.8, and 13.5 (GeV /c)2. This extends the existing high-precision data for Gn M by about a factor of four. The ratio technique was used, which involved the simultaneous measurement of exclusive quasielastic scattering of D(e,e?n)p and D(e,e?p)n from a deuterium target. Pre-preliminary results for D(e,e?n)p and D(e,e?p)n quasi-elastic ratio, and the neutron magnetic form factor Gn M , for Q2 points 3.0, 9.8, and 13.5 (GeV /c)2 are presented.

Mudiyanselage, Anuruddha Rathnayake↗