Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Short ensembles: An Efficient Method for Discerning Climate-relevant Sensitivities in Atmospheric General Circulation Models

This paper explores the feasibility of an experimentation strategy for investigating sensitivities in fast components of atmospheric general circulation models. The basic idea is to replace the traditional serial-in-time long-term climate integrations by representative ensembles of shorter simulations. The key advantage of the proposed method lies in its efficiency: since fewer days of simulation are needed, the computational cost is less, and because individual realizations are independent and can be integrated simultaneously, the new dimension of parallelism can dramatically reduce the turnaround time in benchmark tests, sensitivities studies, and model tuning exercises. The strategy is not appropriate for exploring sensitivity of all model features, but it is very effective in many situations. Two examples are presented using the Community Atmosphere Model version 5. The first example demonstrates that the method is capable of characterizing the model cloud and precipitation sensitivity to time step length. A nudging technique is also applied to an additional set of simulations to help understand the contribution of physics-dynamics interaction to the detected time step sensitivity. In the second example, multiple empirical parameters related to cloud microphysics and aerosol lifecycle are perturbed simultaneously in order to explore which parameters have the largest impact on the simulated global mean top-of-atmosphere radiation balance. Results show that in both examples, short ensembles are able to correctly reproduce the main signals of model sensitivities revealed by traditional long-term climate simulations for fast processes in the climate system. The efficiency of the ensemble method makes it particularly useful for the development of high-resolution, costly and complex climate models.

atmospheric general circulation models↗

Baseline Emissions Measurements from Reciprocating Compressor Packing

A novel liquid reciprocating compressor packing seal concept is being developed to reduce methane emissions from reciprocating compressor packing by 95%. To provide the leakage basis for this project, a benchmark test was performed to measure the leakage from a 50Hp Ariel reciprocating compressor that will be used for the testing comparisons throughout the project. Brand-new packing replacement parts were installed to minimize leakage that could be due to worn internals and rings

03 NATURAL GAS↗

High-order diamond differencing schemes for the Boltzmann Fokker-Planck equation in 3D Cartesian geometries

The Boltzmann Fokker-Planck, an approximate form of the linear Boltzmann equation is commonly used to treat efficiently the transport of charged particles in matter. This paper introduces the application of high-order diamond differencing schemes (HODD), specifically the DD1 and DD2 schemes which are 4- and 6-order accurate respectively, to handle the spatial discretization of that equation in 3D Cartesian geometries. The energy deposition solutions for the coupled transport of electrons and photons presented in this work shows that HODD, compared to classical DD scheme, provides correction to the oscillations and a reduced propensity to yield negative fluxes. They are useful tools to minimize local error, notably in regions with abrupt variations of the flux solution. They also can be used to reduced execution time by decreasing the needed number of voxels to obtain a fixed accuracy. On the tested benchmarks, the DD1 scheme is 87%- 92%-91% more accurate than the classical DD scheme for total, mean per-voxels and maximum deviation of energy deposition values respectively. For comparison, a calculation with 8 times more voxels, requiring roughly 2.5 times more time to execute, is 92%-90%-77% more accurate. (authors)

97 MATHEMATICS AND COMPUTING↗

Benchmarking FFTF LOFWOS Test# 13 using SAM code: Baseline model development and uncertainty quantification

The development and deployment of advanced reactors, such as the sodium-cooled fast reactor (SFR), relies on sophisticated modeling tools to ensure the safety of the design under various transients. The predictive capability of these advanced modeling tools requires validation to garner trust in supporting the licensing of the advanced reactors. For this reason, the International Atomic Energy Agency (IAEA) initiated a coordinated research project (CRP) in 2018 for the analysis of the Fast Flux Test Facility (FFTF) Loss of Flow Without Scram (LOFWOS) Test #13.In this study, we present and discuss the benchmarking efforts of the modern system code SAM on the FFTF LOFWOS Test #13. Further, the SAM baseline model was developed according to the benchmark specification, which included a detailed core model with reactivity feedback. Generally, good agreement was observed between the baseline results and benchmark measurements; however, discrepancies persisted, particularly in predicted fuel assembly coolant outlet temperatures. Utilizing the baseline model, uncertainty quantification (UQ) and sensitivity analysis (SA) were conducted with the assistance of various statistical learning and machine learning methods, including kernel density estimation, Gaussian processes, and Sobol indices. Following the baseline model prediction and UQ and SA results, we discuss the reasons for the simulation discrepancies and propose further improvements to the model. This benchmarking effort adheres to the best-estimate plus uncertainty approach and can serve as a valuable example for supporting risk-informed licensing of advanced reactors.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Benchmark Evaluation of Start-Up and Zero-Power Measurements at the High-Temperature Engineering Test Reactor

Benchmark models were developed to evaluate six cold-critical and two warm-critical, zero-power measurements of the HTTR. Additional measurements of a fully-loaded subcritical configuration, core excess reactivity, shutdown margins, six isothermal temperature coefficients, and axial reaction-rate distributions were also evaluated as acceptable benchmark experiments. Insufficient information is publicly available to develop finely-detailed models of the HTTR as much of the design information is still proprietary. However, the uncertainties in the benchmark models are judged to be of sufficient magnitude to encompass any biases and bias uncertainties incurred through the simplification process used to develop the benchmark models. Dominant uncertainties in the experimental keff for all core configurations come from uncertainties in the impurity content of the various graphite blocks that comprise the HTTR. Monte Carlo calculations of keff are between approximately 0.9 % and 2.7 % greater than the benchmark values. Reevaluation of the HTTR models as additional information becomes available could improve the quality of this benchmark and possibly reduce the computational biases. High-quality characterization of graphite impurities would significantly improve the quality of the HTTR benchmark assessment. Simulation of the other reactor physics measurements are in good agreement with the benchmark experiment values. The complete benchmark evaluation details are available in the 2014 edition of the International Handbook of Evaluated Reactor Physics Benchmark Experiments.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

IAEA CRP on HTGR Uncertainty Analysis: Benchmark Definition and Test Cases

Uncertainty and sensitivity studies are essential elements of the reactor simulation code verification and validation process. Although several international uncertainty quantification activities have been launched in recent years in the LWR, BWR and VVER domains (e.g. the OECD/NEA BEMUSE program, from which the current OECD/NEA LWR Uncertainty Analysis in Modelling (UAM) benchmark effort was derived), the systematic propagation of uncertainties in cross-section, manufacturing and model parameters for High Temperature Reactor (HTGR) designs has not been attempted yet. This paper summarises the scope, objectives and exercise definitions of the IAEA Coordinated Research Project (CRP) on HTGR UAM. Note that no results will be included here, as the HTGR UAM benchmark was only launched formally in April 2012, and the specification is currently still under development.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Benchmark Specification for FFTF LOFWOS Test #13

The Fast Flux Test Facility (FFTF) at the Hanford site in Washington was designed by the Westinghouse Electric Corporation for the U.S. Department of Energy. FFTF was a 400 MW thermal, oxide-fueled, liquid sodium cooled test reactor, built to assist development and testing of advanced fuels and materials for fast breeder reactors. After reaching criticality in 1980, FFTF operated until 1992, providing the U.S. Department of Energy (DOE) with the means to test fuels, materials, and other components in a fast neutron flux environment. In July 1986, a series of unprotected transients (with the plant protection system intentionally disabled) were performed in FFTF as part of the passive safety demonstration program. Among these were thirteen loss of flow without scram (LOFWOS) tests. The goals of this program included confirming the liquid metal reactor safety margins, providing data for computer code validation, and demonstrating the inherent and passive safety benefits of specific design features. The test defined in this benchmark is LOFWOS Test #13, which was initiated at 50% power and 100% flow with the pump pony motors turned off. This benchmark specification is intended to support collaborative efforts within international partnerships on the validation of simulation tools and models in the area of Sodium-cooled Fast Reactor (SFR) safety. Validated tools and models are needed to evaluate SFR inherent safety characteristics and assess the impact of passive design features in response to accident initiators. Comparisons with experimental data and the results of safety analyses from other groups create unique opportunities to improve predictive capabilities of computational codes and methods for SFR modeling and simulation. The conditions of the LOFWOS test along with the feedback from FFTF’s limited free bow core restraint system and the novel passive reactivity control Gas-Expansion Modules (GEMs) pose a very challenging and uniquely valuable benchmark exercise.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Identification of Integral Benchmarks for Nuclear Data Testing Using DICE (Database for the International Handbook of Evaluated Criticality Safety Benchmark Experiments)

Typical users of the International Criticality Safety Evaluation Project (ICSBEP) Handbook have specific criteria to which they desire to find matching experiments. Depending on the application, those criteria may consist of any combination of physical or chemical characteristics and/or various neutronic parameters. The ICSBEP Handbook contains a structured format helping the user narrow the search for experiments of interest. However, with nearly 4300 different experimental configurations and the ever increasing addition of experimental data, the necessity to perform multiple criteria searches have rendered these features insufficient. As a result, a relational database was created with information extracted from the ICSBEP Handbook. A users’ interface was designed by OECD and DOE to allow the interrogation of this database. The database and the corresponding users’ interface are referred to as DICE. DICE currently offers the capability to perform multiple criteria searches that go beyond simple fuel, physical form and spectra and includes expanded general information, fuel form, moderator/coolant, neutron-absorbing material, cladding, reflector, separator, geometry, benchmark results, spectra, and neutron balance parameters. DICE also includes the capability to display graphical representations of neutron spectra, detailed neutron balance, sensitivity coefficients for capture, fission, elastic scattering, inelastic scattering, nu-bar and mu-bar, as well as several other features.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

A proposed benchmark for simulation in radiographic testing

The purpose of this benchmark study is to compare simulation results predicted by various models of radiographic testing, in particular those that are capable of separately predicting primary and scatter radiation for specimens of arbitrary geometry.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Derivation and experimental test of fidelity benchmarks for remote preparation of arbitrary qubit states

Remote state preparation (RSP) is the act of preparing a quantum state at a remote location without actually transmitting the state itself. Using at most two classical bits and a single shared maximally entangled state, one can in theory remotely prepare any qubit state with certainty and with perfect fidelity. However, in any experimental implementation the average fidelity between the target and output states cannot be perfect. In order for an RSP experiment to demonstrate genuine quantum advantages, it must surpass the optimal threshold of a comparable classical protocol. Here we study the fidelity achievable by RSP protocols lacking shared entanglement and determine the optimal value for the average fidelity in several different cases. We implement an experimental scheme for deterministic remote preparation of arbitrary photon polarization qubits, preparing 178 different pure and mixed qubit states with an average fidelity of 0.995. Our experimentally achieved average fidelities surpass our derived classical thresholds whenever the classical protocol does not trivially allow for perfect RSP.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A High-Fidelity Cyber-Physical Testbed-Based Benchmarking Dataset For Testing Operational Technology Specific Intrusion Detection Systems

Quality datasets serve a critical purpose in cyber security research. Data is needed to understand system behavior and develop security controls to protect critical systems. However, for critical infrastructure operational environments there is a lack of available datasets to study because of the high cost and specialized capabilities necessary to generate them. This paper documents the development of a dataset of high fidelity hardware in the loop laboratory simulated models of electric and natural gas distribution systems with real cyber attack test cases. A deep dive discussion for the experimental setup and controls for generating the data is provided along with observations from using the data in evaluating intrusion detection approaches.

Ashok, Aditya↗

Comparative analysis of energy deposition modes available in Serpent 2 within the framework of the supercritical water reactor - Fuel qualification test reactor physics benchmark

A joint European Canadian Chinese development of a supercritical water-cooled small modular reactor (SCW-SMR) technology is in progress since September 2020 in the framework of a Horizon 2020 project called ECC-SMART. As a main purpose of the project, proper estimates of energy deposition and its spatial distribution are prerequisites for the accurate analysis of safety related parameters of the SCW-SMR concept under development. A supercritical water reactor fuel computational benchmark model, provided by Canadian Nuclear Laboratories, was applied for detailed comparison of different energy deposition calculation options available in the Serpent 2 Monte Carlo code. The effect of energy deposition options on the normalization of the results as well as on the spatial distribution of the energy deposition are discussed. Consistent energy deposition calculation methods are presented between three Monte Carlo codes, viz., Serpent 2, MCNP6 and OpenMC. Although resource-intensive, the use of the coupled neutron-photon transport mode of Serpent 2 is recommended for accurate spatial and quantitative characterization of energy deposition in the SCW-SMR fuel assemblies, accounting for both neutron and photon heating of all the materials. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Design, Manufacture, and Testing of an Open-Source Benchmark Composite Hydrokinetic Turbine Blade: Preprint

In a trend toward clean energy alternatives, recent years have seen great strides in the marine energy space. Consequently, there is a pressing need for the design, development, and validation of novel energy harvesting technologies such as hydrokinetic devices, which capture kinetic energy from waves, tides, and currents. However, these devices span numerous concepts and designs that often lack solid benchmark research that can be freely referenced throughout their development. This work focuses on the design process of an open-source composite hydrokinetic turbine blade for a three-bladed marine turbine rotor assembly with a diameter of 2.5 m. The proposed blade consists of two structural composite skins that are bonded with an adhesive and filled with a foam core. This study also explores and contrasts the efficiency and resolution of low-fidelity rapid design methodologies and comprehensive high-fidelity approaches in the context of blade design, modeling, and analysis efforts, a key objective in this research. Blade hydrodynamic loads were modeled and applied to finite-element blade models to study deformations and potential failure. Ongoing and upcoming efforts will result in blade manufacture and structural testing at the National Renewable Energy Laboratory. In future work, multiple blades will be deployed at the Living Bridge site at the University of New Hampshire and will be compared to rigid aluminum blades of the same geometry, developed by Sandia National Laboratories. Ultimately, this research will lay foundational groundwork for researchers and manufacturers, establishing a baseline composite blade design that will serve as a benchmark in the development of future hydrokinetic turbine blades.

blade design↗

Data Testing CIELO Evaluations with ICSBEP Benchmarks

We review criticality data testing performed at Los Alamos with a combination of ENDF/B-VII.1 + potential CIELO nuclear data evaluations. The goal of CIELO is to develop updated, best available evaluated nuclear data files for 1 H, 16 O, 56 Fe, 235,238 U and 239 Pu. because the major international evaluated nuclear data libraries don’t agree on the internal cross section details of these most important nuclides.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Initial Benchmarks of UV LEDs and Comparisons with White LEDs

The primary goal of this report is to benchmark the initial level of performance of a selection of commercial UV LEDs across all three bands (i.e., UV-A, UV-B, UV-C). To provide the initial performance benchmarks, a test matrix containing 13 different UV LED products was created in association with the LED Systems Reliability Consortium (LSRC). The products in this test matrix were all commercially available as of June 2021, and at least 22 samples of each product were tested. In addition, two common, commercial white LEDs were tested to provide a benchmark against blue-pumped white LEDs. Testing of the samples included electrical performance testing (e.g., current-voltage measurements) and photometric testing in a calibrated integrating sphere capable of measuring devices in the UV-A, UV-B, and UV-C bands. The electrical testing provided insights into the performance of the semiconductor layers in the LEDs, allowing parameters such as the threshold voltage (V th ) and serial resistance (R serial ) of each sample to be determined. The photometric testing provided insights into emission wavelengths, peak shapes, and radiant efficiencies for each sample. Combined, the information from these tests permits the overall device efficiencies to be compared and provides insights into the electrical and optical performance of the technology.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Benchmarking materials property prediction methods: the Matbench test set and Automatminer reference algorithm

Abstract We present a benchmark test suite and an automated machine learning procedure for evaluating supervised machine learning (ML) models for predicting properties of inorganic bulk materials. The test suite, Matbench, is a set of 13 ML tasks that range in size from 312 to 132k samples and contain data from 10 density functional theory-derived and experimental sources. Tasks include predicting optical, thermal, electronic, thermodynamic, tensile, and elastic properties given a material’s composition and/or crystal structure. The reference algorithm, Automatminer, is a highly-extensible, fully automated ML pipeline for predicting materials properties from materials primitives (such as composition and crystal structure) without user intervention or hyperparameter tuning. We test Automatminer on the Matbench test suite and compare its predictive power with state-of-the-art crystal graph neural networks and a traditional descriptor-based Random Forest model. We find Automatminer achieves the best performance on 8 of 13 tasks in the benchmark. We also show our test suite is capable of exposing predictive advantages of each algorithm—namely, that crystal graph methods appear to outperform traditional machine learning methods given ~10 4 or greater data points. We encourage evaluating materials ML algorithms on the Matbench benchmark and comparing them against the latest version of Automatminer.

36 MATERIALS SCIENCE↗