Search NASA⌕ Search

SEARCH · Search NASA

Results for “benchmark and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Verification of Flux Sensitivity Estimates Using the MCNP Tally Perturbation Tool

Nuclear data is commonly used in applications such as nuclear nonproliferation, safeguards, and criticality safety. More specifically, nuclear data is used in predictive simulation codes like the Monte-Carlo N-Particle (MCNP ® ) transport code, Serpent, and similar radiation transport codes. The improvement of nuclear data enables more precise and accurate simulations, which result in higher fidelity designs and reduced operational/procedural costs. Therefore, the improvement of nuclear data is of paramount importance across the nuclear community. Nuclear data is improved and validated through integral benchmark experiments. The design of benchmark experiments is an extensive process; therefore, these experiments are often optimized on multiple characteristics, including sensitivity to the nuclear data, during the design process. Sensitivity is a measure of how much a quantity changes due to changes in independent variables such as experimental configuration. An experimental design that has a larger sensitivity to the nuclear data of interest will have a larger impact on the accuracy and precision of the validated data. Past integral benchmark experiments have primarily used the effective multiplication factor ($k_{eff}$) as the predominant measured quantity; however, experiments designed with other quantities in mind would be able to optimize on validating different areas of the nuclear data. A primary goal of the EUCLID project is to design, constrain, and reduce compensating errors in experiments focused on quantities other than $k_{eff}$ to better validate nuclear data across the board. Currently, there is a capability in MCNP to easily calculate the sensitivity of $k_{eff}$ to specific nuclear data of numerous reactions types and isotopes (KSEN card); however, the sensitivity of other quantities must be estimated in more strenuous manners. For example, the perturbation feature (PERT card) of MCNP can be used to estimate first-order sensitivities of some response in fixed source simulations. A recent announcement revealed that the first- and second-order perturbation features in previous releases of MCNP contained a bug. It was identified that particles were being scored into the wrong energy bin. The bug is in the most recent public release (MCNP6.2); however, a patch has been added to the most up to date version (MCNP6.2.2) that has not been released publicly. A direct comparison of the PERT card results for an F4 (neutron flux averaged over a cell) tally before and after the patch are shown in figure 1. All simulations used in the sensitivity estimates in this report were performed with MCNP6.2.2. This work verifies the patched MCNP perturbation tool by comparing first order sensitivities made using the PERT card to estimates made using manual perturbation of the compact ENDF (ACE) files.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Hardware-In-the-Loop Benchmarking Setup for Phasor Based Control Validation

Phasor Based Control is a novel approach to controlling Distributed Energy Resources that aims at relieving various constraints that arise in the distribution grid. It is a two-layer control system with a supervisory control that coordinates distributed controllers to reach voltage phasor targets. The distributed controllers use local synchrophasor measurements and operate as feedback controllers. This control method is currently under development with several algorithms being under consideration for both the central and distributed components. In this report, we present the experimental setup that was prepared to prototype a hardware implementation and validate the control method in Hardware-In-the-Loop.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Hardware-In-the-Loop Benchmarking Setup for Phasor Based Control Validation

Phasor Based Control is a novel approach to controlling Distributed Energy Resources that aims at relieving various constraints that arise in the distribution grid. It is a two-layer control system with a supervisory control that coordinates distributed controllers to reach voltage phasor targets. The distributed controllers use local synchrophasor measurements and operate as feedback controllers. This control method is currently under development with several algorithms being under consideration for both the central and ditributed components. In this paper, we present the experimental setup that was prepared to prototype a hardware implementation and validate the control method in Hardware-In-the-Loop.

Baudette, Maxime↗
DOE OSTI2021

Propagating neutronic uncertainties for FFTF LOFWOS Test #13

The safety evaluation conducted for licensing a Sodium-cooled Fast Reactor (SFR) may require modeling transients with best-estimate calculation tools that must first be validated against real-world measurements. To provide the community with a valuable benchmarking opportunity for validating SFR analysis tools and methods, the International Atomic Energy Agency (IAEA) initiated a coordinated research project (CRP) in 2018 for the analysis of the Fast Flux Test Facility (FFTF) Loss of Flow Without Scram (LOFWOS) Test #13. The impact of nuclear data uncertainties on neutronics parameters was previously investigated based on the COMMARA-2.0 covariance matrix. Since the transient simulation results are very sensitive to certain reactivity coefficients, it was decided to employ rigorous uncertainty propagation methods to quantify the impact of nuclear data uncertainties on the best-estimate predication of FFTF LOFWOS Test #13. The DAKOTA code is used to propagate neutronic uncertainties through SAS4A/SASSYS-1 transient simulations, while taking into account spatial and reaction-wise correlations within these uncertainties. This study shows that the remaining discrepancies observed between the Argonne National Laboratory (ANL) best-estimate results and the experimental measurements can be partly explained by the uncertainty associated with Gas Expansion Module (GEM) worth, which contributes the majority of the overall nuclear data uncertainty on the output from the FFTF LOFWOS Test #13 transient simulation. This study also confirmed the importance of including spatial and reaction-wise correlations of nuclear data uncertainties on feedback coefficients in the uncertainty propagation to avoid under-estimating their impact during the transient simulations.

42 ENGINEERING↗
DOE OSTI2026

Systematic Benchmarking of Climate Models: Methodologies, Applications, and New Directions

As climate models become increasingly complex, there is a growing need to comprehensively and systematically assess model performance with respect to observations. Given the increasing number and diversity of climate model simulations in use, the community has moved beyond simple model intercomparison and toward developing methods capable of benchmarking a large number of simulations against a suite of climate metrics. Here, we present a detailed review of evaluation and benchmarking methods and approaches developed in the last decade, focusing primarily on scientific implications for Coupled Model Intercomparison Project (CMIP) simulations and CMIP6 results that contributed to the Intergovernmental Panel on Climate Change (IPCC) Sixth Assessment Report (AR6). Based on this review, we explain the resulting contemporary philosophy of model benchmarking, and provide clear distinctions and definitions of the terms model verification, process validation, evaluation, and benchmarking. While significant progress has been made in model development based on systematic evaluation and benchmarking efforts, some climate system biases still remain. The development of open‐source community software packages has played a fundamental role in identifying areas of significant model improvement and bias reduction. We review the key features of several software packages that have been commonly used over the past decade to evaluate and benchmark global and regional climate models. Additionally, we discuss best practices for the selection of evaluation and benchmarking metrics and for interpreting the obtained results, the importance of selecting suitable sources of reference data and accurate uncertainty quantification.

Environmental sciences↗
DOE OSTI2022

Estimating List-Mode Data Sensitivities to Nuclear Data with MCNP6

Nuclear data are a vital component of predictive simulations used in applications like experiment design, stockpile stewardship, nuclear nonproliferation/safeguards, health physics, and criticality safety. A singular simulation requires the coalescence of different areas of nuclear data such as cross sections, angular distributions, and energy distributions of emitted neutrons for different materials and energy ranges. Improving nuclear data and thus reducing the uncertainty in simulated parameters could enable smaller, better-informed safety factors and ultimately reduce operational and procedural costs. There is a constant effort to garner a better understanding of the physical quantities represented by nuclear data through experiments. Integral experiment benchmarks use simulated and measured results to validate current nuclear data values. In the past, benchmarks primarily focused on the effective multiplication factor (k eff ); however, this limited scope has caused compensating errors and areas of nuclear data that lack validation. Compensating errors are inaccuracies in nuclear data that are obfuscated by cancellation when observing integrated values such as k eff . Diverse integral benchmark experiments that look for quantities of interest other than k eff and include multiple responses minimize the possibility of compensating errors and provides validation to areas of nuclear data previously lacking experimental validation. Benchmark experiments can be optimized during the design process to be highly dependent on specific areas of nuclear data. The dependence of a response in an experiment to a specific area/type of nuclear data is defined as sensitivity. A larger sensitivity means that nuclear data uncertainties will play a larger role in the response(s) resulting in larger bias. Currently, the sensitivity capabilities of the Monte Carlo N-Particle (MCNP ®1 ) transport code are limited to responses of k eff and tallied values (e.g., flux, surface current). As a part of the EUCLID project, this work explores estimating list-mode nuclear data sensitivities that can be used to design experiments aimed to constrain and reduce compensating errors in nuclear data by focusing on responses other than k eff . Tallied values are ideal quantities that are estimated with detectors during experiments. List-mode data (a list of neutron collection times) are the direct output of detector systems in subcritical neutron noise experiments. Expanding MCNP sensitivity capabilities to include the sensitivity of responses estimated from list-mode data, such as the prompt neutron decay constant (α) and multiplicity estimates (S and D), enables more direct comparison of simulated and measured experimental quantities. Additionally, deterministic tools such as SENSMG are capable of obtaining sensitivities to a wide variety of responses; however, these tools cannot handle complex geometries due to the assumptions made in discretizing the phase-space variables of the Boltzman transport equation.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Availability of Shielding Benchmark Experiment Data in the ICSBEP Handbook

The International Criticality Safety Benchmark Evaluation Projects (ICSBEP) has over two decades of experience under the auspices of the Organisation for Economic Co-operation and Development (OECD) Nuclear Energy Agency (NEA) to identify, evaluate, and compile a comprehensive data set of criticality safety benchmark data [1]. The activities of the ICSBEP have led to the continued success of the International Handbook of Evaluated Criticality Safety Benchmark Experiments (ICSBEP Handbook) [2]. The contents of the ICSBEP Handbook include benchmark specifications derived from nuclear facilities around the world, which have been utilized to support criticality safety efforts and also integral testing of nuclear data [3]. The 2020 edition of the ICSBEP Handbook will include data from 582 evaluations containing benchmark specifications for 5,053 critical, subcritical, or near-critical configurations. There are seven criticality-alarm-placement/shielding evaluations containing a total of 45 benchmark configurations, and ten fundamental physics evaluations containing a total of 237 measurements relevant to criticality safety applications. The purpose of this paper is to summarize those benchmarks relative to validation needs in shielding applications currently found within the ICSBEP Handbook. The current and future plans of shielding benchmarks with regards to the ICSBEP will also be discussed.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Development, validation, and verification of multi-pass thermo-mechanical welding simulations using the open-source MOOSE framework: NeT TG4 benchmark weldment

This study develops and validates a sequentially coupled thermo-mechanical welding simulation for the three-pass 316L stainless steel NeT TG4 benchmark weldment using the open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) and the Nuclear Engineering Material model Library (NEML). A diffused ellipsoidal heat source was calibrated against thermocouple data and weld macrographs to accurately model the fusion zone geometry and transient thermal fields. Material hardening is represented using the Lemaitre-Chaboche mixed isotropic-kinematic hardening model, while four annealing models - no annealing, single-stage at 1050 °C and 1300 °C, and two-stage at 800 °C/1300 °C - were implemented to assess the impact of annealing models on the accuracy of the predicted welding-induced plasticity, distortions, and residual stresses. The predictions were validated against experimental measurements and benchmarked against results from commercial software, demonstrating that thermo-mechanical MOOSE welding simulations achieve comparable accuracy with enhanced computational efficiency. This work highlights the potential of using open-source finite element frameworks like MOOSE for advanced manufacturing simulations.

Ji, Wendy [Australian Nuclear Science and Technolo↗
DOE OSTI2022

Correlation Calculations for the Russian Pu Metal Fast Experiments

Nuclear criticality experiments are often conducted in campaigns with multiple variations. These experiments reuse the same basic components, like the fuel, moderator, or positioning machines. The components have uncertainties in their geometry and composition that propagate to models of the experiments. Shared components create shared uncertainty between the $k_{eff}$ of benchmarks. The shared uncertainty is commonly quantified with a covariance, or correlation coefficient. These covariances can impact criticality safety and nuclear data validation applications. While benchmark evaluations tabulate an experiment’s uncertainty, they often lack a detailed calculation of correlations between experiments. Even some very commonly used benchmarks, like the Russian Pu Metal Fast (PMF) experiments, have missing correlations. This paper presents our approach to calculate the correlations for five of the Russian PMF experiments. The experiments share hemispherical Pu shells that induce a correlation between modeled $k_{eff}$ values. We estimated the correlations with simplified and detailed models of the experiments through linear-perturbation theory. The correlations between the experiments vary significantly between the detailed vs. simplified models. We also investigate how the correlations affect validation metrics of the experiments.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

On the novel 3-D neutron transport kinetic tRAPID algorithm and its validation

The Real-time Analysis for Particle-transport and In-situ Detection (RAPID) Code System, based on the Multi-stage Response-function Transport (MRT) methodology, allows for real-time simulation of nuclear systems based on 3-D continuous-energy particle transport. RAPID's steady-state (criticality) neutron transport algorithm is based on the Fission Matrix (FM) method, and has been extensively verified and validated against computational benchmarks and experiments. This paper introduces the novel 3-D time-dependent transport algorithm that has been implemented into the code, tRAPID, and its validation using the JSI TRIGA Mark-II reactor. tRAPID accurately and efficiently calculates neutron kinetics parameters (such as β{sub eff}, l{sub eff} , Λ, α{sub Rossi}) and 3-D time-dependent neutron fission source distribution and neutron importances for both prompt and delayed neutrons. tRAPID is used to simulate a rod insertion experiment performed at the JSI TRIGA Mark-II reactor, during which signals from four fission chambers at four different locations in the core were collected. The results demonstrate how tRAPID is capable of calculating detailed and accurate results with only a minimal use computational resources and time.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Verification of the AGREE and Serpent for the steady state HTR-10 benchmark problems

This paper provides results from steady state HTR-10 benchmark calculations performed using AGREE and Serpent that are compared to experimental results as well as calculations performed by INET. The purpose of completing this benchmark is to validate AGREE and Serpent for the prediction of HTGR operation so that they may ultimately be used to support licensing and deployment efforts of advanced reactors. The benchmark consists of several problems ranging from control rod worth calculations to k{sub eff} calculations at various temperatures. Overall, AGREE and Serpent show good agreement with the reference solutions and are effectively able to predict HTGR operation for a variety of steady state cases. The largest difference from the reference was for the initial core single rod worth, which is possibly due to the larger core helium cavity causing inaccuracies in the diffusion calculation whereas the control rod worth determined by Serpent and AGREE is much closer to the reference result for the full core loading. Moreover, in every benchmark problem, increasing the number of energy groups in the cross sections results in improved agreement of the AGREE result with the reference, although it also causes an increase in computation time. For problems with experimental data available, the accuracy of results generated by Serpent and AGREE is comparable to results obtained by other benchmark participants. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS2012

IMAGESEER - IMAGEs for Education and Research

IMAGESEER is a new Web portal that brings easy access to NASA image data for non-NASA researchers, educators, and students. The IMAGESEER Web site and database are specifically designed to be utilized by the university community, to enable teaching image processing (IP) techniques on NASA data, as well as to provide reference benchmark data to validate new IP algorithms. Along with the data and a Web user interface front-end, basic knowledge of the application domains, benchmark information, and specific NASA IP challenges (or case studies) are provided.

Le Moigne, Jacqueline↗
NASA NTRS2003

Computational Fluid Dynamics Technology for Hypersonic Applications

Several current challenges in computational fluid dynamics and aerothermodynamics for hypersonic vehicle applications are discussed. Example simulations are presented from code validation and code benchmarking efforts to illustrate capabilities and limitations. Opportunities to advance the state-of-art in algorithms, grid generation and adaptation, and code validation are identified. Highlights of diverse efforts to address these challenges are then discussed. One such effort to re-engineer and synthesize the existing analysis capability in LAURA, VULCAN, and FUN3D will provide context for these discussions. The critical (and evolving) role of agile software engineering practice in the capability enhancement process is also noted.

Gnoffo, Peter A.↗
DOE OSTI2021

Neutronic Characteristics of ENDF/B-VIII.0 Compared to ENDF/B-VII.1 for Light-Water Reactor Analysis

The Evaluated Nuclear Data File (ENDF)/B-VIII.0 data library was released in 2018. To assess the new data during development and shortly after release, many validation calculations were performed with static, room-temperature benchmarks. Recently, when performing validation of ENDF/B-VIII.0 for pressurized water reactor depletion calculations, a regression in performance compared to ENDF/B-VII.1 was observed. This paper documents extensive benchmark calculations for light-water reactors performed using continuous energy Monte Carlo code with ENDF/B-VII.1 and -VIII.0 and neutronic characteristics of ENDF/B-VIII.0 are discussed and compared to those of ENDF/B-VII.1. It is our recommendation that ENDF/B data library assessment should include reactor-specific benchmark assessments, including depletion cases, such that these types of regressions may be caught earlier in the data development cycle.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Uncertainty Quantification of a Light Water Pulsed-Neutron Die-Away Experiment to Thermal Neutron Scattering Laws

Thermal neutron scattering laws are important nuclear data for many nuclear science and engineering applications. Validation helps to ensure that a thermal neutron scattering law has a high quality and often employs critical benchmarks as integral experiments. Recently, pulsed-neutron die-away benchmarks have been used as an experiment to validate thermal neutron scattering laws. Herein, we evidence how this alternative integral experiment has a high sensitivity to these nuclear data by performing an uncertainty quantification analysis. The analysis randomly sampled the nuclear model parameters associated with hydrogen bound in light water thermal neutron scattering law and sampled other nuclear data that influenced the experiment’s integral parameter (e.g., elastic scattering, absorption in hydrogen and oxygen) from their respective covariance matrices. The thermal neutron scattering law caused an uncertainty in the integral parameter that reached 2.67%, which exceeds by an order of magnitude the uncertainties induced in commonly used thermal solution critical benchmarks. The validation performed here, although limited due to a poor description of the historical experiment, indicated that the ENDF/B-VIII.0 thermal neutron scattering law well predicted the integral parameter. These results motivate further benchmark and validation efforts using pulsed-neutron die-away experiments.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Progress in the HTTF Benchmark and RELAP5-3D Gas-Cooled Reactor Validation

This slide set presents results of analysis done in the first year after the kickoff of the HTTF benchmark. This work includes the development of a new RELAP5-3D model of the facility and collection and presentation of results from one of the exercises in the benchmark. Highlights include good agreement between the new and legacy models for full-power steady state and similar predictions of maximum block temperature during a pressurized conduction cooldown. Additionally, code-to-code comparisons of a full-power steady state and depressurized conduction cooldown (DCC) from full-power steady state as part of Problem 2 Exercises 1A and 1B show good agreement on temperatures predicted in the core but considerable spread in central reflector temperatures. During the DCC, temperature behavior is similar across all models, but cooldown rates are dictated by the performance of the reactor cavity cooling system

22 GENERAL STUDIES OF NUCLEAR REACTORS↗