Search NASA⌕ Search

SEARCH · Search NASA

Results for “benchmark and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Advanced modeling and simulation of research reactors using dynamic mode decomposition

Full text of publication follows. Due to the ever-increasing safety requirements, the current trend of nuclear reactor analysis is shifting towards high-fidelity multi-physics models, which have a very high computational cost and modelling complexity. As the cost of even a single model run makes it impossible to analyse the behaviour and performance of these models on large-scale commercial plants, it has become even more significant to provide suitable benchmarks to validate and test them extensively. In this sense, research reactors offer a promising solution for the initial validation of high-fidelity models, as they are significantly smaller than commercial reactors and their characteristics are well known. In particular, the reactors of the TRIGA family have been used to assess and validate models and methods for Generation-IV designs, as they have some similar features (such as the dominance of natural convection as cooling mechanism and the difficulties in performing sub-channel analysis using standard codes). Still, the computational requirements of high-fidelity models make them unsuitable for real-time analysis, even following their assessment on research reactors. In this sense, Model Order Reduction (MOR) techniques give an additional strategy to reduce the computational cost of high-fidelity models (whilst preserving sufficient accuracy). In particular, this work focuses on Dynamic Mode Decomposition (DMD), a non-intrusive MOR technique that aims at representing models with explicit temporal dynamics by extracting the time-varying characteristics and the governing structures based only on a set of available data, thus without needing any underlying knowledge of the governing equations. In addition, DMD also computes a low-dimensional surrogate of the dynamic matrix of the system, making it suited for stability analysis and real-time evaluations. This work focuses on the application and validation of the DMD method on the Computational Fluid-Dynamics (CFD) model TRIGA Mark II reactor, also discussing in detail the potentiality of this algorithm as an advanced modelling tool for nuclear reactor analysis. (author)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Instrumentation for the In-Core Real-Time Mechanical Testing of Structural Materials (INCREASE) Project

Idaho National Laboratory (INL), in collaboration with the Electric Power Research Institute (EPRI), the Nuclear Regulatory Commission (NRC), the French Atomic and Alternative Energies Commission (CEA), the Joint Research Center (JRC), the Nuclear Research and Consultancy Group (NRG), and the Research Center Rez (CVR), started a Joint Experimental Program (JEEP) project that operates within the Nuclear Energy Agency’s Framework for Irradiation Experiments (FIDES II) program in order to develop capabilities for the in-core real-time mechanical testing of structural materials. This effort will focus on designing a shared capsule capable of housing a variety of in-core mechanical testing instrumentation allowing enhanced experiments for the material science community. The outcome of the project would be high-priority, stress relaxation data for stainless-steel-based materials provided by EPRI and CEA. Stress relaxation is a major phenomenon that contributes to material degradation in nuclear reactor components. Currently, nuclear material stress relaxation is assessed both before and after irradiation, using complex and costly post-irradiation examination (PIE) activities. In-situ data would support the development of precision modeling and simulation of this degradation phenomena and would provide validation and benchmarking for existing models using the PIE data. As part of the U.S. Department of Energy Advanced Sensor and Instrumentation (ASI) program, INL has fabricated and tested out-of-core mechanical test instrumentation. This instrumentation was designed for easy adaptation to the irradiation capsule proposed under this JEEP, and can be deployed to measure real-time stress relaxation under pressurized-water reactor (PWR) conditions. This initial effort will partially serve to replace the testing capabilities lost because of shutting down the Halden Boiling Water Reactor (HBWR). The project would provide these capabilities to the international community via a shared capsule design that is easily adaptable to additional material test reactors. The design features will incorporate expansion to PWR and non-light-water reactor (LWR) environments that will be developed in future work. The capsule and instrumentation will be demonstrated in the Massachusetts Institute of Technology Reactor (MITR) for phase I and the Petten High Flux Reactor (HFR) for phase II irradiations to deliver real-time stress relaxation data on high priority stainless steel structural materials.

36 MATERIALS SCIENCE↗

High-Temperature Gas-Cooled Reactors Multiphysics Simulation Demonstration and Code Validation

This study presents a comprehensive benchmarking and verification effort of several thermal-hydraulic and multiphysics capabilities for high-temperature gas-cooled reactor applications. The first part of this effort focuses on the running-in verification of Griffin’s multiphysics capabilities, specifically for simulating the evolution of pebble-bed reactor cores from startup to equilibrium. Since Fiscal Year 2024, improvements and enhancements have been implemented in Griffin, including simplifying the process to specify streamlines and developing the online cross-section generation capability. In the absence of validation data, code-to-code comparisons are conducted with kugelpy, showing good agreement for integral quantities like k-eff predictions and predictions for maximum power density. However, accuracy issues are noted for more detailed quantities like the spatial distribution of fission rate densities which will require further work to address. The second part of this report presents an improved System Analysis Module (SAM) core channel model where the effects of cross flow are considered during the pressurized loss of forced cooling transient, resulting in an improved agreement of the predicted pebble temperature with respect to the predictions from the SAM 2D porous media model. Additionally, the wall channeling effect due to variable porosity at the near wall region of the core is also investigated. Furthermore, to demonstrate Griffin’s online cross-section generation capability, a Multiphysics simulation is performed by coupling Griffin to the SAM core channel model. In the third part of the report, as a part of the Organisation for Economic Co-operation and Development/Nuclear Energy Agency (OECD/NEA) thermal-hydraulic code validation benchmark activity for a high-temperature gas-cooled reactor, the High Temperature Test Facility (HTTF) is investigated first using the NekRS computational fluid dynamics (CFD) code to study the flow mixing phenomenon in the lower plenum of the facility. Then, code-to-code and code-to-data comparisons are performed for Test PG27, which is a pressurized conduction cooldown (PCC) test, using five different codes by six organizations from five countries. The different simulations show good agreements in terms of the general trend but there are differences in some results such as the peak temperatures of different regions and heat removal rate.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

The HTR-Proteus Benchmark: Analysis and Use as a Verification and Validation Case

This presentation outlines the evaluation and application of the HTR-Proteus benchmark as a verification and validation (V&V) case for advanced reactor modeling tools. The work supports the U.S. Department of Energy’s HALEU Availability Program (HAP) and the joint DOE/NRC DNCSH project, which aims to reduce criticality safety uncertainties in commercial-scale HALEU fuel cycle and transportation systems. The HTR-Proteus experiments, conducted at the Paul Scherrer Institute, provide high-fidelity data for TRISO-fueled, graphite-moderated pebble bed reactors with high neutron leakage—conditions relevant to HALEU transport scenarios. This study focuses on Cores 4.2 and 4.3 of the HTR-Proteus benchmark, analyzing key sources of uncertainty including pebble packing, TRISO particle positioning, and core height. Using Project Chrono for realistic pebble geometries and Serpent, SHIFT, and MCNP for neutronics simulations, the study quantifies the impact of these uncertainties on the effective multiplication factor (keff). Results show that a sample size of 110 pebble configurations is sufficient to converge keff, with ±30 pcm uncertainty due to packing randomness. TRISO positioning and core height variations also significantly influence keff, highlighting the importance of detailed modeling in V&V efforts. The benchmark serves as a valuable test case for validating the Griffin reactor physics code and improving confidence in HALEU system simulations.

73 - NUCLEAR PHYSICS AND RADIATION PHYSICS↗

IRDFF-II: A New Neutron Metrology Library

High quality nuclear data is the most fundamental underpinning for all neutron metrology applications. This paper describes the release of version II of the International Reactor Dosimetry and Fusion File (IRDFF-II) that contains a consistent set of nuclear data for fission and fusion neutron metrology applications up to 60 MeV neutron energy. The library is intended to support: a) applications in research reactors; b) safety and regulatory applications in the nuclear power generation in commercial fission reactors; and c) material damage studies in support of the research and development of advanced fusion concepts. The paper describes the contents of the library, documents the thorough verification process used in its preparation, and provides an extensive set of validation data gathered from a wide range of neutron benchmark fields. The new IRDFF-II library includes 119 metrology reactions, four cover material reactions to support self-shielding corrections, five metrology metrics used by the dosimetry community, and cumulative fission products yields for seven fission products in three different neutron energy regions. In support of characterizing the measurement of the residual nuclei from the dosimetry reactions and the fission product decay modes, the present document lists the recommended decay data, particle emission energies and probabilities for 68 activation products. It also includes neutron spectral characterization data for 29 neutron benchmark fields for the validation of the library contents. Additional six reference fields were assessed (four from plutonium critical assemblies, two measured fields for thermal-neutron induced fission on 233U and 239Pu targets) but not used for validation due to systematic discrepancies in C/E reaction rate values or lack of reaction-rate experimental data. Another ten analytical functions are included that can be useful for calculating average cross sections, average energy, thermal spectrum average cross sections and resonance integrals. The IRDFF-II library and comprehensive documentation is available online at www-nds.iaea.org/IRDFF/. Evaluated cross sections can be compared with experimental data and other evaluations at www-nds.iaea.org/exfor/endf.htm. The new library is expected to become the international reference in neutron metrology for multiple applications.

IAEA IRDFF↗

Validation of SAS4A/SASSYS-1 for Steady-State Single-Phase Natural Circulation

The development of numerical software for engineering applications requires the validation of the code against experimental benchmark datasets. To support the validation of SAS4A/SASSYS-1 in simulating the single-phase natural circulation within nuclear reactor systems, high-precision steady-state experiments are performed to capture single-phase natural circulation phenomena on an existing scaled facility with comprehensive instrumentation. Forced convection tests are designed precisely capturing the facility’s critical thermal-hydraulic parameters required for one-dimensional modeling, and a comprehensive single-phase natural circulation dataset is obtained with well-documented experimental uncertainty and facility description. Analysis and discussion based on the finalized dataset confirm the dataset’s ability in capturing dominant phenomena and address important physical interpretation of parameters. The dataset reported in this project provides a valuable benchmark resource for the validation of system analysis codes under single-phase natural circulation inside nuclear reactors. Validation of SAS4A/SASSYS-1 is then performed against the obtained benchmark dataset to confirm the capability of its physics model in simulating the steady-state single-phase natural circulation. The experimental facility is modeled in the candidate code. Solution verification is performed using Richardson-extrapolation-based estimators to quantify and restrict numerical errors from discretization. Numerical uncertainty originating from finite maximum pseudo-transient time is also quantified and restricted. Input uncertainty provided by the benchmark dataset is forward propagated through the candidate code, directly quantifying the simulation output in a Monte Carlo approach. The composition of the output uncertainty is further determined by the estimators of Sobol’ indices through a variance-based sensitivity analysis. With uncertainty quantified for each individual condition, detailed comparison between the simulation results and experimental data is performed covering the whole dataset, which shows consistent agreement for all important quantities. The validation activity reported in this project demonstrates that SAS4A/SASSYS-1 can predict the primary parameters with satisfactorily accuracy under steady-state single-phase natural circulation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Study the thermal radiation effects in gas-solid flows with gray and non-gray P1 models implemented in MFiX

Thermal radiation is a dominant mode of heat transfer in combustion/gasification, packed/circulating bed reactors, and energy storage/conversion devices. In hydrocarbon fuel combustion, the high absorption and emission of product gases like CO 2 and H 2 O significantly affect the heat transfer characteristics. It is essential to understand the radiative energy propagation in such applications. In the present study, we developed a framework to implement the P-1 radiation model in MFiX for Eulerian-Eulerian (Two fluid-TFM) and Eulerian-Lagrangian (Discrete element-DEM and Particle in cell-PIC) models. The P-1 is a simplification of the spherical harmonics method. The framework has been verified on various flue gas conditions. The verified solver results are validated against the benchmark results available in the literature. The verified and validated gray and non-gray weighted sum of gray gas models is evaluated for dry and wet flue gas conditions. Further, they are also used to study gas-solid flows in a fluidized bed/riser.

20 FOSSIL-FUELED POWER PLANTS↗

Development of SAM Code Capabilities for Safety Analysis of GCR Air-ingress Events

Air-ingress following a depressurized loss-of-forced-cooling (DLOFC) event is a challenging, multiphysics safety scenario for High-Temperature Gas-Cooled Reactors (HTGRs), involving coupled gas composition transport, buoyancy-driven flow redistribution, graphite oxidation, and structural heat-up. Despite its importance — air ingress is a key scenario identified in the PIRT process for the HTGRs — existing system-level safety codes have lacked the integrated capability to simulate the complete event sequence with high confidence. This report documents the development, validation, and demonstration of three new capabilities in the SAM code to address this gap: (1) a multi-component gas mixture flow model with binary diffusion to track the helium-air composition and its effect on system density and flow; (2) a 0-D graphite oxidation model based on the Roes correlation, including oxygen consumption and exothermic heat release; and (3) an isentropic critical flow model for accurate representation of primary system depressurization through a break. These capabilities are validated against two benchmark experiments. The NSTF heavy-gas ingress experiment validates the multi-component flow model: SAM correctly reproduces the rapid buoyancydriven flow stagnation and subsequent natural circulation recovery driven by composition-dependent density changes. The NACOK graphite oxidation experiment validates the oxidation model: SAM predicts a bottom-level graphite weight loss of 25%, in close agreement with the measured 24%, and reproduces the strong axial nonuniformity and block-geometry dependence of oxidation, at a level comparable to the SPECTRA and TINTE codes. The validated capabilities are then exercised together in an integrated, reactor-scale simulation of a DLOFC air-ingress transient in a simplified HTR-PM pebble-bed reactor. In a single calculation spanning approximately 8 days, SAM reproduces the complete accident sequence: rapid depressurization, densityand diffusion-driven air ingress over ˜15 hours, onset of buoyancy-driven natural circulation, exothermic graphite oxidation with a peak fuel temperature at ˜62 hours, and eventual passive cooldown. These results demonstrate that SAM now provides the nuclear community with a preliminarily validated, modern systemlevel tool for HTGR air-ingress safety analysis, filling a recognized capability gap. Future extensions to broaden species tracking, improve oxidation chemistry, and refine the reactor model are discussed.

Yang, Gang↗

Survey of Relevant Data from the MSRP to Guide Development of MSR Chemistry Modeling Benchmarks

The Multiphysics Applications technical area of the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program is tasked with assessing, demonstrating, and applying NEAMS tools in solving multiphysics problems of nuclear reactors, such as liquid-fueled molten salt reactors (MSRs), which are the focus of this report. MSRs are actively being pursued as a potential candidate for power and heat generation by the nuclear industry. However, the inherent multiphysics nature of liquid-fueled MSRs, stemming from the strong interrelationship of neutronics, thermal fluids, and chemistry phenomena, provides unique challenges in modeling and simulation (mod/sim). Therefore, it will be important to develop mod/sim tools with varying types of multiphysics coupling that depend on the problem. The current work is focused on the initiation and development of MSR chemistry modeling benchmarks useful for validating current and potential future NEAMS tools. Development of such benchmarks include the following actions: 1) Summarize available chemistry data from the Oak Ridge National Laboratory (ORNL) MSR Program (MSRP) including operation of the Molten Salt Reactor Experiment (MSRE) and design studies for the Molten Salt Breeder Reactor (MSBR) concept; 2) Recommend simulation problems in MSR chemistry mod/sim based on (1); 3) Assess the current state of NEAMS tools that may support (2); 4) Demonstrate and validate the capabilities of the NEAMS tools in (3) while providing iterative feedback on future code development activities. The objective of this report is to initiate this effort by completing actions (1) and (2), which are discussed in Section 2. An overview of the relevant available MSRE experimental data is provided with examples of how this data may be useful in chemistry mod/sim problems, with the caveat that most of this data is over 50 years old, therefore some problem details as well as uncertainty estimates are often not provided. In Section 3, the current state of NEAMS tools is assessed for potential use in these mod/sim problems, with considerations for future code development activities, supporting action (3). Future work supporting this project under NEAMS may include a deeper dive into the specific phenomena discussed here including tasks such as the compilation of additional available data, updates in code development activities, and ultimately the demonstration and validation of these tools, supporting action (4).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Estimates of Southern Hemispheric Gravity Wave Momentum Fluxes across Observations, Reanalyses, and Kilometer-Scale Numerical Weather Prediction Model

Abstract Gravity waves (GWs) are among the key drivers of the meridional overturning circulation in the mesosphere and upper stratosphere. Their representation in climate models suffers from insufficient resolution and limited observational constraints on their parameterizations. This obscures assessments of middle atmospheric circulation changes in a changing climate. This study presents a comprehensive analysis of stratospheric GW activity above and downstream of the Andes from 1 to 15 August 2019, with special focus on GW representation ranging from an unprecedented kilometer-scale global forecast model (1.4 km ECMWF IFS), ground-based Rayleigh lidar (CORAL) observations, modern reanalysis (ERA5), to a coarse-resolution climate model (EMAC). Resolved vertical flux of zonal GW momentum (GWMF) is found to be stronger by a factor of at least 2–2.5 in IFS compared to ERA5. Compared to resolved GWMF in IFS, parameterizations in ERA5 and EMAC continue to inaccurately generate excessive GWMF poleward of 60°S, yielding prominent differences between resolved and parameterized GWMFs. A like-to-like validation of GW profiles in IFS and ERA5 reveals similar wave structures. Still, even at ∼1 km resolution, the resolved waves in IFS are weaker than those observed by lidar. Further, GWMF estimates across datasets reveal that temperature-based proxies, based on midfrequency approximations for linear GWs, overestimate GWMF due to simplifications and uncertainties in GW wavelength estimation from data. Overall, the analysis provides GWMF benchmarks for parameterization validation and calls for three-dimensional GW parameterizations, better upper-boundary treatment, and vertical resolution increases commensurate with increases in horizontal resolution in models, for a more realistic GW analysis. Significance Statement Gravity wave–induced momentum forcing forms a key component of the middle atmospheric circulation. However, complete knowledge of gravity waves, their atmospheric effects, and their long-term trends are obscured due to limited global observations, and the inability of current climate models to fully resolve them. This study combines a kilometer-scale forecast model, modern reanalysis, and a coarse-resolution climate model to first compare the resolved and parameterized momentum fluxes by gravity waves generated over the Andes, and then evaluate the fluxes using a state-of-the-art ground-based Rayleigh lidar. Our analysis reveals shortcomings in current model parameterizations of gravity waves in the middle atmosphere and highlights the sensitivity of the estimated flux to the formulation used.

Meteorology & Atmospheric Sciences↗

Heat Transfer Coefficients of Additively Manufactured Tubes with Internal Pin Fins for Supercritical Carbon Dioxide Cycle Recuperators

This paper describes the measurement of convective heat transfer coefficients and friction factors for sCO 2 flowing in additively manufactured tubes with internal pin fins at the US DOE’s National Energy Technology Laboratory in Morgantown, WV. The measurement procedures were validated by conducting benchmark tests with smooth stainless-steel tube and comparing the results with published correlations for Nusselt number (Nu) and friction factor. Over Reynolds numbers (ReD) ranging from 5 × 10 4 to 2.5 × 10 5 , measured Nu was within 5% of the Dittus-Boelter correlation and measured friction factors were within 5% of the McAdams correlation for smooth tube flow. The candidate pin fin patterned pipes were additively manufactured (AM) at the Oak Ridge National Laboratory. The pins were circular or elliptical in cross-section and were printed at a 30⁰ angle relative to the inner wall (to meet AM constraints). The pin arrangement was helical to promote enhanced heat transfer due to swirl flow. Pin length to diameter aspect ratio was 1.33, 2, and 8, while the pin diameter to tube diameter ratio was 0.188, 0.125, and 0.063. Tests were performed for ReD varying from 6.9 × 10 4 to 2.2 × 10 5 and at conditions equivalent to the low pressure (LP) outlet (8.69 MPa, 361 K) and the high pressure (HP) inlet (20.7 MPa, 350 K) of the low temperature recuperator (LTR) in an indirect sCO 2 cycle. The Wilson plot technique was utilized to measure the bulk heat transfer coefficients. For the best performing design (tube A, pin length to tube diameter ratio: 1.33, pin diameter to tube diameter ratio: 0.19), the local heat transfer coefficient increased by 136% relative to the Dittus-Boelter correlation at the LTR low pressure outlet and 194% at the LTR high pressure inlet. These correspond to a 282% and a 271% increase in the product of the heat transfer coefficient and surface area (adjusted for fin efficiency) product, respectively. Large pressure drops across the test articles were observed. For Tube Design A, the average friction factor, across the range of ReD considered, was significantly larger than the McAdams correlation at both the LTR LP outlet and the LTR HP inlet. A thermal performance factor was utilized to express the ratio of material required to build a finned heat exchanger relative to a finless heat exchanger with the same heat duty and pumping power. Tube Design A was estimated to decrease the required heat exchanger material by 13%.

Searle, Matthew↗

A Verification of Flux Sensitivity Estimates Using the MCNP Tally Perturbation Tool

Nuclear data is commonly used in applications such as nuclear nonproliferation, safeguards, and criticality safety. More specifically, nuclear data is used in predictive simulation codes like the Monte-Carlo N-Particle (MCNP ® ) transport code, Serpent, and similar radiation transport codes. The improvement of nuclear data enables more precise and accurate simulations, which result in higher fidelity designs and reduced operational/procedural costs. Therefore, the improvement of nuclear data is of paramount importance across the nuclear community. Nuclear data is improved and validated through integral benchmark experiments. The design of benchmark experiments is an extensive process; therefore, these experiments are often optimized on multiple characteristics, including sensitivity to the nuclear data, during the design process. Sensitivity is a measure of how much a quantity changes due to changes in independent variables such as experimental configuration. An experimental design that has a larger sensitivity to the nuclear data of interest will have a larger impact on the accuracy and precision of the validated data. Past integral benchmark experiments have primarily used the effective multiplication factor ($k_{eff}$) as the predominant measured quantity; however, experiments designed with other quantities in mind would be able to optimize on validating different areas of the nuclear data. A primary goal of the EUCLID project is to design, constrain, and reduce compensating errors in experiments focused on quantities other than $k_{eff}$ to better validate nuclear data across the board. Currently, there is a capability in MCNP to easily calculate the sensitivity of $k_{eff}$ to specific nuclear data of numerous reactions types and isotopes (KSEN card); however, the sensitivity of other quantities must be estimated in more strenuous manners. For example, the perturbation feature (PERT card) of MCNP can be used to estimate first-order sensitivities of some response in fixed source simulations. A recent announcement revealed that the first- and second-order perturbation features in previous releases of MCNP contained a bug. It was identified that particles were being scored into the wrong energy bin. The bug is in the most recent public release (MCNP6.2); however, a patch has been added to the most up to date version (MCNP6.2.2) that has not been released publicly. A direct comparison of the PERT card results for an F4 (neutron flux averaged over a cell) tally before and after the patch are shown in figure 1. All simulations used in the sensitivity estimates in this report were performed with MCNP6.2.2. This work verifies the patched MCNP perturbation tool by comparing first order sensitivities made using the PERT card to estimates made using manual perturbation of the compact ENDF (ACE) files.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Hardware-In-the-Loop Benchmarking Setup for Phasor Based Control Validation

Phasor Based Control is a novel approach to controlling Distributed Energy Resources that aims at relieving various constraints that arise in the distribution grid. It is a two-layer control system with a supervisory control that coordinates distributed controllers to reach voltage phasor targets. The distributed controllers use local synchrophasor measurements and operate as feedback controllers. This control method is currently under development with several algorithms being under consideration for both the central and distributed components. In this report, we present the experimental setup that was prepared to prototype a hardware implementation and validate the control method in Hardware-In-the-Loop.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Hardware-In-the-Loop Benchmarking Setup for Phasor Based Control Validation

Phasor Based Control is a novel approach to controlling Distributed Energy Resources that aims at relieving various constraints that arise in the distribution grid. It is a two-layer control system with a supervisory control that coordinates distributed controllers to reach voltage phasor targets. The distributed controllers use local synchrophasor measurements and operate as feedback controllers. This control method is currently under development with several algorithms being under consideration for both the central and ditributed components. In this paper, we present the experimental setup that was prepared to prototype a hardware implementation and validate the control method in Hardware-In-the-Loop.

Baudette, Maxime↗

Propagating neutronic uncertainties for FFTF LOFWOS Test #13

The safety evaluation conducted for licensing a Sodium-cooled Fast Reactor (SFR) may require modeling transients with best-estimate calculation tools that must first be validated against real-world measurements. To provide the community with a valuable benchmarking opportunity for validating SFR analysis tools and methods, the International Atomic Energy Agency (IAEA) initiated a coordinated research project (CRP) in 2018 for the analysis of the Fast Flux Test Facility (FFTF) Loss of Flow Without Scram (LOFWOS) Test #13. The impact of nuclear data uncertainties on neutronics parameters was previously investigated based on the COMMARA-2.0 covariance matrix. Since the transient simulation results are very sensitive to certain reactivity coefficients, it was decided to employ rigorous uncertainty propagation methods to quantify the impact of nuclear data uncertainties on the best-estimate predication of FFTF LOFWOS Test #13. The DAKOTA code is used to propagate neutronic uncertainties through SAS4A/SASSYS-1 transient simulations, while taking into account spatial and reaction-wise correlations within these uncertainties. This study shows that the remaining discrepancies observed between the Argonne National Laboratory (ANL) best-estimate results and the experimental measurements can be partly explained by the uncertainty associated with Gas Expansion Module (GEM) worth, which contributes the majority of the overall nuclear data uncertainty on the output from the FFTF LOFWOS Test #13 transient simulation. This study also confirmed the importance of including spatial and reaction-wise correlations of nuclear data uncertainties on feedback coefficients in the uncertainty propagation to avoid under-estimating their impact during the transient simulations.

42 ENGINEERING↗

Systematic Benchmarking of Climate Models: Methodologies, Applications, and New Directions

As climate models become increasingly complex, there is a growing need to comprehensively and systematically assess model performance with respect to observations. Given the increasing number and diversity of climate model simulations in use, the community has moved beyond simple model intercomparison and toward developing methods capable of benchmarking a large number of simulations against a suite of climate metrics. Here, we present a detailed review of evaluation and benchmarking methods and approaches developed in the last decade, focusing primarily on scientific implications for Coupled Model Intercomparison Project (CMIP) simulations and CMIP6 results that contributed to the Intergovernmental Panel on Climate Change (IPCC) Sixth Assessment Report (AR6). Based on this review, we explain the resulting contemporary philosophy of model benchmarking, and provide clear distinctions and definitions of the terms model verification, process validation, evaluation, and benchmarking. While significant progress has been made in model development based on systematic evaluation and benchmarking efforts, some climate system biases still remain. The development of open‐source community software packages has played a fundamental role in identifying areas of significant model improvement and bias reduction. We review the key features of several software packages that have been commonly used over the past decade to evaluate and benchmark global and regional climate models. Additionally, we discuss best practices for the selection of evaluation and benchmarking metrics and for interpreting the obtained results, the importance of selecting suitable sources of reference data and accurate uncertainty quantification.

Environmental sciences↗