Search NASA⌕ Search

SEARCH · Search NASA

Results for “Validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Development of criticality safety validation guidance for NRC-regulated activities

Some guidance documents support validation of criticality safety evaluations for many system types, and some accommodate different regulators or regulatory requirements. While these reports address specific applications, no single document applies to all systems and incorporates both traditional and sensitivity/uncertainty-based validation techniques. Oak Ridge National Laboratory will develop a single reference report for the US Nuclear Regulatory Commission (NRC) that provides modern examples and suggested recommendations for a uniform approach for performing and documenting k{sub eff} calculation validations applicable to any system containing fissile material. The project will include a literature review and development of a new, comprehensive NUREG/CR document on nuclear criticality safety validation. The NUREG will address the following experiences from recent reviews: 1) Inappropriate critical experiment selection; 2) Insufficient trending analysis; 3) Incorrectly calculated 95/95 uncertainties; 4) Failure to address how validation applies to safety analysis; 5) Lack of goodness-of-fit testing in trending analyses; 6) Incorrect use of commercial reactor critical data; 7) Failure to meet bias determination methodology prerequisites; 8) Inappropriate extrapolation of bias and uncertainty; 9) Failure to identify validation analysis gaps and weaknesses; and 10) Inadequate validation analysis documentation. The final report will provide guidance on use of trending analysis results, acceptable confidence levels for normality testing and trending analysis, management of validation analysis gaps and weaknesses, use of sensitivity and uncertainty analysis in validation analysis, and potential impact of correlations among critical experiments used in validation. It will also consolidate existing guidance documents using both traditional and sensitivity/uncertainty-based approaches. This document will support NRC-regulated activities, as well as validation used in criticality safety within other regulatory frameworks. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2017

OECD/NEA EGMPEBV Activities in Multi-Physics Verification and Validation

The Expert Group on Multi-Physics Experimental Data, Benchmarks and Validation (EGMPEBV) was established in 2014 by the Nuclear Energy Agency (NEA) at the Organization for Economic Co-operation and Development (OECD) to establish processes for certification of experimental data and development of benchmark models for validation of Multi-Physics (MP) Modeling and Simulation (M and S) computational systems. The aim of the group is to provide member countries with guidelines and recommendations for validating and improving their novel multi-physics simulations, and with an access to key experimental data. To be used to their full potential, these novel tools will require a more complex suite of validation experiments due to the multiple energy, spatial and temporal scales, as well as the number of physical phenomena being simulated. They will also require accurate measurements of all terms, including coupling terms representing the feedback effects, in situations where the appropriate experimental techniques and facilities might not exist today. However, the ability to conduct validation experiments has for certain applications progressed at a slower pace than for computational methods, or in some cases it has significantly regressed (through shutdown of facilities, or through retirement of experts). In addition, as modelling capabilities are reaching deeper levels in single physics phenomena and in coupled behaviors, appropriate experimental techniques may not yet exist. Validation of multiple physics models requires a wide variety of experimental data, which emphasizes the importance of maximizing the use of historically accumulated data to avoid the significant cost of performing similar experiments today. Moreover, the pedigree and quality of this data as it pertains to modern day simulation methods may be insufficient for rigorous validation of high fidelity modeling and simulation tools. Thus, these data can often exhibit limitations including a lack of measurement accuracy or a lack of knowledge of the measurement uncertainty in the experimental techniques; an absence of measurements for some critical parameters; an obscuring of relationships between individual physics phenomena owing to the output data resolution; or simply an inability to allow for a detailed reinterpretation of the experiments based on the documented information. The organization of the EGMPEBV relies on three Task Forces, focused on 1) Experimental Data Qualification and Benchmark Evaluation; 2) Methods and Standards, and 3) Specific Applications. Task Force 1 is focused on providing better and more accurate experimental datasets, which support validation of high-fidelity multi-physics modelling and simulation (M and S) tools. At the same time it has to consider the incompleteness of past experimental data sets (in terms of data, documentation, or uncertainties) and with the limited number of available multi-physics experimental facilities. Task Force 2 is designed to develop validation methods and guidelines, and uncertainty qualification for the new family of multi-physics, multi-scale codes in the context of emerging demands such as longer fuel cycles and power uprate. Task Force 3 is focused on applications of validation experiments and has the objective to organize multi-physics M and S benchmarks involving validation experiments.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Blind Modeling Validation Exercises Using the Horizontal Dry Cask Simulator

The U.S. Department of Energy (DOE) established a need to understand the thermal-hydraulic properties of dry storage systems for commercial spent nuclear fuel (SNF) in response to a shift towards the storage of high-burnup (HBU) fuel (> 45 gigawatt days per metric ton of uranium, or GWd/MTU). This shift raises concerns regarding cladding integrity, which faces increased risk at the higher temperatures within spent fuel assemblies present within HBU fuel compared to low-burnup fuel (≤ 45 GWd/MTU). A dry cask simulator (DCS) was built at Sandia National Laboratories (SNL) in Albuquerque, New Mexico to produce validation-quality data that can be used to test the accuracy of the modeling used to predict cladding temperatures. These temperatures are critical to evaluating cladding integrity throughout the storage cycle of commercial spent nuclear fuel. A model validation exercise was previously carried out for the DCS in a vertical configuration. Lessons learned during the previous validation exercise have been applied to a new, blind study using a horizontal dry cask simulator (HDCS). Three modeling institutions – the Nuclear Regulatory Commission (NRC), Pacific Northwest National Laboratory (PNNL), and Empresa Nacional del Uranio, S.A., S.M.E. (ENUSA) – were granted access to the input parameters from the DCS Handbook, SAND2017-13058R, and results from a limited data set from the horizontal BWR dry cask simulator tests reported in the HDCS update report, SAND2019-11688R. With this information, each institution was tasked to calculate peak cladding temperatures and air mass flow rates for ten HDCS test cases. Axial as well as vertical and horizontal transverse temperature profiles were also calculated. These calculations were done using modeling codes (ANSYS/Fluent, STAR-CCM+, or COBRA-SFS), each with their own unique combination of modeling assumptions and boundary conditions. For this validation study, the ten test cases of the horizontal dry cask simulator were defined by three independent variables – fuel assembly decay heat (0.5 kW, 1 kW, 2.5 W, and 5 kW), internal backfill pressure (100 kPa and 800 kPa), and backfill gas (helium and air). The plots provided in Chapter 3 of this report show the axial, vertical, and horizontal temperature profiles obtained from the dry cask simulator experiments in the horizontal configuration and the corresponding models used to describe the thermal-hydraulic behavior of this system. The tables provided in Chapter 3 illustrate the closeness of fit of the model data to the experiment data through root mean square (RMS) calculations of the error in peak cladding temperatures (PCTs), PCT axial locations, axial temperature profiles, vertical and horizontal temperature profiles at two different axial locations, and air mass flow rates for the ten test cases, normalized by the experimental results. The model results are assigned arbitrary model numbers to retain anonymity. Due to the relatively flat axial temperature profiles, small temperature gradients resulted in large deviations of all models’ PCT axial location from the experimental PCT axial location. When the PCT axial location error is excluded in the calculation of the combined RMS of the normalized errors that considers PCT, the temperature profiles, and the air mass flow rates, the model data fits the experimental data to within 5%. When the vault information is excluded, the model data fits the experimental data to within 2.5%. An error analysis was developed further for one model, using the model and experimental uncertainties in each validation parameter to calculate validation uncertainties. The uncertainties for each parameter were used to define quantifiable validation criteria. For this analysis, the model was considered validated for a given comparison metric if the normalized error in that metric divided by the validation uncertainty was less than or equal to 1. When considering the combined RMS of the normalized errors of all metrics divided by their validation uncertainties, the model was found to have satisfied the criterion for model validation.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS2020

Validation on the Europa Clipper Mission in the Formulation Phase

The Europa Clipper mission will perform a detailed investigation of Europa’s environment and habitability by placing a spacecraft into a looping orbit around Jupiter and performing repeated close Europa flybys. The Europa Clipper Project Verification and Validation (V&V) team is responsible for verifying that the Project System is compliant with the science and engineering requirements levied on it and validating that it is capable of meeting the mission objectives. Europa Clipper has developed a V&V framework to ensure that mission capabilities are realized at a reasonable confidence level prior to launch. The V&V team will coordinate the validation activities defined by this framework throughout the project lifecycle as the spacecraft evolves from design to implementation and operations. Validation of space missions is typically grouped into three categories: 1) requirements validation, 2) model validation, and 3) system validation. System validation, along with verification activities, are often the primary focus of V&V programs and these efforts peak in the implementation phase. However, the increasing reliance of exploration missions on models and simulations, and the complexity of the Europa Clipper mission, have dictated attention to requirements and model validation in the formulation phase. This paper discusses the development of the Europa Clipper V&V framework with emphasis on the validation of models and simulations, and of requirements. Approaches include the adaptation of institutional guidelines, such as those in the NASA-STD-7009 document, to create a Project Model V&V Plan and the use of tools such as DOORS NG to plan model validation and derive requirement quality metrics.

Srivastava, Priyanka↗
DOE OSTI2019

Improvements to Bison Validation Base for LWR Fuel

The Bison fuel performance code is being developed by Idaho National Laboratory (INL) to analyze a wide range of fuel forms and operating conditions with the flexibility to conduct simulations in 1D through 3D geometries. Bison plays an important role in the development of advanced fuel with improved accident tolerance for existing light water reactors (LWRs), the improved understanding of mechanisms in fuel designs in a wider range of operating conditions, and the facilitation of future development of fuel for advanced reactor designs. Additionally, industry interest in advanced fuels has also driven Bison development for advanced fuels in the last few years. All of these factors combine to create a potential increase of new Bison users. These new users will potentially be interested in the Bison LWR validation base for examples of Bison's capabilities and how to use Bison. Current users also rely on the Bison validation base as a demonstration of the best practices settings and material behavioral models. Bison, both through direct updates to the Bison source code and to the nearly daily changes and improvements in the underlying MOOSE framework source code, undergoes nearly continuous changes. It is necessary that these source code changes are reflected with updated input files in the Bison validation base. Periodic updates of the validation case input files should also be accompanied by a review and revision of the validation case documentation on which our users rely. Therefore our efforts under this milestone were guided by two objectives: (1) Update the Bison LWR validation input files to use current Bison simulation settings, and (2) Enhance the ease-of-use for Bison through improved documentation These objectives build on the goals identified FY18 CASL milestone reports. During FY18 we addressed issues with the robustness of Bison and identified a need to migrate the assessment suite to the more reliable tensor mechanics module system and set a goal of standardizing the Bison input files. The FY18 CASL milestone report on updates to the Bison documentation system listed continued development of user-focused guides and examples along with revisions to the validation case documentation as future development goals. During FY19 we have separated these two objectives into four different focus areas for our work on this milestone. Our first two focus areas target the existing Bison validation suite: update the existing LWR validation cases and the accompanying documentation. The second two focus areas center on adding new capabilities: a new Bison LWR validation case and expansion of user support documentation. In the following report each of these four focus areas are discussed as a separate section. We conclude the report with a summary of potential future work to build on the efforts of this year.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Progress Towards the Validation of a new RELAP5-3D model of the High Temperature Test Facility

Validation is a key step in the development of any type of systems model. As the next generation of reactors approaches, the need for codes that have been validated for these new types of systems continues to grow. An example of a prominent option is the Reactor Excursion Leak Analysis Program (RELAP5-3D), developed by Idaho National Laboratory. This code was developed for the purpose of systems level thermal-hydraulic modeling of light water reactors (LWRS) and postulated transients that can occur in LWRS.RELAP5-3D has been substantially validated against LWR data. Due to its long history as a reactor safety analysis tool, there has been an effort to adapt RELAP5-3D for the purposes of advanced reactor concepts such as prismatic high-temperature gas-cooled reactors (HTGRs). However, RELAP5-3D has not nearly been validated and verified for HTGRs to the degree of LWRs, warranting verification and validation opportunities with computational benchmarks and existing experimental facilities. Examples of such facilities include the modular high-temperature gas-cooled reactor (MHTGR) 350 and the high temperature engineering test reactor (HTTR) from Japan. The MHTGR 350 is a benchmark design concept for code-to-code verification purposes; therefore, it does not provide any experimental data for validation opportunities The HTTR provides useful multiphysics validation data but does not have the in-core instruments to generate thermal-hydraulic experimental data to help with RELAP5-3D validation. Consequently, a facility that could provide key in-core temperatures for thermal-hydraulic validation was still needed. The High Temperature Test Facility (HTTF) is an integral effects facility for HTGR thermal hydraulics developed and operated by Oregon State University. HTTF represents ¼ length scale of the General Atomics MHTGR and is rated for a total power of 2.2 MW. Axially, the core consists of an upper and lower reflector and 10 blocks, numbered from bottom to top (Block 1 is right above lower reflector). The core is heated via graphite resistive heater rods, with respective channels distributed throughout the core. The primary coolant is helium and heat can radiate out of the core to the reactor cavity cooling system (RCCS), which is cooled by water. The primary purpose of the facility is to investigate pressurized conduction cooldown (PCC) and depressurized conduction cooldown (DCC) transients, which are also referred to as the pressurized and depressurized loss of forced cooling respectively. Two experiments were chosen to perform the validation study with a RELAP5-3D model of HTTF. These experiments are PG-27 (PCC) and PG-29 (DCC). These were chosen based off of the quality of available experimental data before and during the experiment which led to their inclusion in the HTGR Thermal Hydraulics Benchmark.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2018

Argonne Reactor Code Software Verification and Validation Plan for the Versatile Test Reactor

This Argonne Reactor Code (ARC) software has taken a pivotal role in the design work of the Versatile Test Reactor. The ARC software itself consists of several independent or semiindependent pieces of software that are used in concert to carry out specific analysis and design objectives. In this report, we discuss the validation that is needed for the ARC software overall and discuss available, usable experimental data and how it relates to the VTR project. We also discuss the verification that is required for the ARC software with respect to its usage in the VTR project and give some concrete examples as to what it must accomplish. A list of goals for the verification and validation work are presented as the general procedure that will be implemented for every individual piece of software in ARC. The ARC software has been continuously developed for over 50 years and many of the features used for the VTR project have thus been rigorously tested for accuracy during that timeframe. The ARC software is presently under source code control and undergoes nightly regression testing of not only the individual pieces, but the interconnected aspects that are common to most analysis projects. Its greatest weakness, from a software verification and validation perspective, is both a lack of documentation and outdated documentation. As an example, much of the previous validation work will require contemporary effort to re-evaluate given the updated base data libraries and modifications to the software that have occurred since. This requires new documentation to be produced to support the usage of the software in the VTR project. The bulk of the ARC software is focused on steady state neutronics and TH behavior of a reactor system. In this regard, we have identified the ZPPR, FFTF, and EBR-II programs as the best candidates for sources of validation data for the VTR project. The ZPPR-15 series in the ZPPR program was chosen as it is the best representation of the VTR fuel form and reactor core layout of all of the ZPPR measurements. Additional support for it comes from the fact that most of the data has already been processed into usable ARC models for VTR validation. The FFTF and EBR-II reactors were both functional reactors that provide reactor scale measurements. FFTF is clearly a closer match to VTR and its startup measurements would be a very valuable validation basis. EBR-II has invaluable burnup related measurements that will play a key role in the ARC software validation work. Each piece of ARC software will require a report detailing the software verification. The validation work will be logically split into the results of modeling each identified experimental resource (ZPPR, EBR-II, FFTF). The intended timeline for the VTR project is to have the ARC software verified and validated by the end of fiscal year 2020. This is incumbent on sufficient funding allocated to V&V activities and no change in programmatic priorities.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

Adaptive Protection and Validated Models to Enable Deployment of High Penetrations of Solar PV (PV-MOD)

The availability and validation of various PV models in commercial tools differ, with some models not yet thoroughly validated for advanced inverter functionalities and reliable performance under weak system conditions. Many existing models do not fully incorporate new inverter control functions, which can affect system stability. The increasing deployment of solar PV and other inverter-based resources (IBRs), including distributed energy resources (DERs), is influencing the reliable operation of protection schemes in distribution systems and microgrids. Emerging adaptive protection schemes (APS) offer new opportunities for protecting these systems during varying configurations and DER operating conditions, though their demonstration and validation remain limited. Adaptive protection schemes face similar challenges, as they are typically designed for specific configurations. There is a growing need for tools and methodologies to streamline the deployment of adaptive protection for safe and reliable DER integration. The project main objective was to develop and validate high-fidelity generic models of solar PV facilities for stability, protection, EMT, and QSTS analyses. This objective was achieved, and these models can now be integrated into commercial software tools, enabling utilities, vendors, and developers to study high-penetration PV systems more confidently. The project also demonstrated advanced applications of these models, including the design and deployment of adaptive protection schemes in high-penetration field applications and microgrids, supporting grid safety and reliability. Several milestones were reached by the end of the project. A sophisticated inverter test plan was developed, and inverters representative of the North American marketplace were selected. EPRI and NREL tested various inverters, conforming to IEEE standards. Improvements were made to existing generic models of IBR units, IBR plants, and aggregated feeders for various analyses. The first generic electromagnetic transient (EMT) model for a solar PV plant was developed, conforming to IEEE Std 2800™-2022 and validated against laboratory measurements of a 2.2 MVA large-scale battery energy storage system (BESS) inverter. That model was then used to produce reference responses illustrating examples of validated and verified IBR plant models that pass or fail tests for technical minimum capability and performance as specified in the IEEE standard. The developed, tested, and validated generic models can be used for transmission planning, stability assessments, expansion planning, and evaluating potential future IBR interconnection requirements. They can also support interconnection screens and conformity assessments of IBR plants, including solar PV. The project significantly contributed to the ongoing standardization and model-based representation and verification of IBR responses. The project further addressed challenges of common distribution protection schemes with increasing deployment of DER by developing, validating, and demonstrating adaptive protection schemes (APS) that can improve the reliable and safe integration of DER into distribution systems. New APS were designed using improved DER models for three common distribution systems: a radial feeder, a meshed network, and a microgrid. Modeling and hardware-in-the-loop (HIL) testing of the APS were conducted, successfully showing their effectiveness and selectivity. Proof-of-concept field demonstration was achieved for two APS, i.e., one on a radial feeder and another one in a microgrid. Field demonstration could not be achieved for the APS on a meshed network, primarily due apprehension of one utility partner and also due to limited access to the protective algorithms in the network protectors. Guidelines developed from the lessons learned in the project lay out the general process followed in the design, installation, and commissioning of APS for various distribution systems. Distribution utility partners’ apprehension about field demonstration of the new APS were addressed—with varying success—by taking a stepped risk-management approach of modeling of a wide range of sensitivities first, performing in-depth proof-of-concept testing in the laboratory including HIL next, and finally deliberately implementing and commissioning the actual protection equipment and algorithms into parts of—or in parallel operation to—the three real distribution systems. Future work should include pilot projects that further show the acceptable performance of the developed APS before these schemes be rolled out more widely. Inclusion of both utility and original equipment manufacturers (OEMs) in future projects could increase chances of successful field demonstration. Despite challenges in achieving the field demonstration goal of the project for all three APS, the research significantly contributed to the innovation of adaptive protection solutions for scalable and reliable DER integration into distribution systems. This project significantly enhances the understanding of the impact of using appropriate inverter models on distribution and transmission (T&D) systems. By addressing the limitations of existing generic models, the project introduces high-fidelity models for stability, protection, electromagnetic transient (EMT), and quasi-static time series (QSTS) analyses. These models, integrated into commercial software tools, enable utilities, vendors, and developers to confidently study high-penetration PV systems. The project also demonstrates advanced applications, including adaptive protection schemes (APS) for distribution systems and microgrids, ensuring grid safety and reliability. The technical effectiveness and economic feasibility of the methods are evident through the development and validation of sophisticated inverter test plans and the selection of representative inverters. Testing by EPRI and NREL on retail, commercial, and utility-scale inverters, conforming to IEEE standards, underscores the robustness of the models. Improvements to existing generic models for various analyses further enhance their validity and applicability. The project also identifies gaps in common distribution protection schemes and designed new APS using improved DER models, demonstrating their effectiveness through modeling and hardware-in-the-loop (HIL) testing. The project’s benefits to the public are manifold. By advancing the standardization and model-based representation of IBR response, it supports transmission planning, stability assessments, and future IBR interconnection requirements. The generic models can facilitate better communication between transmission planners and developers, supporting expected IBR plant capability and performance. Additionally, the development of APS for radial feeders, meshed networks, and microgrids supports the integration of distributed energy resources (DERs) into distribution systems, enhancing grid reliability and safety. The project’s emphasis on thorough testing and simplicity in design ensures practical and scalable solutions for DER integration.

14 SOLAR ENERGY↗
NASA NTRS1987

Design and validation of fault-tolerant flight systems

Flight systems must be validated to show that they are consistent with the requirements of their intended applications. While high reliability is difficult to validate, the additional complexity of fault tolerance further compounds the validation problem. The objective of NASA’s research is to develop a methodology for designing validatable fault-tolerant systems. Under the design-for-validation philosophy, emphasis is placed on developing validation methods that can be incorporated into the design process right from the start and design methods and guidance which, while incorporating fault tolerance, can assure validatability. This paper examines the statistical issues of validating highly reliable, fault tolerant system. There are many problems associated with traditional methods of designing and validating these potentially complex hardware and software systems. Useful design-for-validation methods, which include structured specification and design methodologies, mathematical proof techniques, analytical modeling, simulation and emulation, and physical testing, are discussed. Important design issues associated with fault tolerance are presented along with the related validation concerns which must be addressed. Experience has shown that synchronization and Byzantine resilience must accompany fault tolerance. Other design attributes associated with fault tolerance may be used by a designer on the basis of cost, weight, performance, and validation considerations.

Computer systems↗
NASA NTRS1990

CFD validation for aerodynamic flows - Challenge for the '90's

The process of computational fluid dynamics (CFD) validation is described from two perspectives, numerical error validation and physical model validation. Errors associated with time and space discretization, grid refinement, numerical dissipation, and level of solution convergence, are all possible sources of numerical error. The second aspect of CFD code validation is associated with the governing equations and the physical models, e.g., chemistry and turbulence models. This type of validation requires comparisons with carefully conducted experiments and is the primary aspect of the validation process discussed in this paper. Examples of validation are shown for selected Reynolds-averaged Navier-Stokes codes for conditions from transonic through hypersonic. Since code applications are becoming more complex, it no longer suffices to use data from surface or integral quantities alone to provide the required validation. Flow field surveys and experimental boundary condition measurements are emerging as critical data that must be obtained for CFD code validation. Progress in the overall level of validation is shown to be improving. However, future challenges remain, some of which are enumerated.

Marvin, Joseph G.↗
NASA NTRS1989

NASA's CFD Validation Program

With computational fluid dynamics (CFD) becoming a productive research and design tool, the requirement to validate CFD codes has grown significantly. NASA had emphasized CFD validation activities since 1986 when a separate work element was formed to fund experimental activities related to validation. NASA's CFD and CFD validation programs are closely coordinated to ensure that experimental data bases are available as soon as possible for validating codes. In response to industry and academic requirements, four levels of experimental research have been defined as part of CFD validation with NASA's Aeronautics Advisory Committee (AAC) support although only the fourth level actually has the detailed information necessary for validating codes. Critical flow physics especially turbulence modeling are key to improved CFD codes. NASA has focused additional resources on transition and turbulence physics to meet these requirements. With improved turbulence models, CFD codes will be more accurate, robust, and efficient. However, with the level of detailed information available from CFD codes, highly accurate and detailed experiments are required to capture the critical information for validating codes. Advanced instrumentation especially non-intrusive instrumentation is required to acquire this information in validation experiments. The CFD validation program is being coordinated and managed to address these critical activities. A list of experiments which are currently being supported at least partially are included.

Satran, Dale R.↗
NASA NTRS1996

Validation of NASA Thermal Ice Protection Computer Codes: Program Overview - Part 1

The Icing Technology Branch at NASA Lewis has been involved in an effort to validate two thermal ice protection codes developed at the NASA Lewis Research Center. LEWICE/Thermal (electrothermal deicing & anti-icing), and ANTICE (hot-gas & electrothermal anti-icing). The Thermal Code Validation effort was designated as a priority during a 1994 'peer review' of the NASA Lewis Icing program, and was implemented as a cooperative effort with industry. During April 1996, the first of a series of experimental validation tests was conducted in the NASA Lewis Icing Research Tunnel(IRT). The purpose of the April 96 test was to validate the electrothermal predictive capabilities of both LEWICE/Thermal, and ANTICE. A heavily instrumented test article was designed and fabricated for this test, with the capability of simulating electrothermal de-icing and anti-icing modes of operation. Thermal measurements were then obtained over a range of test conditions, for comparison with analytical predictions. This paper will present an overview of the test, including a detailed description of: (1) the validation process; (2) test article design; (3) test matrix development; and (4) test procedures. Selected experimental results will be presented for de-icing and anti-icing modes of operation. Finally, the status of the validation effort at this point will be summarized. Detailed comparisons between analytical predictions and experimental results are contained in the following two papers: 'Validation of NASA Thermal Ice Protection Computer Codes: Part 2- The Validation of LEWICE/Thermal' and 'Validation of NASA Thermal Ice Protection Computer Codes: Part 3-The Validation of ANTICE'

Miller, Dean↗
DOE OSTI2016

Validation and Spatiotemporal Analysis of CERES Surface Net Radiation Product

The Clouds and the Earth’s Radiant Energy System (CERES) generates one of the few global satellite radiation products. The CERES ARM Validation Experiment (CAVE) has been providing long-term in situ observations for the validation of the CERES products. However, the number of these sites is low and their distribution is globally sparse, and particularly the surface net radiation product has not been rigorously validated yet. Therefore, additional validation efforts are highly required to determine the accuracy of the CERES radiation products. In this study, global land surface measurements were comprehensively collected for use in the validation of the CERES net radiation (R n ) product on a daily (340 sites) and a monthly (260 sites) basis, respectively. The validation results demonstrated that the CERES R n product was, overall, highly accurate. The daily validations had a Mean Bias Error (MBE) of 3.43 W·m -2 , Root Mean Square Error (RMSE) of 33.56 W·m -2 , and R 2 of 0.79, and the monthly validations had an MBE of 3.40 W·m -2 , RMSE of 25.57 W·m -2 , and R 2 of 0.84. The accuracy was slightly lower for the high latitudes. Following the validation, the monthly CERES R n product, from March 2000 to July 2014, was used for a further analysis. The global spatiotemporal variation of the R n , which occurred during the measurement period, was analyzed. In addition, two hot spot regions, the southern Great Plains and south-central Africa, were then selected for use in determining the driving factors or attribution of the R n variation. We determined that R n over the southern Great Plains decreased by -0.33 W·m -2 per year, which was mainly driven by changes in surface green vegetation and precipitation. In south-central Africa, R n decreased at a rate of -0.63 W·m -2 per year, the major driving factor of which was surface green vegetation

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

TROPOMI Methane Validation in the GeoCarb Domain

Objective: GeoCarb validation •Purpose: •Do the four TCCON sites in the GeoCarb domain characterize satellite validation? •Method: •Assess the error estimates globally vs. in the GeoCarb domain •Assess the use of GML/DOE aircraft for CH4 validation •Systematic error estimate (from Kulawik, 2016): •+ regional bias (stdevof average bias versus validation at each station) •+ correlated error (stdev of daily average versus validation) •-co-location error (above quantities for model@satelliteminus model@valid) •-validation error (systematic error of validation estimates, e.g. model extension) Aircraft validation error •CH4 profile extension error estimate: 7 ppb •Extend aircraft observations to the top of the atmosphere with different model configurations •CAMS, scaling to stratosphere, then taking stratosphere •Extend by repeating top aircraft value to the troposphere, then taking CAMS •Extend with GEOS-Chem run (from 2010). •Difference CAMS-Scale vs. CAMS-Tropo: •3.6 +-5.5 ppb •Systematic error: 6.6 ppb •Extending aircraft to tropopause is blatantly wrong, but these two have the same stratosphere. •Difference CAMS-Scale vs. GEOS-Scale: •6.2 +-8.7 ppb •Do not consider bias since model is 10 years off. Systematic error ~8.7 ppb. •Estimate of systematic error from profile extension (possibly underestimated, resulting in possibly higher TROPOMI error estimate) •7 ppb •Would likely vary some by season, site, other conditions •Aircraft measurement error: 1 ppb •1 ppb from measurement error3

TROMPOMI↗
DOE OSTI2020

Development and validation of a 30-day mortality index based on pre-existing medical administrative data from 13,323 COVID-19 patients: The Veterans Health Administration COVID-19 (VACO) Index

Background Available COVID-19 mortality indices are limited to acute inpatient data. Using nationwide medical administrative data available prior to SARS-CoV-2 infection from the US Veterans Health Administration (VA), we developed the VA COVID-19 (VACO) 30-day mortality index and validated the index in two independent, prospective samples. Methods and findings We reviewed SARS-CoV-2 testing results within the VA between February 8 and August 18, 2020. The sample was split into a development cohort (test positive between March 2 and April 15, 2020), an early validation cohort (test positive between April 16 and May 18, 2020), and a late validation cohort (test positive between May 19 and July 19, 2020). Our logistic regression model in the development cohort considered demographics (age, sex, race/ethnicity), and pre-existing medical conditions and the Charlson Comorbidity Index (CCI) derived from ICD-10 diagnosis codes. Weights were fixed to create the VACO Index that was then validated by comparing area under receiver operating characteristic curves (AUC) in the early and late validation cohorts and among important validation cohort subgroups defined by sex, race/ethnicity, and geographic region. We also evaluated calibration curves and the range of predictions generated within age categories. 13,323 individuals tested positive for SARS-CoV-2 (median age: 63 years; 91% male; 42% non-Hispanic Black). We observed 480/3,681 (13%) deaths in development, 253/2,151 (12%) deaths in the early validation cohort, and 403/7,491 (5%) deaths in the late validation cohort. Age, multimorbidity described with CCI, and a history of myocardial infarction or peripheral vascular disease were independently associated with mortality–no other individual comorbid diagnosis provided additional information. The VACO Index discriminated mortality in development (AUC = 0.79, 95% CI: 0.77–0.81), and in early (AUC = 0.81 95% CI: 0.78–0.83) and late (AUC = 0.84, 95% CI: 0.78–0.86) validation. The VACO Index allows personalized estimates of 30-day mortality after COVID-19 infection. For example, among those aged 60–64 years, overall mortality was estimated at 9% (95% CI: 6–11%). The Index further discriminated risk in this age stratum from 4% (95% CI: 3–7%) to 21% (95% CI: 12–31%), depending on sex and comorbid disease. Conclusion Prior to infection, demographics and comorbid conditions can discriminate COVID-19 mortality risk overall and within age strata. The VACO Index reproducibly identified individuals at substantial risk of COVID-19 mortality who might consider continuing social distancing, despite relaxed state and local guidelines.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

Plan to Verify and Validate Multi-Hazard Risk-Informed Margin Management Methods and Tools

This report describes the development plan for verification and validation (V&V) of multihazard risk informed margin management (RIMM) methods and tools. The plan will include using new INL developed tool sets coupled with the new multi-partner External Hazards Experimental Group (EHEG) coordinated by the Idaho National Laboratory (INL) within the Risk-Informed Safety Margin Characterization (RISMC) technical pathway of the Light Water Reactor Sustainability Program. The RISMC pathway is developing a suite of new tools and methods for advanced evaluation of facility risk. The RISMC toolkit, includes tools and methods focused on evaluation of risk from external hazards (e.g, seismic and flooding events), which have been shown to be dominant risk contributors in probabilistic risk assessments performed for operating nuclear power plants (NPPs). The external events activity (EEA) within the RISMC pathway is tasked with developing tools and methods focused on evaluation of multi-hazard external risk (e.g, seismic and flooding events). The EHEG will provide technical expertise and experimental large-scale testing data needed for development and validation of tools and methods in the RISMC toolkit for external hazard safety evaluations. The EEA within RISMC has two key elements: (1) an organizational and research framework provided by INL and (2) a coordinated group of university and national laboratory partners with the complementary expertise and experimental capabilities needed to conduct large-scale external hazard-focused experiments. This cooperative group will allow INL to leverage a range of existing capabilities to meet the unique needs of RISMC tool and method development. In addition, the capabilities of the EHEG could be used to address needs of other national laboratories. Currently, there is limited data available for development and validation of the tools and methods being developed in the RISMC Toolkit. The EHEG is being developed to obtain high quality, large-scale experimental data validation of RISMC tools and methods in a timely and cost-effective way. The group of universities and national laboratories that will eventually form the EHEG (which is ultimately expected to include both the initial participants and other universities and national laboratories that have been identified) have the expertise and experimental capabilities needed to both obtain and compile existing data archives and perform additional seismic and flooding experiments. The data developed by EHEG will be stored in databases within RISMC. These databases will be used to validate the advanced external hazard tools and methods. The numerical tools under development in RISMC are used to evaluate the impacts that external hazards have on nuclear power plants. These analysis tools will be used to advanced current risk calculation processes and reduce uncertainty in these calculations. To have confidence in the predictive capability of these numerical tools it is important to verify and validate them. To verify and validate numerical tools the user must understand what physics is being used to represent the problem (i.e. constitutive models used to represent the soil and structure), the numerical solver approach used (i.e. finite element), what data already exists that could be used to validate the tools, and what data is needed for validation.

31274↗
DOE OSTI2021

A Verification and Validation Approach for COMSOL Multiphysics to Support High Flux Isotope Reactor (HFIR)

Over the last several decades, many reactors have successfully been converted from high enriched uranium (HEU) to low enriched uranium (LEU) fuels in United States in support of its global non-proliferation objectives. Of the reactors slated for conversion, five high-performance research reactors (HPRRs) remain. The high-flux isotope reactor (HFIR) at Oak Ridge National Laboratory (ORNL) is one of the five. Conversion of HFIR requires the qualification of a new fuel. To aid in the conversion process as well as improve safety margins, COMSOL Multiphysics was chosen to support and eventually supplement the steady-state heat-transfer code (SSHTC). In order to use COMSOL as a supplementary tool, i.e. one used for safety basis calculations, the code needs to undergo a verification and validation process. Verification and validation are important processes for the use of any software; however, physics-based solvers have a unique challenge, in that the validation is much more rigorous and its domain is only applicable to the general scope of the problem. This document details the verification and validation of COMOSL for analysis of HFIR. This report is intended for two different users: (1) current and future users of COMSOL for HFIR and (2) for individuals interested in assessing the scope of the validation of COMSOL for HFIR. This report is divided into six sections. The first of which details the process of verification and validation. Section one provides an overview of key concepts from ASME V&V 10, 20, and 40 and how these concepts are used to provide a validation of any multiphysics software. Section two covers the approach needed in order to validate multiphysics software for a given HFIR analysis. Parts three through five split up the key concepts highlighted in section two. The final section provides concluding remarks which demonstrate the scope and limitations of the validation study and how these may be expanded and improved.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2019

Photofission Validation Measurements for Active Interrogation Standards

Executive Summary PNNL has been developing technical standards for the detection of special nuclear material (SNM) in cargo containers using active interrogation approaches. Since the use of kilogram-scale quantities of SNM is impractical for standards testing, the work focused on designing surrogates that provided similar signature as the SNM. There are many active interrogation approaches to detect SNM; the standards development considered photofission, differential die-away and nuclear resonance fluorescence. Of these approaches, photofission currently carries the most interest. However, uncertainties in the photofission cross sections are as large as 50%, so that validation measurements are necessary to provide confidence that the surrogate will perform similarly to the SNM. This document describes approaches for conducting those validation measurements as well as provides a rough order-of-magnitude cost estimate for performing them. Photofission provides many observables for the detection of SNM. The standards development considered prompt neutrons, delayed neutrons and delayed gamma rays as possible detected particles. The surrogates, which consist primarily of depleted uranium (DU), were designed to reproduce the strength of these particles for a targeted quantity of 2 kg of HEU. Different surrogates are required for different measurement approaches because of the difference in cross section between 235 U and 238 U. For the standards validations, the relative strength of the detected particles for HEU and DU must be measured. To avoid measurements with kilogram-sized SNM, the validation approach is focused on two comparisons. In the first comparison, the relative rates for approximately 10 g samples of DU and HEU will be measured. This comparison will provide the basis for validating the relative strength of the detected particles for HEU and DU. In the second comparison, the relative rates for the approximately 10 g and 1 kg sample of DU will be measured. This comparison will enable the validation of secondary processes that occur after photofission that impact the measurement for kilogram-scale samples. For instance, for 2 kg of HEU, as much as 50% of the fission induced in the sample is induced from fission neutrons generated through photofission. There is a limited range of facilities in the U.S. that have both the necessary accelerator capability and the ability and willingness to handle SNM. Two such facilities, the Idaho Accelerator Center at Idaho State University and the Nuclear Engineering Program at the University of Michigan, have both expressed an interest in participating in such measurements. Both would need to amend their licensing to handle the 10 g of HEU proposed for these measurements. The rough order-of-magnitude cost estimate for all three validation measurements is $1.1M, spread over three years. Each of the three measurements could be performed separately, at approximately one-third of that cost. There are a number of critical assumptions in these cost estimate that will need to be confirmed before a more reliable cost estimate can be established. PNNL will manage the effort to ensure that the overall objective of the validation measurements can be achieved and will fabricate the 10 g samples of HEU and DU. Most the labor for the measurements will be provided by two collaborating universities as research opportunities for students. The successful completion of these validation measurements will reduce the uncertainties in the proposed standards from about 50% to 5%. The measurements would also provide some much-needed nuclear data on photofission.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗