Results for “Validation” — Page 4

NASA NTRS2003

Validation and Error Characterization for the Global Precipitation Measurement

The Global Precipitation Measurement (GPM) is an international effort to increase scientific knowledge on the global water cycle with specific goals of improving the understanding and the predictions of climate, weather, and hydrology. These goals will be achieved through several satellites specifically dedicated to GPM along with the integration of numerous meteorological satellite data streams from international and domestic partners. The GPM effort is led by the National Aeronautics and Space Administration (NASA) of the United States and the National Space Development Agency (NASDA) of Japan. In addition to the spaceborne assets, international and domestic partners will provide ground-based resources for validating the satellite observations and retrievals. This paper describes the validation effort of Global Precipitation Measurement to provide quantitative estimates on the errors of the GPM satellite retrievals. The GPM validation approach will build upon the research experience of the Tropical Rainfall Measuring Mission (TRMM) retrieval comparisons and its validation program. The GPM ground validation program will employ instrumentation, physical infrastructure, and research capabilities at Supersites located in important meteorological regimes of the globe. NASA will provide two Supersites, one in a tropical oceanic and the other in a mid-latitude continental regime. GPM international partners will provide Supersites for other important regimes. Those objectives or regimes not addressed by Supersites will be covered through focused field experiments. This paper describes the specific errors that GPM ground validation will address, quantify, and relate to the GPM satellite physical retrievals. GPM will attempt to identify the source of errors within retrievals including those of instrument calibration, retrieval physical assumptions, and algorithm applicability. With the identification of error sources, improvements will be made to the respective calibration, assumption, or algorithm. The instrumentation and techniques of the Supersites will be discussed. The GPM core satellite, with its dual-frequency radar and conically scanning radiometer, will provide insight into precipitation drop-size distributions and potentially increased measurement capabilities of light rain and snowfall. The ground validation program will include instrumentation and techniques commensurate with these new measurement capabilities.

Bidwell, Steven W.↗

NASA NTRS2014

Observations on CFD Verification and Validation from the AIAA Drag Prediction Workshops

The authors provide observations from the AIAA Drag Prediction Workshops that have spanned over a decade and from a recent validation experiment at NASA Langley. These workshops provide an assessment of the predictive capability of forces and moments, focused on drag, for transonic transports. It is very difficult to manage the consistency of results in a workshop setting to perform verification and validation at the scientific level, but it may be sufficient to assess it at the level of practice. Observations thus far: 1) due to simplifications in the workshop test cases, wind tunnel data are not necessarily the “correct” results that CFD should match, 2) an average of core CFD data are not necessarily a better estimate of the true solution as it is merely an average of other solutions and has many coupled sources of variation, 3) outlier solutions should be investigated and understood, and 4) the DPW series does not have the systematic build up and definition on both the computational and experimental side that is required for detailed verification and validation. Several observations regarding the importance of the grid, effects of physical modeling, benefits of open forums, and guidance for validation experiments are discussed. The increased variation in results when predicting regions of flow separation and increased variation due to interaction effects, e.g., fuselage and horizontal tail, point out the need for validation data sets for these important flow phenomena. Experiences with a recent validation experiment at NASA Langley are included to provide guidance on validation experiments.

Computational Fluid Dynamics↗

NASA NTRS2015

A Vehicle Management End-to-End Testing and Analysis Platform for Validation of Mission and Fault Management Algorithms to Reduce Risk for NASAs Space Launch System

The engineering development of the National Aeronautics and Space Administration's (NASA) new Space Launch System (SLS) requires cross discipline teams with extensive knowledge of launch vehicle subsystems, information theory, and autonomous algorithms dealing with all operations from pre-launch through on orbit operations. The nominal and off-nominal characteristics of SLS's elements and subsystems must be understood and matched with the autonomous algorithm monitoring and mitigation capabilities for accurate control and response to abnormal conditions throughout all vehicle mission flight phases, including precipitating safing actions and crew aborts. This presents a large and complex systems engineering challenge, which is being addressed in part by focusing on the specific subsystems involved in the handling of off-nominal mission and fault tolerance with response management. Using traditional model-based system and software engineering design principles from the Unified Modeling Language (UML) and Systems Modeling Language (SysML), the Mission and Fault Management (M&FM) algorithms for the vehicle are crafted and vetted in Integrated Development Teams (IDTs) composed of multiple development disciplines such as Systems Engineering (SE), Flight Software (FSW), Safety and Mission Assurance (S&MA) and the major subsystems and vehicle elements such as Main Propulsion Systems (MPS), boosters, avionics, Guidance, Navigation, and Control (GNC), Thrust Vector Control (TVC), and liquid engines. These model-based algorithms and their development lifecycle from inception through FSW certification are an important focus of SLS's development effort to further ensure reliable detection and response to off-nominal vehicle states during all phases of vehicle operation from pre-launch through end of flight. To test and validate these M&FM algorithms a dedicated test-bed was developed for full Vehicle Management End-to-End Testing (VMET). For addressing fault management (FM) early in the development lifecycle for the SLS program, NASA formed the M&FM team as part of the Integrated Systems Health Management and Automation Branch under the Spacecraft Vehicle Systems Department at the Marshall Space Flight Center (MSFC). To support the development of the FM algorithms, the VMET developed by the M&FM team provides the ability to integrate the algorithms, perform test cases, and integrate vendor-supplied physics-based launch vehicle (LV) subsystem models. Additionally, the team has developed processes for implementing and validating the M&FM algorithms for concept validation and risk reduction. The flexibility of the VMET capabilities enables thorough testing of the M&FM algorithms by providing configurable suites of both nominal and off-nominal test cases to validate the developed algorithms utilizing actual subsystem models such as MPS, GNC, and others. One of the principal functions of VMET is to validate the M&FM algorithms and substantiate them with performance baselines for each of the target vehicle subsystems in an independent platform exterior to the flight software test and validation processes. In any software development process there is inherent risk in the interpretation and implementation of concepts from requirements and test cases into flight software compounded with potential human errors throughout the development and regression testing lifecycle. Risk reduction is addressed by the M&FM group but in particular by the Analysis Team working with other organizations such as S&MA, Structures and Environments, GNC, Orion, Crew Office, Flight Operations, and Ground Operations by assessing performance of the M&FM algorithms in terms of their ability to reduce Loss of Mission (LOM) and Loss of Crew (LOC) probabilities. In addition, through state machine and diagnostic modeling, analysis efforts investigate a broader suite of failure effects and associated detection and responses to be tested in VMET to ensure reliable failure detection, and confirm responses do not create additional risks or cause undesired states through interactive dynamic effects with other algorithms and systems. VMET further contributes to risk reduction by prototyping and exercising the M&FM algorithms early in their implementation and without any inherent hindrances such as meeting FSW processor scheduling constraints due to their target platform - the ARINC 6535-partitioned Operating System, resource limitations, and other factors related to integration with other subsystems not directly involved with M&FM such as telemetry packing and processing. The baseline plan for use of VMET encompasses testing the original M&FM algorithms coded in the same C++ language and state machine architectural concepts as that used by FSW. This enables the development of performance standards and test cases to characterize the M&FM algorithms and sets a benchmark from which to measure their effectiveness and performance in the exterior FSW development and test processes. This paper is outlined in a systematic fashion analogous to a lifecycle process flow for engineering development of algorithms into software and testing. Section I describes the NASA SLS M&FM context, presenting the current infrastructure, leading principles, methods, and participants. Section II defines the testing philosophy of the M&FM algorithms as related to VMET followed by section III, which presents the modeling methods of the algorithms to be tested and validated in VMET. Its details are then further presented in section IV followed by Section V presenting integration, test status, and state analysis. Finally, section VI addresses the summary and forward directions followed by the appendices presenting relevant information on terminology and documentation.

Trevino, Luis↗

NASA NTRS2020

Hardware Verification and Validation for a Navigation Sensor Software Model in Support of Flight Vehicle Performance Analysis

… or, “It’s in the details, how to make complicated software perform like complicated hardware.” In attempts to minimize development time and quickly build an operational vehicle, NASA’s Space Launch System (SLS) has had to be intentional about integrated testing. Constraints on budget and schedule have required balance between testing needs and the desire for an integrated flight vehicle as soon as possible. To provide key insights early in design and analysis cycles, a large amount of effort has shifted into maturing and validating models at the component level with integrated testing as a means to validate their integration. In terms of SLS Navigation, this, and the model-based design approach have pushed explicit requirements for sensor models to be validated against flight hardware to high precision. This paper covers the approach taken to verify and validate the models for the two key navigation sensors on the SLS vehicle, the Redundant Inertial Navigation Sensor and the Rate Gyro Assembly. These models are used in performance evaluation, fault detection, and operations development extensively. Using a mix of data from hardware vendor documentation and testing reports, limited in-house testing, and integration activities, these models were able to be validated against flight hardware at multiple levels, from the internal software design to statistical behavior at the raw sensor and integrated box levels. The high level of insight into the hardware elements is instrumental to support flight certification activities and building confidence in SLS Navigation capability. Focused testing enabled additional insight and validation that proved invaluable and the resulting insights were used to focus and mature models. Additionally, of having validated performance-based hardware models enables a wide breadth of activities including detailed fault detection studies and integration into future vehicle frameworks, such as an upper stage and provide a valuable asset to continued SLS analysis and design.

Evan J Anzalone↗

NASA NTRS2020

Hardware Verification and Validation for a Navigation Sensor Software Model in Support of Flight Vehicle Performance Analysis

… or, “It’s in the details, how to make complicated software perform like complicated hardware.” In attempts to minimize development time and quickly build an operational vehicle, NASA’s Space Launch System (SLS) has had to be intentional about integrated testing. Constraints on budget and schedule have required balance between testing needs and the desire for an integrated flight vehicle as soon as possible. To provide key insights early in design and analysis cycles, a large amount of effort has shifted into maturing and validating models at the component level with integrated testing as a means to validate their integration. In terms of SLS Navigation, this, and the model-based design approach have pushed explicit requirements for sensor models to be validated against flight hardware to high precision. This paper covers the approach taken to verify and validate the models for the two key navigation sensors on the SLS vehicle, the Redundant Inertial Navigation Sensor and the Rate Gyro Assembly. These models are used in performance evaluation, fault detection, and operations development extensively. Using a mix of data from hardware vendor documentation and testing reports, limited in-house testing, and integration activities, these models were able to be validated against flight hardware at multiple levels, from the internal software design to statistical behavior at the raw sensor and integrated box levels. The high level of insight into the hardware elements is instrumental to support flight certification activities and building confidence in SLS Navigation capability. Focused testing enabled additional insight and validation that proved invaluable and the resulting insights were used to focus and mature models. Additionally, of having validated performance-based hardware models enables a wide breadth of activities including detailed fault detection studies and integration into future vehicle frameworks, such as an upper stage and provide a valuable asset to continued SLS analysis and design.

Thomas Park↗

NASA NTRS2017

Land Surface Temperature Product Validation Best Practice Protocol Version 1.0 - October, 2017

The Global Climate Observing System (GCOS) has specified the need to systematically generate andvalidate Land Surface Temperature (LST) products. This document provides recommendations on goodpractices for the validation of LST products. Internationally accepted definitions of LST, emissivity andassociated quantities are provided to ensure the compatibility across products and reference data sets. Asurvey of current validation capabilities indicates that progress is being made in terms of up-scaling and insitu measurement methods, but there is insufficient standardization with respect to performing andreporting statistically robust comparisons.Four LST validation approaches are identified: (1) Ground-based validation, which involvescomparisons with LST obtained from ground-based radiance measurements; (2) Scene-based intercomparisonof current satellite LST products with a heritage LST products; (3) Radiance-based validation,which is based on radiative transfer calculations for known atmospheric profiles and land surface emissivity;(4) Time series comparisons, which are particularly useful for detecting problems that can occur during aninstrument's life, e.g. calibration drift or unrealistic outliers due to undetected clouds. Finally, the need foran open access facility for performing LST product validation as well as accessing reference LST datasets isidentified.

best practice↗

NASA NTRS2020

Chrono-Validation of Near-Real-Time Landslide Susceptibility Models via Plugin Statistical Simulations

The idea behind any validation scheme in landslide susceptibility studies is to test whether a model calibrated on a certain data can predict an unknown dataset of the same nature (landslide presences/absences and covariates). Almost the entirety of landslide susceptibility studies are validated by subsetting a single dataset into a training and test sets. This dataset usually corresponds either to event-specific or to historical inventories. Very rarely, a multi-temporal inventory is available and, in the few cases where this condition is met, the validation practices involve training a model on a specific landslide inventory, deriving a single predictive equation and validating it on a subsequent landslide inventory. This commonly leads landslide predictive studies, even those with a strong statistical rigor, to neglect the uncertainty estimation in their modeling scheme. In statistics, validation can also be performed via statistical simulations. This means that after fitting a given model, one can generate any number of predictive functions and test their predictive skills on any type and number of unknown datasets. In this work, we take a similar direction and we apply it to model and validate three separate co-seismic inventories, including an uncertainty estimation phase. We mapped these inventories within the same area in Indonesia, for three earthquakes occurred in 2012, 2017 and 2018. Specifically, we build three event-specific Bayesian Generalize Additive Models of the binomial family. From each model we then simulate 1000 predictive realizations over the remaining two inventories, by using a plug-in scheme where all the morphometric covariates are kept fixed and only the ground motion is replaced according to the prediction target. By doing so, we introduce a new analytical tool for near-real-time landslide predictive purposes, which is able to produce a probabilistic model which stands in between the definitions of susceptibility and hazard. In fact, our model is able to accurately estimate “where” and “when” - although not “how frequently” - landslide have occurred by featuring the multitemporal information of the trigger. In our findings, the simulations are quite similar to the fitted models; and the nine combinations we analyse produce excellent performance. This result confirms the assumption that “the past is the key to the future”, as we show that the relative contribution of each variable and their interactions in each probabilistic model remains practically the same across temporal replicates. This information is not trivial because it supports the routines implemented in global near-real-time applications.

Temporal validation↗

NASA NTRSDate not supplied

Validation of Rendered Natural and Artificial Lighting Environments in Real Time Lunar South Pole Simulations

NASA’s Artemis campaign is making heavy use of simulation to help return humans to the lunar surface by the end of the decade. There are several aspects of the lunar surface and its environment which must be accurately modeled before these simulations can be relied upon to influence decisions being made under these programs. Digital Lunar Exploration Sites, a paper submitted to the 2022 IEEE Aerospace Conference, outlined the process used to generate the lunar surface in a digital environment. This paper will expand upon this topic and delve into the steps being taken by the NASA Exploration Systems Simulations (NExSyS) team at NASA’s Johnson Space Center (JSC) to properly verify and validate these simulations, with a focus on the visual aspects of the environment. Natural lighting validation relies in part on the wealth of data generated during the Apollo program. Many images taken by Apollo astronauts on the lunar surface have been replicated in the simulated environments to gain confidence in the accuracy of terrain and lighting models. However, because the environment the Artemis astronauts will experience at the Lunar South Pole (LSP) is dissimilar from the near-equatorial Apollo sites, other validation techniques must be applied. At the LSP, the sun crests only about three degrees above the horizon and when combined with the lack of a lunar atmosphere, lighting in this region is often very different than what a human would experience on Earth. Solar illumination, earthshine, human eye response, solar blooming, lunar regolith optical properties, and shadows cast by rocks and crater walls will play a significant role in an astronaut’s ability to safely conduct an Extra-Vehicular Activity (EVA) or perform a traverse with a lunar rover. Approaches for validation of these aspects of the rendered LSP environment are considered in this paper. In addition to natural lighting, approaches for the validation of artificial lighting models at the LSP are discussed. The JSC Lighting Lab has been studying the illumination profile of the Exploration Infomatics Subsystem (xINFO) lighting on the Exploration EVA Mobility Unit (xEMU). How these lights interact with the solar illumination and the shadows being cast on the lunar surface is of particular interest, so the validity of models representing these lights in a human-in-the-loop virtual reality environment becomes very important. This paper also touches on some of the simulation performance considerations when a Human in the Loop (HITL) is present, which drives the need for real time rendering of the environment. Natural and artificial lighting will play a crucial role to decisions being made when planning and executing missions at the Lunar South Pole (LSP) and it is vitally important to understand the LSP environment before we return.

Lunar↗

NASA NTRS2023

Approaches for Validation of Lighting Environments in Realtime Lunar South Pole Simulations

NASA’s Artemis campaign is making heavy use of simulation to help return humans to the lunar surface by the end of the decade. There are several aspects of the lunar surface and its environment which must be accurately modeled before these simulations can be relied upon to influence decisions being made under these programs. Digital Lunar Exploration Sites, a paper submitted to the 2022 IEEE Aerospace Conference, outlined the process used to generate the lunar surface in a digital environment. This paper will expand upon this topic and delve into the steps being taken by the NASA Exploration Systems Simulations (NExSyS) team at NASA’s Johnson Space Center (JSC) to properly verify and validate these simulations, with a focus on the visual aspects of the environment. Natural lighting validation relies in part on the wealth of data generated during the Apollo program. Many images taken by Apollo astronauts on the lunar surface have been replicated in the simulated environments to gain confidence in the accuracy of terrain and lighting models. However, because the environment the Artemis astronauts will experience at the Lunar South Pole (LSP) is dissimilar from the near-equatorial Apollo sites, other validation techniques must be applied. At the LSP, the sun crests only about 1.5 degrees above the horizon and when combined with the lack of a lunar atmosphere, lighting in this region is often very different than what a human would experience on Earth. Solar illumination, earthshine, human eye response, solar blooming, lunar regolith optical properties, and shadows cast by rocks and crater walls will play a significant role in an astronaut’s ability to safely conduct an Extra-Vehicular Activity (EVA) or perform a traverse with a lunar rover. Approaches for validation of these aspects of the rendered LSP environment are considered in this paper. In addition to natural lighting, approaches for the validation of artificial lighting models at the LSP are discussed. The JSC Lighting Lab has been studying the illumination profile of the Exploration Informatics Subsystem (xINFO) lighting on the Exploration EVA Mobility Unit (xEMU). How these lights interact with the solar illumination and the shadows being cast on the lunar surface is of particular interest, so the validity of models representing these lights in a human-in-the-loop virtual reality environment becomes very important. This paper also touches on some of the simulation performance considerations when a Human in the Loop (HITL) is present, which drives the need for realtime rendering of the environment. Natural and artificial lighting will play a crucial role to decisions being made when planning and executing missions at the Lunar South Pole (LSP) and it is vitally important to understand the LSP environment before we return.

Lunar↗

DOE OSTI2026

Connect microstructure evolution to property degradation with validated simulation

Reliable material performance is required for plasma-facing material (PFM) candidates. Previous research has shown that plasma and neutron radiation exposure induces microstructural changes in PFMs; changes in thermal and electrical conductivities and in material hardening and embrittlement were also observed after neutron irradiation. These material property changes will negatively impact the performance of the PFMs in a fusion reactor. Despite the well-known connection between material microstructure, properties, and performance, there is a need for validated modeling capabilities connecting PFM property degradation with microstructural evolution under fusion-relevant conditions. We are developing a simulation capability to couple plasma-induced microstructural evolution to material property degradation. Our approach relies on deliberate mapping between individual simulation models and experimental characterization for validation. The open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) software was used for this simulation capability development. A MOOSE phase-field model was coupled with the cluster dynamics code, Xolotl, to predict microstructural evolution. Microstructure characterization techniques, including scanning electron microscopy (SEM), transmission electron microscopy (TEM), and laser scanning confocal microscopy (LSCM) are used to validate these microstructural evolution simulations. Calculation of thermal and electrical conductivities with first principles simulations was performed for bulk material and for grain boundaries; these results are used within MOOSE models to calculate effective thermal and electrical conductivities as a function of grain characteristics. Thermoreflectance and four-probe techniques were employed to measure the thermal and electrical conductivities, respectively. A MOOSE crystal plasticity model was adapted to predict microstructure-sensitive deformation behavior, and X-ray diffraction (XRD) was used to collect bulk dislocation density data for validation. After individual simulation validation, these models are coupled to predict material property changes resulting from plasma exposure. We focused here on an experimental design to emphasize the separate effects of moderate thermal loads and plasma exposure using tungsten. Annealing of tungsten was performed under a protective environment for temperatures ranging from 500$^o$C to 1500$^o$C. The plasma exposure was completed in the Tritium Plasma Experiment at Idaho National Laboratory under a deuterium flux of 1e22 $\frac{D}{m^2s}$. This incremental approach is employed to build confidence in the modeling capability: separate-effects tests ensure that the models capture key mechanisms from single environmental conditions before predicting PFM property degradation under combined loads. We will show our early results from coupling these simulation models to predict PFM property changes from microstructural evolution. Comparisons of the simulation results with preliminary validation data will be discussed.

36 - MATERIALS SCIENCE↗

DOE OSTI2024

SCALE 6.3 Validation: Radiation Shielding

Safe and reliable use of scientific and engineering computer codes requires validation for the types of applications in which they will be used. An example in the nuclear reactor engineering and licensing field is radiation transport employed in shielding analyses. The validity of computer codes for shielding applications is demonstrated in this report for SCALE version 6.3.0. Representative benchmarks corresponding to shielding analyses are selected for the validation study. Typical measurement results analyzed from these benchmarks include neutron fluxes, detector count rates, detector energy response functions, neutron and gamma dose rates, neutron activation rates and activities, neutron leakage fluxes, and skyshine dose rates. Thousands of points of comparison between measurement and calculation are presented in this work. Other than rare outliers typically explained by either a lack of information or large uncertainties in the experiment conditions, material, or dimensions, the Monaco with Automated Variance Reduction using Importance Calculations (MAVRIC) radiation transport computer code with built-in variance reduction methods distributed with the SCALE computer code system agrees well with the measurement results. In selected benchmarks, MAVRIC is also compared to Monte Carlo N- Particle® (MCNP® ) 1 calculations. Both computer codes generally agree well within the estimated uncertainties. With the release of SCALE 6.3.0, Shift was integrated as an alternative transport solver in MAVRIC, denoted MAVRIC-Shift. Although the traditional MAVRIC using Monaco was used primarily in this validation study, many results have also been generated using MAVRIC-Shift. Agreement between MAVRIC-Monaco and MAVRIC-Shift is generally very good. The benchmarks presented in this report were obtained from reliable sources such as the International Criticality Safety Benchmark Evaluation Project Handbook, the Shielding Integral Benchmark Archive & Database, and other shielding validation work found in the literature. Additional datapoints and benchmarks will be added to future versions of this report to expand the shielding validation suite.

61 RADIATION PROTECTION AND DOSIMETRY↗

DOE OSTI2024

Validation Exercise of a Coarse Finite Element Model of Laser Welds

The objective of this project is to validate low-fidelity models of 304L to 304L stainless steel partial-penetration laser welds for thin sheets. Low-fidelity means that the weld is represented by coarsely meshed element blocks. Here, the hexahedral element size is approx imately half the weld penetration depth. The material behavior of the block is represented by a J2 plasticity model with a Voce hardening function. The source of the data used in this work is an extensive experimental study conducted by Sharlotte Kramer (1528) and published in 2015. Figure 1 shows a cross-section of the weld of interest. The nominal thickness of the sheets is 0.063 in. while the target penetration depth of the weld is in the range of 0.028 to 0.032 in., extending about half the sheet thickness. Uniaxial tension tests provided data for calibration of base material and weld models. Results of two validation geometries were also provided. The principal validation geometry is shown in Fig. 2. It consists of a plate specimen with in-plane dimensions 6 in × 2.875 in loaded in tension. A circular plug with a 1.5 in. diameter was cut from the center of the plate and then welded in place. The details of the welding schedule are given. An important assumption is that the welds in the calibration and validation specimens have similar geometric and material properties as those in the validation tests. The task was to first calibrate models for the base material and the welds and then simulate the validation tests until the point of weld first failure.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Software Validation Work With The ZPPR-15 Data

The analysis activities for fast reactors involve using many different pieces of software that are relied upon for their predictive capabilities. For this software to be considered reliable, documented proof that the predictions of the software are accurate is required. In this manuscript, the validation work that covers some of the Argonne software used in fast reactor design activities is discussed and displayed. This validation work includes neutron and gamma flux distributions, reaction rate distributions, and reactivity worth. In an ideal world, a reactor development program would have access to a comprehensive set of experimental facilities to help inform the design aspects of the reactor itself. While thermal-hydraulics experiments, and to a limited degree mechanical experiments, can be carried out today for validation needs, neutronics related experimental facilities are rather impractical because of the lack of experimental facilities. Given the desired time table for construction of new reactors, the reconstitution or creation of new neutronic experimental facilities is untenable and thus those reactor development programs must rely upon any available experimental measurements that are qualitatively similar to the design. While a methodology has been proposed to assess the similarity between the past experimental measurements and the reactor itself, that aspect is beyond the scope of this manuscript. In this manuscript, the focus is entirely placed on the analysis results for a series of experiments carried out at the ZPPR facility in Idaho in the mid-1980s. In this regard, this manuscript only shows the validation of the stated neutronics software for specific loadings of the ZPPR reactor. Because of the fuel form, its proposed enrichment, and the material content of the reactor core, the ZPPR-15 experiments were identified as potential validation data for the reactor. The ZPPR-15 experiments were intended as mockups of a 330 MWe Integral Fast Reactor program which was a follow on program to the Clinch River Breeder Reactor. In the ZPPR-15 series of experiments, measurements of the neutron spectrum, control rod worth, sodium void worth, foil reaction rate distributions, Doppler worth of heated samples, gamma dose, and axial expansion worth were all carried out and published. In many cases, these reactivity coefficients are good candidates to validate the reactivity coefficient calculation scheme used by the analysis software and included in the safety analysis activities of fast reactor development projects today. This manuscript discusses the modeling methodology and accuracy of the calculated experimental results using the LANL software MCNP and the ANL software package ARC (Argonne Reactor Codes). As will be shown, for many of the experimental measurements, the two software packages are found to be good predictive analysis tools for those experiments. In other cases, problems with the analysis methodology or underlying cross section data are exposed which indicates where predictive analysis is not as reliable. Finally, in some of the measurements the conclusion is reached that the experimental measurement cannot be reproduced with the analysis software as it is simply too difficult.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2024

Selection and Ranking of Experiments from the Halden Database in support of Multiscale Model Validation

Validating fuel performance codes, such as BISON, requires an extensive amount of experiments covering a wide range of operating conditions and fuel types. Within the light-water reactor (LWR) space, there have been several international experimental programs that have contributed to the wealth of available experimental data available for use. One of those international programs, the Halden Reactor Project (HRP), began in 1958 and utilized the Halden Boiling Water Reactor to conduct many highly instrumented experiments until the reactor closed in 2018. Idaho National Laboratory, through the U.S. Department of Energy, has utilized several Halden experiments to perform the initial validation of the BISON code based upon their inclusion in international modeling and simulation benchmarks. Recently, the HRP has provided member organizations a complete copy of all available data, reports, and presentations since the HRP began. This report provides an initial exploration of the data available in the database for use in validating the multiscale models under development in the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program for LWR applications. Ranking tables that identify potential validation cases are provided for the high priority models of interest. It was found that some of the recommended high priority experiments correspond to additional rods in existing assemblies already available in the BISON validation suite. It is expected that several of these cases will be incorporated into future NEAMS milestones in the fuels technical area for increased validation of BISON.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2026

Software Validation Work With The ZPPR-15 Data

The analysis activities for fast reactors involve using many different pieces of software that are relied upon for their predictive capabilities. For this software to be considered reliable, documented proof that the predictions of the software are accurate is required. In this manuscript, the validation work that covers some of the Argonne software used in fast reactor design activities is discussed and displayed. This validation work includes neutron and gamma flux distributions, reaction rate distributions, and reactivity worth. In an ideal world, a reactor development program would have access to a comprehensive set of experimental facilities to help inform the design aspects of the reactor itself. While thermal-hydraulics experiments, and to a limited degree mechanical experiments, can be carried out today for validation needs, neutronics related experimental facilities are rather impractical because of the lack of experimental facilities. Given the desired time table for construction of new reactors, the reconstitution or creation of new neutronic experimental facilities is untenable and thus those reactor development programs must rely upon any available experimental measurements that are qualitatively similar to the design. While a methodology has been proposed to assess the similarity between the past experimental measurements and the reactor itself, that aspect is beyond the scope of this manuscript. In this manuscript, the focus is entirely placed on the analysis results for a series of experiments carried out at the ZPPR facility in Idaho in the mid-1980s. In this regard, this manuscript only shows the validation of the stated neutronics software for specific loadings of the ZPPR reactor. Because of the fuel form, its proposed enrichment, and the material content of the reactor core, the ZPPR-15 experiments were identified as potential validation data for the reactor. The ZPPR-15 experiments were intended as mockups of a 330 MWe Integral Fast Reactor program which was a follow on program to the Clinch River Breeder Reactor. In the ZPPR-15 series of experiments, measurements of the neutron spectrum, control rod worth, sodium void worth, foil reaction rate distributions, Doppler worth of heated samples, gamma dose, and axial expansion worth were all carried out and published. In many cases, these reactivity coefficients are good candidates to validate the reactivity coefficient calculation scheme used by the analysis software and included in the safety analysis activities of fast reactor development projects today. This manuscript discusses the modeling methodology and accuracy of the calculated experimental results using the LANL software MCNP and the ANL software package ARC (Argonne Reactor Codes). As will be shown, for many of the experimental measurements, the two software packages are found to be good predictive analysis tools for those experiments. In other cases, problems with the analysis methodology or underlying cross section data are exposed which indicates where predictive analysis is not as reliable. Finally, in some of the measurements the conclusion is reached that the experimental measurement cannot be reproduced with the analysis software as it is simply too difficult.

Aliberti, Gerardo↗

DOE OSTI2025

Guiding Principles for Geochemical/Thermodynamic Model Development and Validation in Nuclear Waste Disposal: A Close Examination of Recent Thermodynamic Models for H + —Nd 3+ —NO 3 - (—Oxalate) Systems

Development of a defensible source-term model (STM), usually a thermodynamical model for radionuclide solubility calculations, is critical to a performance assessment (PA) of a geologic repository for nuclear waste disposal. Such a model is generally subjected to rigorous regulatory scrutiny. In this article, we highlight key guiding principles for STM model development and validation in nuclear waste management. We illustrate these principles by closely examining three recently developed thermodynamic models with the Pitzer formulism for aqueous H + —Nd 3+ —NO 3 - (—oxalate) systems in a reverse alphabetical order of the authors: the XW model developed by Xiong and Wang, the OWC model developed by Oakes et al., and the GLC model developed by Guignot et al., among which the XW model deals with trace activity coefficients for Nd(III), while the OWC and GLC models are for concentrated Nd(NO 3 ) 3 electrolyte solutions. The principles highlighted include the following: (1) Principle 1. Validation against independent experimental data: A model should be validated against experimental data or field observations that have not been used in the original model parameterization. We tested the XW model against multiple independent experimental data sets including electromotive force (EMF), solubility, water vapor, and water activity measurements. The results show that the XW model is accurate and valid for its intended use for predicting trace activity coefficients and therefore Nd solubility in repository environments. (2) Principle 2. Testing for relevant and sensitive variables: Solution pH is such a variable for an STM and easily acquirable. All three models are checked for their ability to predict pH conditions in Nd(NO 3 ) 3 electrolyte solutions. The OWC model fails to provide a reasonable estimate for solution pH conditions, thus casting serious doubt on its validity for a source-term calculation. In contrast, both the XW and GLC models predict close-to-neutral pH values, in agreement with experimental measurements. (3) Principle 3. Honoring physical constraints: Upon close examination, it is found that the Nd(III)-NO 3 association schema in the OWC model suffers from two shortcomings. Firstly, its second stepwise stability constant for Nd(NO 3 ) 2+ (log K 2 ) is much higher than the first stepwise stability constant for NdNO 3 2+ (log K 1 ), thus violating the general rule of (log K 2 –log K 1 ) < 0, or $\frac{K1}{K2}$>1. Secondly, the OWC model predicts abnormally high activity coefficients for Nd(NO 3 ) 2 + (up to ~900) as the concentration increases. (4) Principle 4. Minimizing degrees of freedom for model fitting: The OWC model with nine fitted parameters is compared with the GLC model with five fitted parameters, as both models apply to the concentrated region for Nd(NO 3 ) 3 electrolyte solutions. The latter appears superior to the former because the latter can fit osmotic coefficient data equally well with fewer model parameters. The work presented here thus illustrates the salient points of geochemical model development, selection, and validation in nuclear waste management.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

NASA NTRS1991

Display format, highlight validity, and highlight method: Their effects on search performance

Display format and highlight validity were shown to affect visual display search performance; however, these studies were conducted on small, artificial displays of alphanumeric stimuli. A study manipulating these variables was conducted using realistic, complex Space Shuttle information displays. A 2x2x3 within-subjects analysis of variance found that search times were faster for items in reformatted displays than for current displays. Responses to valid applications of highlight were significantly faster than responses to non or invalidly highlighted applications. The significant format by highlight validity interaction showed that there was little difference in response time to both current and reformatted displays when the highlight validity was applied; however, under the non or invalid highlight conditions, search times were faster with reformatted displays. A separate within-subject analysis of variance of display format, highlight validity, and several highlight methods did not reveal a main effect of highlight method. In addition, observed display search times were compared to search time predicted by Tullis' Display Analysis Program. Benefits of highlighting and reformatting displays to enhance search and the necessity to consider highlight validity and format characteristics in tandem for predicting search performance are discussed.

Donner, Kimberly A.↗

NASA NTRS1992

Experimental validation of structural optimization methods

The topic of validating structural optimization methods by use of experimental results is addressed. The need for validating the methods as a way of effecting a greater and an accelerated acceptance of formal optimization methods by practicing engineering designers is described. The range of validation strategies is defined which includes comparison of optimization results with more traditional design approaches, establishing the accuracy of analyses used, and finally experimental validation of the optimization results. Examples of the use of experimental results to validate optimization techniques are described. The examples include experimental validation of the following: optimum design of a trussed beam; combined control-structure design of a cable-supported beam simulating an actively controlled space structure; minimum weight design of a beam with frequency constraints; minimization of the vibration response of helicopter rotor blade; minimum weight design of a turbine blade disk; aeroelastic optimization of an aircraft vertical fin; airfoil shape optimization for drag minimization; optimization of the shape of a hole in a plate for stress minimization; optimization to minimize beam dynamic response; and structural optimization of a low vibration helicopter rotor.

Adelman, Howard M.↗