Search NASA⌕ Search

SEARCH · Search NASA

Results for “Validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Experience with Aero- and Fluid-Dynamic Testing for Engineering and CFD Validation

Ever since computations have been used to simulate aerodynamics the need to ensure that the computations adequately represent real life has followed. Many experiments have been performed specifically for validation and as computational methods have improved, so have the validation experiments. Validation is also a moving target because computational methods improve requiring validation for the new aspect of flow physics that the computations aim to capture. Concurrently, new measurement techniques are being developed that can help capture more detailed flow features pressure sensitive paint (PSP) and particle image velocimetry (PIV) come to mind. This paper will present various wind-tunnel tests the author has been involved with and how they were used for validation of various kinds of CFD. A particular focus is the application of advanced measurement techniques to flow fields (and geometries) that had proven to be difficult to predict computationally. Many of these difficult flow problems arose from engineering and development problems that needed to be solved for a particular vehicle or research program. In some cases the experiments required to solve the engineering problems were refined to provide valuable CFD validation data in addition to the primary engineering data. All of these experiments have provided physical insight and validation data for a wide range of aerodynamic and acoustic phenomena for vehicles ranging from tractor-trailers to crewed spacecraft.

experiment↗
NASA NTRSDate not supplied

AVT-297 Development of a Framework for Validation of Computational Tools

Design of aerospace vehicles relies on system models. These models rely on validation data to ensure accurate representation of the system. This paper provides an introduction to the problem and an overview of a NATO STO Activity, AVT-297, which developed a framework to determine validation experiments required to support system level models. The goals of the research group were to develop a process to identify a validation database for intended uses of a vehicle system, demonstrate the process on vehicles and document the process. Applying this process to multiple intended uses and collecting this set of validation databases identifies a set of validation experiments that could be applied to the design of multiple vehicles, further reducing the cost and time to develop air and naval vehicles. Four subgroup teams focused on different aspects of the validation database process: two focused on different aspects of a missile, and two focused on different aspects of a fixed-wing mobility transport class vehicle. This resulted in three different processes to decompose the multi-disciplinary problem to identify critical validation data. All four of the teams demonstrated their process on a sample problem demonstrating the benefit of the process.

Validation↗
DOE OSTI2017

Achieving reasonable confidence in the validation of control room designs and modifications: A summary of the 2015 NEA experts' workshop - 124

Defining and achieving reasonable confidence in the validation of nuclear power plant control room designs is a complex challenge for many reasons; not the least of which is that validation is a test of strength rather than a test of truth. In 2015, the Nuclear Energy Agency, Committee for Safety of Nuclear Installations' Working Group on Human and Organizational Factors (WGHOF) hosted an experts' workshop to identify and critically examine potential means to address the challenge of achieving reasonable confidence in control room design validations. The experts' principal task was to answer four challenge questions. (1) What are the critical considerations in defining validation objectives and how do these impact achieving reasonable confidence? (2) What methods, approaches, resources, or rationales might be used for deriving performance requirements, selecting measures, and establishing acceptance criteria so as to support reasonable confidence? (3) What methods might be used to develop scenarios that maximize the amount and relevance of information in support of the validation conclusions and achieving reasonable confidence? (4) How should the validation results be aggregated and analyzed to determine the final validation conclusions? This paper provides an overview of the experts' responses to these challenges and briefly touches upon their implications for future directions. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2015

Validation of Scale 6.2 criticality calculations using KENO V.A and KENO-VI - 14603

SCALE 6.2 provides enhancements for criticality calculations relative to SCALE 6.1, especially with updated nuclear data libraries, resonance self-shielding options for multigroup (MG) calculations, and problem-dependent temperature treatments for continuous-energy (CE) calculations. Benchmark experiments from the International Criticality Safety Benchmark Evaluation Project that are available in the Oak Ridge National Laboratory Verified, Archived, Library of Inputs and Data (VALID) were applied in this validation effort to examine the impact of the SCALE 6.2 enhancements. Over 300 cases are drawn from the VALID library for testing KENO V.a, and a more limited set of approximately 50 configurations is used for validation of KENO-VI. A total of 401 unique critical experiments are used in this validation effort. In some cases, the KENO V.a models were converted to KENO-VI models so that the performance of both codes could be assessed for the same systems. Validation results for the SCALE 6.2 ENDF/B-VII.1 nuclear data libraries are presented, including MG neutron data in 56- and 252-group structures as well as CE data. The performance of all three multipurpose neutron libraries in calculating k{sub eff} is examined with critical experiment models covering a wide range of fuels, moderators, reflectors, absorbers, and neutron spectra. The SCALE 6.2 ENDF/B-VII.1 coupled neutron-photon library, primarily developed for shielding applications with 200 neutron groups and 47 gamma groups, provides additional energy resolution for fast systems relative to the MG neutron libraries which were developed to produce low biases for thermal neutron systems. The applicability of this library to criticality calculations for fast neutron spectrum systems is not investigated here but will be addressed in the final SCALE 6.2 validation report. The results of these calculations demonstrate that SCALE 6.2 generally provides lower biases than SCALE 6.1 and that the two versions of KENO provide equivalent results on the same suite of problems. SCALE 6.2 also includes an updated neutron cross-section covariance library based on the ENDF/B-VII.1 covariance data. The effect of these updated covariances on the uncertainty in k{sub eff} due to cross section uncertainties is also presented for some cases. A more thorough investigation of the updated covariance data is presented in a companion paper. (authors)

42 ENGINEERING↗
DOE OSTI2021

An Application for Validation of Power Distribution System Models in an ADMS Environment

An accurate model of a power distribution system is the foundation for model-based applications that ensure efficient and reliable grid operation in an advanced distribution management system (ADMS) environment. However, these models are error-prone and comprehensive model validation is challenging due to lack of standards-based systems, data originating from disparate databases and other sources, and the constantly evolving nature of modern power distribution systems. In this paper, a novel framework for comprehensive model validation is described. The proposed application, the Model Validator, ensures that a model is both consistent and feasible by validating the derivative static and operational network model. A modular architecture for the application has been implemented and integrated with an open-source standards-based platform for ADMS application development, GridAPPS-D, allowing new validation capability to be added with minimal time and effort. The Model Validator application is demonstrated on the IEEE 13-bus, 123-bus, and 8500-node test cases over three validation scenarios.

Poudel, Shiva↗
DOE OSTI2020

Certification Validation Libraries

“The code “Certificate Validation Libraries” provides the ability to validate x509 certificates in python using a simple library. These libraries validate certificates in 3 ways: 1) validation that certificate has not expired. 2) validation that certificate has not been revoked and 3) validation of certificate's digital signature. The benefits of this code is that one can properly validate x509 certificates, thus strengthening the security of systems using these libraries.”

Maerz, StefanA↗
DOE OSTI2016

Modeling and Validation of Sodium Plugging for Heat Exchangers in Sodium-cooled Fast Reactor Systems

The project “Modeling and Validation of Sodium Plugging for Heat Exchangers in Sodium-cooled Fast Reactor Systems” was conducted jointly by Westinghouse Electric Company (Westinghouse) and Argonne National Laboratory (ANL), over the period October 1, 2013- March 31, 2016. The project’s motivation was the need to provide designers of Sodium Fast Reactors (SFRs) with a validated, state-of-the-art computational tool for the prediction of sodium oxide (Na 2 O) deposition in small-diameter sodium heat exchanger (HX) channels, such as those in the diffusion bonded HXs proposed for SFRs coupled with a supercritical CO 2 (sCO 2 ) Brayton cycle power conversion system. In SFRs, Na 2 O deposition can potentially occur following accidental air ingress in the intermediate heat transport system (IHTS) sodium and simultaneous failure of the IHTS sodium cold trap. In this scenario, oxygen can travel through the IHTS loop and reach the coldest regions, represented by the cold end of the sodium channels of the HXs, where Na 2 O precipitation may initiate and continue. In addition to deteriorating HX heat transfer and pressure drop performance, Na 2 O deposition can lead to channel plugging especially when the size of the sodium channels is small, which is the case for diffusion bonded HXs whose sodium channel hydraulic diameter is generally below 5 mm. Sodium oxide melts at a high temperature well above the sodium melting temperature such that removal of a solid plug such as through dissolution by pure sodium could take a lengthy time. The Sodium Plugging Phenomena Loop (SPPL) was developed at ANL, prior to this project, for investigating Na 2 O deposition phenomena within sodium channels that are prototypical of the diffusion bonded HX channels envisioned for SFR-sCO 2 systems. In this project, a Computational Fluid Dynamic (CFD) model capable of simulating the thermal-hydraulics of the SPPL test section and provided with Na 2 O deposition prediction capabilities, was developed. This state-of-the-art computational tool incorporates a first-principles Na 2 O deposition model developed by ANL, and combines it with predictive capabilities for the spatial and temporal variation of temperature, velocity, dissolved oxygen concentration, and wall temperature under flowing sodium conditions. The CFD model was validated under no-deposition conditions using experimental data collected with the SPPL, demonstrating the model’s capability to predict the thermal-hydraulics of the SPPL test section within the measurement uncertainty characterizing the SPPL instrumentation. The model’s deposition prediction capability was not, however, validated as the SPPL could not be operated under plugging conditions during the project, resulting in the lack of deposition data with adequate pedigree for a CFD model validation. Two novel diagnostic techniques to detect and characterize Na 2 O deposits, i.e. Ultrasonic Time Domain Reflectometry (UTDR) and Potential Drop (PD) techniques, were developed to ultimately assist in the validation effort under plugging conditions, which can be performed once the SPPL becomes operational. This development effort consisted first in demonstrating, analytically and/or computationally, the capability of these techniques to diagnose Na 2 O deposits inside of small channels (particularly the deposit’s thickness), and subsequently in the fabrication and testing of prototypical UTDR and PD instrumentation. The testing, performed on mockups of the SPPL test section, demonstrated the capability of these techniques to detect and characterize material discontinuities like those induced by sodium oxide deposition on stainless steel channel walls. Because of the mentioned impossibility to run the SPPL in a plugging mode, the developed instrumentation could not be tested in-situ, i.e. at the SPPL while deposits are being formed inside of the SPPL test section. Recommended future work includes a possible enhancement in the CFD modeling technique and installation of the developed UTDR and PD instrumentation on the test section, followed by plugging tests to be conducted with the SPPL. The installation of the UTDR and PD diagnostic instrumentation on the SPPL test section will allow collection of Na 2 O deposition data after the onset of deposition to nearly complete channel plugging, which can ultimately be used for the validation of the CFD model.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2019

Validation of RELAP-7 for Forced Convective and Natural Circulation Reactor Flows

New reactor designs such as small modular reactors and newly proposed passive safety systems rely on natural circulation two-phase flow cooling of the reactor core under accident conditions. Without a pump to drive the flow, the circulation rate is a strong function of void fraction. Under these conditions where the flow rate, pressure drop, and heat transfer are all closely coupled through the mixture density, the accurate prediction of void fraction is pivotal. Furthermore, while these passive systems boast a more reliable cooling option, the sensitivity of the system to two-phase flow instabilities is more likely than in forced convective cooling and can result in dangerous density wave or flow excursion conditions. Improvement of one-dimensional system codes and further development of three-dimensional code capability are all important efforts in addressing these issues. The proposed work is a critical step towards providing confidence through validation of the capability of thermal-hydraulic tools to meet the emerging challenges in reactor simulation. Historically, system analysis codes develop a stiffness over time due to fine-tuning of adjustable code parameters that result in a reluctance or inability to adopt improved models. Therefore, a rigorous effort should be invested in preparation for RELAP-7 to identify required modeling improvements of constitutive relations and validation of the code applicability to the wide range of possible reactor flow conditions. Part of this validation involves the uncertainty quantification which is a critical step in the Code Scaling, Applicability, and Uncertainty (CSAU) methodology and Best-Estimate Plus Uncertainty (BEPU) licensing method. BEPU allows recovery of safety margins and potentially optimizing plant performance, resulting in economic benefit for the reactor operators. Beginning with the collection of past experimental data covering prototypic LWR conditions, this document reviews three critical datasets available for the validation of system codes. The experimental methods, procedures, facilities, instrumentation, and supplementary information are consolidated to aid the validation efforts of this work and future projects. These three datasets (824 conditions) are chosen due to their wide acceptance and demonstrated application for code validation. Two experimental datasets from the Nuclear Power Engineering Corporation (NUPEC) span prototypic conditions of Boiling Water Reactors (BWR) and Pressurized Water Reactors (PWR). These datasets are labeled as the BFBT (BWR Full-size Fine-mesh Bundle Test) and the PSBT (PWR Subchannel and Bundle Test) datasets respectively. The third dataset is taken in an internally heated annulus that is scaled based on a BWR subassembly and includes two-phase flows important to post-blowdown analysis. Together, the datasets consider the full range of pressure and mass flux expected in LWR forced convective flows from steady-state to long term cooling conditions. Recognizing the deficiency of available data under low-pressure, low-flow, a new natural circulation dataset consisting of 107 conditions is obtained. With system pressures ranging between 145 and 950 kPa, natural circulation is driven by a heat flux up to 275 kW/m2 in a 3 m long heated region of the annulus test section which is followed by a 2 m long unheated region before entering a condenser. Stable natural circulation flow rates in this facility were measured to be between 182 and 590 kg/m2-s. Pressure and temperature are measured at ten locations in the facility. In addition, measurements of void fraction, interfacial area concentration, and gas velocity are obtained at five axial locations in the annulus test section. The RELAP5-3D code demonstrates good accuracy for void fraction predictions in BFBT and PSBT benchmarks, but shows relatively large discrepancy in the annulus experiments at lower pressures. In two-phase natural circulation, large oscillations are observed in many system parameters during the simulation of 19 experiment cases which were conducted at low pressures, and the model shows good predictive performance with the remaining 88 experiment cases. However, some discrepancies in void fraction and temperature predictions are found which may require improved constitutive modeling under these low-pressure, low-flow condition. The quantified uncertainty in void fraction predictions in three forced convection benchmarks due to uncertainties in boundary conditions and geometries are relatively small, except for a few cases in the annulus experiment. The output uncertainty of the natural circulation model is quantified by considering the measurement error in boundary conditions, and it is shown that the output uncertainty ranges are relatively small for the range of conditions considered. Through this work, a clear pathway is provided for validation of RELAP7 once the code is available.

42 ENGINEERING↗
NASA NTRS1999

EOS Terra Validation Program

The EOS Terra mission will be launched in July 1999. This mission has great relevance to the atmospheric radiation community and global change issues. Terra instruments include ASTER, CERES, MISR, MODIS and MOPITT. In addition to the fundamental radiance data sets, numerous global science data products will be generated, including various Earth radiation budget, cloud and aerosol parameters, as well as land surface, terrestrial ecology, ocean color, and atmospheric chemistry parameters. Significant investments have been made in on-board calibration to ensure the quality of the radiance observations. A key component of the Terra mission is the validation of the science data products. This is essential for a mission focused on global change issues and the underlying processes. The Terra algorithms have been subject to extensive pre-launch testing with field data whenever possible. Intensive efforts will be made to validate the Terra data products after launch. These include validation of instrument calibration (vicarious calibration) experiments, instrument and cross-platform comparisons, routine collection of high quality correlative data from ground-based networks, such as AERONET, and intensive sites, such as the SGP ARM site, as well as a variety field experiments, cruises, etc. Airborne simulator instruments have been developed for the field experiment and underflight activities including the MODIS Airborne Simulator (MAS), AirMISR, MASTER (MODIS-ASTER), and MOPITT-A. All are integrated on the NASA ER-2, though low altitude platforms are more typically used for MASTER. MATR is an additional sensor used for MOPITT algorithm development and validation. The intensive validation activities planned for the first year of the Terra mission will be described with emphasis on derived geophysical parameters of most relevance to the atmospheric radiation community. Detailed information about the EOS Terra validation Program can be found on the EOS Validation program homepage i/e.: http://ospso.gsfc.nasa.gov/validation/valpage.html).

Starr, David↗
NASA NTRS2003

Validation and Error Characterization for the Global Precipitation Measurement

The Global Precipitation Measurement (GPM) is an international effort to increase scientific knowledge on the global water cycle with specific goals of improving the understanding and the predictions of climate, weather, and hydrology. These goals will be achieved through several satellites specifically dedicated to GPM along with the integration of numerous meteorological satellite data streams from international and domestic partners. The GPM effort is led by the National Aeronautics and Space Administration (NASA) of the United States and the National Space Development Agency (NASDA) of Japan. In addition to the spaceborne assets, international and domestic partners will provide ground-based resources for validating the satellite observations and retrievals. This paper describes the validation effort of Global Precipitation Measurement to provide quantitative estimates on the errors of the GPM satellite retrievals. The GPM validation approach will build upon the research experience of the Tropical Rainfall Measuring Mission (TRMM) retrieval comparisons and its validation program. The GPM ground validation program will employ instrumentation, physical infrastructure, and research capabilities at Supersites located in important meteorological regimes of the globe. NASA will provide two Supersites, one in a tropical oceanic and the other in a mid-latitude continental regime. GPM international partners will provide Supersites for other important regimes. Those objectives or regimes not addressed by Supersites will be covered through focused field experiments. This paper describes the specific errors that GPM ground validation will address, quantify, and relate to the GPM satellite physical retrievals. GPM will attempt to identify the source of errors within retrievals including those of instrument calibration, retrieval physical assumptions, and algorithm applicability. With the identification of error sources, improvements will be made to the respective calibration, assumption, or algorithm. The instrumentation and techniques of the Supersites will be discussed. The GPM core satellite, with its dual-frequency radar and conically scanning radiometer, will provide insight into precipitation drop-size distributions and potentially increased measurement capabilities of light rain and snowfall. The ground validation program will include instrumentation and techniques commensurate with these new measurement capabilities.

Bidwell, Steven W.↗
NASA NTRS2014

Observations on CFD Verification and Validation from the AIAA Drag Prediction Workshops

The authors provide observations from the AIAA Drag Prediction Workshops that have spanned over a decade and from a recent validation experiment at NASA Langley. These workshops provide an assessment of the predictive capability of forces and moments, focused on drag, for transonic transports. It is very difficult to manage the consistency of results in a workshop setting to perform verification and validation at the scientific level, but it may be sufficient to assess it at the level of practice. Observations thus far: 1) due to simplifications in the workshop test cases, wind tunnel data are not necessarily the “correct” results that CFD should match, 2) an average of core CFD data are not necessarily a better estimate of the true solution as it is merely an average of other solutions and has many coupled sources of variation, 3) outlier solutions should be investigated and understood, and 4) the DPW series does not have the systematic build up and definition on both the computational and experimental side that is required for detailed verification and validation. Several observations regarding the importance of the grid, effects of physical modeling, benefits of open forums, and guidance for validation experiments are discussed. The increased variation in results when predicting regions of flow separation and increased variation due to interaction effects, e.g., fuselage and horizontal tail, point out the need for validation data sets for these important flow phenomena. Experiences with a recent validation experiment at NASA Langley are included to provide guidance on validation experiments.

Computational Fluid Dynamics↗
NASA NTRS2015

A Vehicle Management End-to-End Testing and Analysis Platform for Validation of Mission and Fault Management Algorithms to Reduce Risk for NASAs Space Launch System

The engineering development of the National Aeronautics and Space Administration's (NASA) new Space Launch System (SLS) requires cross discipline teams with extensive knowledge of launch vehicle subsystems, information theory, and autonomous algorithms dealing with all operations from pre-launch through on orbit operations. The nominal and off-nominal characteristics of SLS's elements and subsystems must be understood and matched with the autonomous algorithm monitoring and mitigation capabilities for accurate control and response to abnormal conditions throughout all vehicle mission flight phases, including precipitating safing actions and crew aborts. This presents a large and complex systems engineering challenge, which is being addressed in part by focusing on the specific subsystems involved in the handling of off-nominal mission and fault tolerance with response management. Using traditional model-based system and software engineering design principles from the Unified Modeling Language (UML) and Systems Modeling Language (SysML), the Mission and Fault Management (M&FM) algorithms for the vehicle are crafted and vetted in Integrated Development Teams (IDTs) composed of multiple development disciplines such as Systems Engineering (SE), Flight Software (FSW), Safety and Mission Assurance (S&MA) and the major subsystems and vehicle elements such as Main Propulsion Systems (MPS), boosters, avionics, Guidance, Navigation, and Control (GNC), Thrust Vector Control (TVC), and liquid engines. These model-based algorithms and their development lifecycle from inception through FSW certification are an important focus of SLS's development effort to further ensure reliable detection and response to off-nominal vehicle states during all phases of vehicle operation from pre-launch through end of flight. To test and validate these M&FM algorithms a dedicated test-bed was developed for full Vehicle Management End-to-End Testing (VMET). For addressing fault management (FM) early in the development lifecycle for the SLS program, NASA formed the M&FM team as part of the Integrated Systems Health Management and Automation Branch under the Spacecraft Vehicle Systems Department at the Marshall Space Flight Center (MSFC). To support the development of the FM algorithms, the VMET developed by the M&FM team provides the ability to integrate the algorithms, perform test cases, and integrate vendor-supplied physics-based launch vehicle (LV) subsystem models. Additionally, the team has developed processes for implementing and validating the M&FM algorithms for concept validation and risk reduction. The flexibility of the VMET capabilities enables thorough testing of the M&FM algorithms by providing configurable suites of both nominal and off-nominal test cases to validate the developed algorithms utilizing actual subsystem models such as MPS, GNC, and others. One of the principal functions of VMET is to validate the M&FM algorithms and substantiate them with performance baselines for each of the target vehicle subsystems in an independent platform exterior to the flight software test and validation processes. In any software development process there is inherent risk in the interpretation and implementation of concepts from requirements and test cases into flight software compounded with potential human errors throughout the development and regression testing lifecycle. Risk reduction is addressed by the M&FM group but in particular by the Analysis Team working with other organizations such as S&MA, Structures and Environments, GNC, Orion, Crew Office, Flight Operations, and Ground Operations by assessing performance of the M&FM algorithms in terms of their ability to reduce Loss of Mission (LOM) and Loss of Crew (LOC) probabilities. In addition, through state machine and diagnostic modeling, analysis efforts investigate a broader suite of failure effects and associated detection and responses to be tested in VMET to ensure reliable failure detection, and confirm responses do not create additional risks or cause undesired states through interactive dynamic effects with other algorithms and systems. VMET further contributes to risk reduction by prototyping and exercising the M&FM algorithms early in their implementation and without any inherent hindrances such as meeting FSW processor scheduling constraints due to their target platform - the ARINC 6535-partitioned Operating System, resource limitations, and other factors related to integration with other subsystems not directly involved with M&FM such as telemetry packing and processing. The baseline plan for use of VMET encompasses testing the original M&FM algorithms coded in the same C++ language and state machine architectural concepts as that used by FSW. This enables the development of performance standards and test cases to characterize the M&FM algorithms and sets a benchmark from which to measure their effectiveness and performance in the exterior FSW development and test processes. This paper is outlined in a systematic fashion analogous to a lifecycle process flow for engineering development of algorithms into software and testing. Section I describes the NASA SLS M&FM context, presenting the current infrastructure, leading principles, methods, and participants. Section II defines the testing philosophy of the M&FM algorithms as related to VMET followed by section III, which presents the modeling methods of the algorithms to be tested and validated in VMET. Its details are then further presented in section IV followed by Section V presenting integration, test status, and state analysis. Finally, section VI addresses the summary and forward directions followed by the appendices presenting relevant information on terminology and documentation.

Trevino, Luis↗
NASA NTRS2020

Hardware Verification and Validation for a Navigation Sensor Software Model in Support of Flight Vehicle Performance Analysis

… or, “It’s in the details, how to make complicated software perform like complicated hardware.” In attempts to minimize development time and quickly build an operational vehicle, NASA’s Space Launch System (SLS) has had to be intentional about integrated testing. Constraints on budget and schedule have required balance between testing needs and the desire for an integrated flight vehicle as soon as possible. To provide key insights early in design and analysis cycles, a large amount of effort has shifted into maturing and validating models at the component level with integrated testing as a means to validate their integration. In terms of SLS Navigation, this, and the model-based design approach have pushed explicit requirements for sensor models to be validated against flight hardware to high precision. This paper covers the approach taken to verify and validate the models for the two key navigation sensors on the SLS vehicle, the Redundant Inertial Navigation Sensor and the Rate Gyro Assembly. These models are used in performance evaluation, fault detection, and operations development extensively. Using a mix of data from hardware vendor documentation and testing reports, limited in-house testing, and integration activities, these models were able to be validated against flight hardware at multiple levels, from the internal software design to statistical behavior at the raw sensor and integrated box levels. The high level of insight into the hardware elements is instrumental to support flight certification activities and building confidence in SLS Navigation capability. Focused testing enabled additional insight and validation that proved invaluable and the resulting insights were used to focus and mature models. Additionally, of having validated performance-based hardware models enables a wide breadth of activities including detailed fault detection studies and integration into future vehicle frameworks, such as an upper stage and provide a valuable asset to continued SLS analysis and design.

Evan J Anzalone↗
NASA NTRS2020

Hardware Verification and Validation for a Navigation Sensor Software Model in Support of Flight Vehicle Performance Analysis

… or, “It’s in the details, how to make complicated software perform like complicated hardware.” In attempts to minimize development time and quickly build an operational vehicle, NASA’s Space Launch System (SLS) has had to be intentional about integrated testing. Constraints on budget and schedule have required balance between testing needs and the desire for an integrated flight vehicle as soon as possible. To provide key insights early in design and analysis cycles, a large amount of effort has shifted into maturing and validating models at the component level with integrated testing as a means to validate their integration. In terms of SLS Navigation, this, and the model-based design approach have pushed explicit requirements for sensor models to be validated against flight hardware to high precision. This paper covers the approach taken to verify and validate the models for the two key navigation sensors on the SLS vehicle, the Redundant Inertial Navigation Sensor and the Rate Gyro Assembly. These models are used in performance evaluation, fault detection, and operations development extensively. Using a mix of data from hardware vendor documentation and testing reports, limited in-house testing, and integration activities, these models were able to be validated against flight hardware at multiple levels, from the internal software design to statistical behavior at the raw sensor and integrated box levels. The high level of insight into the hardware elements is instrumental to support flight certification activities and building confidence in SLS Navigation capability. Focused testing enabled additional insight and validation that proved invaluable and the resulting insights were used to focus and mature models. Additionally, of having validated performance-based hardware models enables a wide breadth of activities including detailed fault detection studies and integration into future vehicle frameworks, such as an upper stage and provide a valuable asset to continued SLS analysis and design.

Thomas Park↗
DOE OSTI2023

Model validation and selection in metabolic flux analysis and flux balance analysis

13C-Metabolic Flux Analysis (13C-MFA) and Flux Balance Analysis (FBA) are widely used to investigate the operation of biochemical networks in both biological and biotechnological research. Both methods use metabolic reaction network models of metabolism operating at steady state so that reaction rates (fluxes) and the levels of metabolic intermediates are constrained to be invariant. They provide estimated (MFA) or predicted (FBA) values of the fluxes through the network in vivo, which cannot be measured directly. These fluxes can shed light on basic biology and have been successfully used to inform metabolic engineering strategies. Several approaches have been taken to test the reliability of estimates and predictions from constraint-based methods and to compare alternative model architectures. Despite advances in other areas of the statistical evaluation of metabolic models, such as the quantification of flux estimate uncertainty, validation and model selection methods have been underappreciated and underexplored. We review the history and state-of-the-art in constraint-based metabolic model validation and model selection. Applications and limitations of the χ 2 -test of goodness-of-fit, the most widely used quantitative validation and selection approach in 13C-MFA, are discussed, and complementary and alternative forms of validation and selection are proposed. A combined model validation and selection framework for 13C-MFA incorporating metabolite pool size information that leverages new developments in the field is presented and advocated for. Finally, we discuss how adopting robust validation and selection procedures can enhance confidence in constraint-based modeling as a whole and ultimately facilitate more widespread use of FBA in biotechnology.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Development, verification, and validation of comprehensive acoustic fluid-structure interaction capabilities in an open-source computational platform

The acoustic fluid-structure interaction (FSI) formulation is a practical numerical approach for the seismic analysis of fluid-filled tanks. However, there are no verification and validation studies reported in the literature that demonstrate the ability of an acoustic FSI numerical model to predict responses important to structural and mechanical design for intense translational and rotational earthquake inputs. Herein, an acoustic FSI formulation is implemented in the open-source Multiphysics Object-Oriented Simulation Environment (MOOSE), and is formally verified and validated using analytical solutions and code-to-code verification, and experimental data, respectively. The analytical solutions are for small amplitude, unidirectional seismic inputs. The code-to-code verification utilizes a previously verified and validated Arbitrary Lagrangian-Eulerian (ALE) numerical model in the commercial finite element code LS-DYNA. The validation studies utilize a comprehensive data set assembled from results of 3D earthquake-simulator tests of a fluid-filled vessel. The acoustic numerical model in MOOSE is verified and validated for hydrodynamic pressures and support reactions except for cases that involve significant convective response. For small amplitude inputs, numerically predicted wave heights match those of the analytical solutions. The numerical model is not verified and validated for wave height calculations under intense 3D seismic inputs. The run times for the acoustic FSI simulations in MOOSE are an order of magnitude, or more, shorter than for the corresponding ALE simulations in LS-DYNA. The utility of the MOOSE acoustic FSI implementation is demonstrated by seismic analysis of a building equipped with a fluid-filled, advanced nuclear reactor.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Validation of SAS4A/SASSYS-1 for predicting steady-state single-phase natural circulation

The validation of system analysis codes for nuclear reactor systems is required for the development and application of these computational tools. Designed as a comprehensive system analysis code for advanced nuclear reactors, SAS4A/SASSYS-1 requires validation of its physics model for capturing single-phase natural circulation behavior. To support the validation of SAS4A/SASSYS-1, high-precision experiments are performed capturing steady-state single-phase natural circulation on a scaled facility with comprehensive instrumentation. In this work, dedicated tests are performed quantifying the critical modeling parameters, and a single-phase natural circulation benchmark dataset is obtained with well-documented uncertainty and comprehensive facility description. The validation is then performed against the dataset examining the capability of SAS4A/SASSYS-1 in simulating steady-state single-phase natural circulation. The experimental facility is modeled in the candidate code. Solution verification is performed using Richardson-extrapolation-based estimators which quantify and restrict numerical errors from discretization. Input uncertainty provided by the benchmark dataset is forward propagated through the candidate code, quantifying the output uncertainty in a Monte Carlo approach. The composition of the output uncertainty is also quantified through a variance-based sensitivity analysis. With the uncertainty quantified for each individual condition, a detailed comparison between the simulation results and experimental data is performed covering the whole dataset. The results show consistent agreement for all primary parameters. The current validation activity provides a valuable benchmark dataset for the validation of system analysis codes in capturing single-phase natural circulation and demonstrates satisfactory prediction capability of SAS4A/SASSYS1 for steady-state single-phase natural circulation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Enhancement of risk informed validation framework for external hazard scenario

In recent years, the U.S. Nuclear Regulatory Commission (USNRC) and the International Atomic Energy Agency (IAEA) have developed methodologies to assess the vulnerabilities of nuclear plants against site specific extreme hazards. In many cases, advanced simulation tools are being considered to simulate multi-physics, multi-scale phenomena and to evaluate vulnerability of nuclear facilities. The credibility of advanced simulation tools is assessed based on a formal verification, validation, and uncertainty quantification procedure. One of the key limitations in validation is the lack of relevant experimental data at system-level. This limitation leads to a decrease in the confidence of system-level risk predictions. Therefore, a robust validation framework is needed to formalize the confidence in predictive capability of advanced simulation results. Additionally, this study enhances the existing risk informed validation methodology, originally proposed by Kwag et al. [1] and Bodda et al. [2], by developing additional attributes and a new set of validation indicies for a complete and wider applicability of the framework. In this manuscript, the methodology to identify the critical path that leads to the system-level failure is illustrated. The overall validation is checked for completeness and consistency by comparing the critical path for both the system-level simulation and experimental models. The applicability of the code for an intended application is represented in terms of various maturity levels and helps in the process of decision making.

42 ENGINEERING↗