Search NASA⌕ Search

SEARCH · Search NASA

Results for “Validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Enhancement of risk informed validation framework for external hazard scenario

In recent years, the U.S. Nuclear Regulatory Commission (USNRC) and the International Atomic Energy Agency (IAEA) have developed methodologies to assess the vulnerabilities of nuclear plants against site specific extreme hazards. In many cases, advanced simulation tools are being considered to simulate multi-physics, multi-scale phenomena and to evaluate vulnerability of nuclear facilities. The credibility of advanced simulation tools is assessed based on a formal verification, validation, and uncertainty quantification procedure. One of the key limitations in validation is the lack of relevant experimental data at system-level. This limitation leads to a decrease in the confidence of system-level risk predictions. Therefore, a robust validation framework is needed to formalize the confidence in predictive capability of advanced simulation results. Additionally, this study enhances the existing risk informed validation methodology, originally proposed by Kwag et al. [1] and Bodda et al. [2], by developing additional attributes and a new set of validation indicies for a complete and wider applicability of the framework. In this manuscript, the methodology to identify the critical path that leads to the system-level failure is illustrated. The overall validation is checked for completeness and consistency by comparing the critical path for both the system-level simulation and experimental models. The applicability of the code for an intended application is represented in terms of various maturity levels and helps in the process of decision making.

42 ENGINEERING↗
DOE OSTI2016

Understanding Dynamic Model Validation of a Wind Turbine Generator and a Wind Power Plant

Regional reliability organizations require power plants to validate the dynamic models that represent them to ensure that power systems studies are performed to the best representation of the components installed. In the process of validating a wind power plant (WPP), one must be cognizant of the parameter settings of the wind turbine generators (WTGs) and the operational settings of the WPP. Validating the dynamic model of a WPP is required to be performed periodically. This is because the control parameters of the WTGs and the other supporting components within a WPP may be modified to comply with new grid codes or upgrades to the WTG controller with new capabilities developed by the turbine manufacturers or requested by the plant owners or operators. The diversity within a WPP affects the way we represent it in a model. Diversity within a WPP may be found in the way the WTGs are controlled, the wind resource, the layout of the WPP (electrical diversity), and the type of WTGs used. Each group of WTGs constitutes a significant portion of the output power of the WPP, and their unique and salient behaviors should be represented individually. The objective of this paper is to illustrate the process of dynamic model validations of WTGs and WPPs, the available data recorded that must be screened before it is used for the dynamic validations, and the assumptions made in the dynamic models of the WTG and WPP that must be understood. Without understanding the correct process, the validations may lead to the wrong representations of the WTG and WPP modeled.

17 WIND ENERGY↗
DOE OSTI2019

Validation of Numerical Models of the Offshore Wind Turbine From the Alpha Ventus Wind Farm Against Full-Scale Measurements Within OC5 Phase III

The main objective of the Offshore Code Comparison Collaboration Continuation, with Correlation (OC5) project is validation of aero-hydro-servo-elastic simulation tools for offshore wind turbines (OWTs) through comparison of simulated results to the response data of physical systems. Phase III of the OC5 project validates OWT models against the measurements recorded on a Senvion 5M wind turbine supported by the OWEC Quattropod from the alpha ventus offshore wind farm. The following operating conditions of the wind turbine were chosen for the validation: (1) Idling below the cut-in wind speed; (2) Rotor-nacelle assembly (RNA) rotation maneuver below the cut-in wind speed; (3) Power production below and above the rated wind speed; and (4) Shutdown. A number of validation load cases were defined based on these operating conditions. The following measurements were used for validation: (1) Strains and accelerations recorded on the support structure; (2) Pitch, yaw, and azimuth angles, generator speed, and electrical power recorded from the RNA. Strains were not directly available from the majority of the OWT simulation tools. Therefore, strains were calculated based on out-of-plane bending moments, axial forces, and cross-sectional properties of the structural members. Also, a number of issues arose during the validation: (1) The need for a thorough quality check of sensor measurements; (2) The sensitivity of the turbine loads to the controller and airfoil properties, which were only approximated in the modeling approach; (3) The importance of estimating and applying an appropriate damping value for the structure; and (4) The importance of wind characteristics beyond turbulence on the loads. The simulation results and measurements were compared in terms of time series, discrete Fourier transforms, power spectral densities, probability density functions of strains and accelerometers. A good match was achieved between the measurements and models set up by OC5 Phase III participants.

17 WIND ENERGY↗
DOE OSTI2017

Electron Temperature Fluctuation Measurements and Transport Model Validation at Alcator C-Mod

The tokamak is a type of toroidal device used to confine a fusion plasma using large magnetic fields. Tokamaks and stellarators the leading devices for confining plasmas for fusion, and the capability to predict performance in these magnetically confined plasmas is essential for developing a sustainable fusion energy source. The magnetic configuration of tokamaks and stellarators does not exist in Nature, yet, the fundamental processes governing transport in fusion plasmas are universal – turbulence and instabilities, driven by inhomogeneity and asymmetry in the plasma, conspire to transport heat and particles across magnetic field lines and can play critical roles in impurity confinement and generation of intrinsic rotation. Turbulence exists in all plasmas, and in neutral fluids as well. The study of turbulence is essential to developing a fundamental understanding of the nature of the fourth state of matter, plasmas. Experimental studies of turbulence in tokamaks date back to early scattering observations from the late 1970s. Since that time, great advances in turbulence diagnostics have been made, all of which have significantly enhanced our knowledge and understanding of turbulence in tokamaks. Through comparisons with advanced gyrokinetic theory and turbulent-transport models a great deal of evidence exists to implicate turbulent-driven transport as an important mechanism determining transport in all channels: heat, particle and momentum However, prediction and control of turbulent-driven transport remains elusive. Key to development of predictive transport models for magnetically confined fusion plasmas is validation of the nonlinear gyrokinetic transport model, which describes transport due to turbulence. Validation of gyrokinetic codes must include detailed and quantitative comparisons with measured turbulence characteristics, in addition to comparisons with inferred transport levels and equilibrium profiles. For this reason, advanced plasma diagnostics for studying core turbulence are needed in order to assess the accuracy of gyrokinetic models for turbulent-driven particle, heat and momentum transport. New core turbulence diagnostics at the world-class tokamaks Alcator C-Mod at MIT and ASDEX Upgrade at the Max Planck Institute for Plasma Physics have been designed, developed, and operated over the course of this project. These new instruments are capable of measuring electron temperature fluctuations and the phase angle between density and temperature fluctuations locally and quantitatively. These new data sets from Alcator C-Mod and ASDEX Upgrade are being used to fill key gaps in our understanding of turbulent transport in tokamaks. In particular, this project has results in new results on the topics of the Transport Shortfall, the role of ETG turbulence in tokamak plasmas, profile stiffness, the LOC/SOC transition, and intrinsic rotation reversals. These data are used in a rigorous process of “Transport model validation”, and this group is a world-leader on using turbulence models to design new hardware and new experiments at tokamaks. A correlation electron cyclotron emission (CECE) diagnostic is an instrument used to measure micro-scale fluctuations (mm-scale, compared to the machine size of meters) of electron temperature in magnetically confined fusion plasmas, such as those in tokamaks and stellarators. These micro-scale fluctuations are associated with drift-wave type turbulence, which leads to enhanced cooling and mixing of particles in fusion plasmas and limits achieving the required temperatures and densities for self-sustained fusion reactions. A CECE system can also be coupled with a reflectometer system that measured micro-scale density fluctuations, and from these simultaneous measurements, one can extract the phase between the density (n) and temperature (T) fluctuations, creating an nT phase diagnostic. Measurements of the fluctuations and the phase angle between them are extremely useful for testing and validating predictive models for the transport of heat and particles in fusion plasmas due to turbulence. Once validated, the models are used to predict performance in ITER and other burning plasmas, such as the MIT ARC design. Most recently, data from the newly developed, so-called “CECE diagnostic” [Cima 1995, White 2008] and “nT phase angle measurements” [Haese 1999, White 2010] ]will be combined with data from density fluctuation diagnostics at ASDEX Upgrade to support a long-term program of physics research in turbulence and transport that will allow for more stringent testing and validation of gyrokinetic turbulent-transport codes. This work directly impacts the development of predictive transport models in the U.S. FES program, such as TGLF, developed by General Atomics, which are used to predict performance in ITER and other burning plasma devices as part of advancing the development of fusion energy sciences.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2017

Land Surface Temperature Product Validation Best Practice Protocol Version 1.0 - October, 2017

The Global Climate Observing System (GCOS) has specified the need to systematically generate andvalidate Land Surface Temperature (LST) products. This document provides recommendations on goodpractices for the validation of LST products. Internationally accepted definitions of LST, emissivity andassociated quantities are provided to ensure the compatibility across products and reference data sets. Asurvey of current validation capabilities indicates that progress is being made in terms of up-scaling and insitu measurement methods, but there is insufficient standardization with respect to performing andreporting statistically robust comparisons.Four LST validation approaches are identified: (1) Ground-based validation, which involvescomparisons with LST obtained from ground-based radiance measurements; (2) Scene-based intercomparisonof current satellite LST products with a heritage LST products; (3) Radiance-based validation,which is based on radiative transfer calculations for known atmospheric profiles and land surface emissivity;(4) Time series comparisons, which are particularly useful for detecting problems that can occur during aninstrument's life, e.g. calibration drift or unrealistic outliers due to undetected clouds. Finally, the need foran open access facility for performing LST product validation as well as accessing reference LST datasets isidentified.

best practice↗
NASA NTRS2020

Chrono-Validation of Near-Real-Time Landslide Susceptibility Models via Plugin Statistical Simulations

The idea behind any validation scheme in landslide susceptibility studies is to test whether a model calibrated on a certain data can predict an unknown dataset of the same nature (landslide presences/absences and covariates). Almost the entirety of landslide susceptibility studies are validated by subsetting a single dataset into a training and test sets. This dataset usually corresponds either to event-specific or to historical inventories. Very rarely, a multi-temporal inventory is available and, in the few cases where this condition is met, the validation practices involve training a model on a specific landslide inventory, deriving a single predictive equation and validating it on a subsequent landslide inventory. This commonly leads landslide predictive studies, even those with a strong statistical rigor, to neglect the uncertainty estimation in their modeling scheme. In statistics, validation can also be performed via statistical simulations. This means that after fitting a given model, one can generate any number of predictive functions and test their predictive skills on any type and number of unknown datasets. In this work, we take a similar direction and we apply it to model and validate three separate co-seismic inventories, including an uncertainty estimation phase. We mapped these inventories within the same area in Indonesia, for three earthquakes occurred in 2012, 2017 and 2018. Specifically, we build three event-specific Bayesian Generalize Additive Models of the binomial family. From each model we then simulate 1000 predictive realizations over the remaining two inventories, by using a plug-in scheme where all the morphometric covariates are kept fixed and only the ground motion is replaced according to the prediction target. By doing so, we introduce a new analytical tool for near-real-time landslide predictive purposes, which is able to produce a probabilistic model which stands in between the definitions of susceptibility and hazard. In fact, our model is able to accurately estimate “where” and “when” - although not “how frequently” - landslide have occurred by featuring the multitemporal information of the trigger. In our findings, the simulations are quite similar to the fitted models; and the nine combinations we analyse produce excellent performance. This result confirms the assumption that “the past is the key to the future”, as we show that the relative contribution of each variable and their interactions in each probabilistic model remains practically the same across temporal replicates. This information is not trivial because it supports the routines implemented in global near-real-time applications.

Temporal validation↗
NASA NTRSDate not supplied

Validation of Rendered Natural and Artificial Lighting Environments in Real Time Lunar South Pole Simulations

NASA’s Artemis campaign is making heavy use of simulation to help return humans to the lunar surface by the end of the decade. There are several aspects of the lunar surface and its environment which must be accurately modeled before these simulations can be relied upon to influence decisions being made under these programs. Digital Lunar Exploration Sites, a paper submitted to the 2022 IEEE Aerospace Conference, outlined the process used to generate the lunar surface in a digital environment. This paper will expand upon this topic and delve into the steps being taken by the NASA Exploration Systems Simulations (NExSyS) team at NASA’s Johnson Space Center (JSC) to properly verify and validate these simulations, with a focus on the visual aspects of the environment. Natural lighting validation relies in part on the wealth of data generated during the Apollo program. Many images taken by Apollo astronauts on the lunar surface have been replicated in the simulated environments to gain confidence in the accuracy of terrain and lighting models. However, because the environment the Artemis astronauts will experience at the Lunar South Pole (LSP) is dissimilar from the near-equatorial Apollo sites, other validation techniques must be applied. At the LSP, the sun crests only about three degrees above the horizon and when combined with the lack of a lunar atmosphere, lighting in this region is often very different than what a human would experience on Earth. Solar illumination, earthshine, human eye response, solar blooming, lunar regolith optical properties, and shadows cast by rocks and crater walls will play a significant role in an astronaut’s ability to safely conduct an Extra-Vehicular Activity (EVA) or perform a traverse with a lunar rover. Approaches for validation of these aspects of the rendered LSP environment are considered in this paper. In addition to natural lighting, approaches for the validation of artificial lighting models at the LSP are discussed. The JSC Lighting Lab has been studying the illumination profile of the Exploration Infomatics Subsystem (xINFO) lighting on the Exploration EVA Mobility Unit (xEMU). How these lights interact with the solar illumination and the shadows being cast on the lunar surface is of particular interest, so the validity of models representing these lights in a human-in-the-loop virtual reality environment becomes very important. This paper also touches on some of the simulation performance considerations when a Human in the Loop (HITL) is present, which drives the need for real time rendering of the environment. Natural and artificial lighting will play a crucial role to decisions being made when planning and executing missions at the Lunar South Pole (LSP) and it is vitally important to understand the LSP environment before we return.

Lunar↗
NASA NTRS2023

Approaches for Validation of Lighting Environments in Realtime Lunar South Pole Simulations

NASA’s Artemis campaign is making heavy use of simulation to help return humans to the lunar surface by the end of the decade. There are several aspects of the lunar surface and its environment which must be accurately modeled before these simulations can be relied upon to influence decisions being made under these programs. Digital Lunar Exploration Sites, a paper submitted to the 2022 IEEE Aerospace Conference, outlined the process used to generate the lunar surface in a digital environment. This paper will expand upon this topic and delve into the steps being taken by the NASA Exploration Systems Simulations (NExSyS) team at NASA’s Johnson Space Center (JSC) to properly verify and validate these simulations, with a focus on the visual aspects of the environment. Natural lighting validation relies in part on the wealth of data generated during the Apollo program. Many images taken by Apollo astronauts on the lunar surface have been replicated in the simulated environments to gain confidence in the accuracy of terrain and lighting models. However, because the environment the Artemis astronauts will experience at the Lunar South Pole (LSP) is dissimilar from the near-equatorial Apollo sites, other validation techniques must be applied. At the LSP, the sun crests only about 1.5 degrees above the horizon and when combined with the lack of a lunar atmosphere, lighting in this region is often very different than what a human would experience on Earth. Solar illumination, earthshine, human eye response, solar blooming, lunar regolith optical properties, and shadows cast by rocks and crater walls will play a significant role in an astronaut’s ability to safely conduct an Extra-Vehicular Activity (EVA) or perform a traverse with a lunar rover. Approaches for validation of these aspects of the rendered LSP environment are considered in this paper. In addition to natural lighting, approaches for the validation of artificial lighting models at the LSP are discussed. The JSC Lighting Lab has been studying the illumination profile of the Exploration Informatics Subsystem (xINFO) lighting on the Exploration EVA Mobility Unit (xEMU). How these lights interact with the solar illumination and the shadows being cast on the lunar surface is of particular interest, so the validity of models representing these lights in a human-in-the-loop virtual reality environment becomes very important. This paper also touches on some of the simulation performance considerations when a Human in the Loop (HITL) is present, which drives the need for realtime rendering of the environment. Natural and artificial lighting will play a crucial role to decisions being made when planning and executing missions at the Lunar South Pole (LSP) and it is vitally important to understand the LSP environment before we return.

Lunar↗
DOE OSTI2018

OECD-NEA Expert Group on Multi-Physics Experimental Data, Benchmarks and Validation

High-fidelity, multi-physics modeling and simulation (M&S) tools are being developed and utilized for a variety of applications in nuclear science and technology and show great promise in their abilities to reproduce observed phenomena for many applications. Even with the increasing fidelity and sophistication of coupled multi-physics M&S tools, the underpinning models and data still need to be validated against experiments that may require a more complex array of validation data because of the great breadth of the time, energy and spatial domains of the physical phenomena that are being simulated. The Expert Group on Multi-Physics Experimental Data, Benchmarks and Validation (MPEBV) of the Nuclear Energy Agency (NEA) of the Organization for Economic Cooperation and Development (OECD) was formed to address the challenges with the validation of such tools. The work of the MPEBV expert group is shared among three task forces to fulfill its mandate and specific exercises are being developed to demonstrate validation principles for common industrial challenges. This paper describes the overall mission of the group, the specific objectives of the task forces, the linkages among the task forces, and the development of a validation exercise that focuses on a specific reactor challenge problem.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2019

Validation of Numerical Models of the Offshore Wind Turbine from the Alpha Ventus Wind Farm Against Full-Scale Measurements Within OC5 Phase III: Preprint

The main objective of the Offshore Code Comparison Collaboration Continuation, with Correlation (OC5) project, is validation of aero-hydro-servo-elastic simulation tools for offshore wind turbines (OWTs) through comparison of simulated results to the response data of physical systems. Phase III of the OC5 project validates OWT models against the measurements recorded on Senvion 5M wind turbine supported by the OWEC Quattropod from the alpha ventus offshore wind farm. The following operating conditions of the wind turbine were chosen for the validation: (1) Idling below the cut-in wind speed; (2) Rotornacelle assembly (RNA) rotation maneuver below the cut-in wind speed; (3) Power production below and above the rated wind speed; and (4) Shut-down. A number of validation load cases were defined based on these operating conditions. The following measurements were used for validation: (1) Strains and accelerations recorded on the support structure; (2) Pitch, yaw, and azimuth angles, generator speed, and electrical power recorded from the RNA. Strains were not directly available from the majority of the OWT simulation tools. Therefore, strains were calculated based on out-of-plane bending moments, axial forces, and geometrical properties of the structural members. Also, a number of issues arose during the validation: (1) The need for a thorough quality check of sensor measurements; (2) The sensitivity of the turbine loads to the controller and airfoil properties, which were only approximated in the modeling approach; (3) The importance of estimating and applying an appropriate damping value for the structure; and (4) The importance of wind characteristics beyond turbulence on the loads. The simulation results and measurements were compared in terms of time series, discrete Fourier transforms (DFTs), power spectral densities (PSDs), probability density functions (PDFs) of strains and accelerometers. A good match was achieved between the measurements and models set up by OC5 Phase III participants.

17 WIND ENERGY↗
DOE OSTI2021

Key Constructs for Reasonable Confidence in System Validation Programs

This paper presents four key constructs developed in preparing IEEE Std P2411TM, Human Factors Engineering Guide for the Validation of System Designs and Integrated Systems Operations at Nuclear Facilities for submission. These constructs reflect judgments that responsible parties make in order to bring validation to closure. Such judgments weigh competing concerns and cost-benefits for stakeholders. Improved processes and methods may support such judgments but cannot alone eliminate uncertainty. Stating and clarifying these constructs aims to reduce uncertainty in the validation process, to better prepare the implementers and reviewers of future validation programs, both in the nuclear industry and beyond. Reasonable Confidence – A proof standard, as used to reach conclusions in a legal case. Legal proof standards are bounded between “preponderance of evidence” and “beyond reasonable doubt”. Reasonable confidence is comparable to an intermediate legal proof standard of “clear and convincing evidence.” Representative Test Set – A collection of scenarios of sufficient variety to represent the anticipated range of operational conditions, events, evolutions, and activities for validating the system(s) under test. Dispositive vs. Diagnostic Criteria – Relevant performance criteria are placed in one of two categories. Dispositive criteria are assessed, without exception, to determine whether a validation test passes or fails overall. Diagnostic criteria are assessed, subject to justified exception, to evaluate the quality or degree of some particular aspect of performance. Repeatability – Consistency in achieving passing results is to be demonstrated for each scenario in the representative test set; thus, the minimum number of formal repetitions for each scenario is two.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2020

Validation and Independent Uncertainty Analysis of the MIX-SOL-THERM-003 ICSBEP Benchmark

The International Criticality Safety Benchmark Evaluation Project (ICSBEP) was started in 1992 by the United States Department of Energy and later in 1995 became an international project with contributions from 22 countries. The project is now organized by the OECD (Organisation for Economic Co-operation and Development) Nuclear Energy Agency (NEA). In its most recent iteration, the ICSBEP handbook contains over five thousand evaluations of critical, near-critical, and subcritical experiments conducted in facilities all around the world. These benchmarks serve as valuable information for criticality safety engineers who can use them to validate calculation techniques and establish minimum subcritical margins for operations with fissionable materials. The benchmarks in the handbook are categorized by their fissile material composition, material form (oxide, solution, or metal), and fission energy spectra. This is especially useful for those looking for benchmarks similar to a system they are working on to compare methods and identify trends. The ICSBEP Handbook Uncertainty Guide is document outlining recommended practices and methods for determining uncertainties in these benchmarks. Quantifying these uncertainties thoroughly is crucial as it allows a higher degree of confidence that data used from them is valid and relevant. The guide stresses the importance of a thorough and well documented uncertainty analysis when evaluating an experiment. All measured values of a system, whether they be dimensions or material compositions, have a certain amount of uncertainty associated with them and can be analyzed one by one to determine their effects on the system. Many evaluated benchmarks in the handbook present this in detail, however some do not, mostly earlier evaluations performed in the 1990’s and early 2000’s. Recently at Los Alamos National Laboratory (LANL), the Nuclear Criticality Safety Division (NCSD) of LANL has been validating MCNP6.2 ® input files of criticality benchmarks for use by Whisper, a criticality safety code developed at LANL. This effort is also part of the OECD NEA Working Party on International Nuclear Data Evaluation Co-operation (WPEC) Subgroup 45, also known as Validation of Nuclear Data Libraries (VaNDaL). The goal of VaNDaL is to compile a set of validated simulation inputs for use in validating nuclear data and simulation codes. As part of these efforts, one of the benchmarks reviewed was the MIX SOL-THERM-003 ICSBEP benchmark. This paper provides an independent uncertainty analysis of this benchmark experiment.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2015

Consideration for the application of artificial neural networks to criticality code validation - 14270

The artificial neural network (ANN) is a consistent example for non-parametric parameter estimation and regression, with powerful universal approximation properties, that is distribution independent. ANN application to criticality code validation centers on aligning mathematical relationships between code validation parameters and neural network prediction for those parameter values. Two specific goals are: (1) an appropriate ANN architecture to determine validation parameter values, and (2) bootstrapping individual ANNs as a 'Prediction Interval' (PI) technique to estimate the Calculation Margin and Upper Subcritical Limit. The individual ANN workflow is segregated into Training, Validation (network), and Testing Sets, where the network predictive properties and performance are expressed in the latter set. The Bootstrap ANN replicates a large set of individual ANNs (taken from the original data distribution) where (1-α/2)100% PI estimates can be determined. Results from the ANN method are compared to a variety of validation examples: a 'toy example' from ANSI/ANS-8.24, Appendix D, a variety of methods noted in a Nuclear Criticality Safety Division 2009 paper, and several validation campaigns for highly enriched uranium and plutonium systems. The ANN method is robust and can be applied to single or multivariate inputs. A trained ANN can provide both an average estimate (e.g., single value) and an estimate over a range of independent variable values where the kUSL can be formed in a regression relationship suitable for extrapolation. The input dependence has been tested, ranging from 15 to over 400 sample size distributions. An expanded Shapiro-Wilk test (for sample sizes up to 5000) was implemented to confirm that the ANN method is transparent to distribution function. The Principle Component Analysis method was applied as an input preprocessing step embedded within the ANN in order to reduce dimensionality and correlation for multivariable inputs. This can potentially aid in determining accurate margins over applying arbitrary margins. The automatic optimization of the Bootstrap ANN PI is left for future consideration. (authors)

42 ENGINEERING↗
DOE OSTI2026

Connect microstructure evolution to property degradation with validated simulation

Reliable material performance is required for plasma-facing material (PFM) candidates. Previous research has shown that plasma and neutron radiation exposure induces microstructural changes in PFMs; changes in thermal and electrical conductivities and in material hardening and embrittlement were also observed after neutron irradiation. These material property changes will negatively impact the performance of the PFMs in a fusion reactor. Despite the well-known connection between material microstructure, properties, and performance, there is a need for validated modeling capabilities connecting PFM property degradation with microstructural evolution under fusion-relevant conditions. We are developing a simulation capability to couple plasma-induced microstructural evolution to material property degradation. Our approach relies on deliberate mapping between individual simulation models and experimental characterization for validation. The open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) software was used for this simulation capability development. A MOOSE phase-field model was coupled with the cluster dynamics code, Xolotl, to predict microstructural evolution. Microstructure characterization techniques, including scanning electron microscopy (SEM), transmission electron microscopy (TEM), and laser scanning confocal microscopy (LSCM) are used to validate these microstructural evolution simulations. Calculation of thermal and electrical conductivities with first principles simulations was performed for bulk material and for grain boundaries; these results are used within MOOSE models to calculate effective thermal and electrical conductivities as a function of grain characteristics. Thermoreflectance and four-probe techniques were employed to measure the thermal and electrical conductivities, respectively. A MOOSE crystal plasticity model was adapted to predict microstructure-sensitive deformation behavior, and X-ray diffraction (XRD) was used to collect bulk dislocation density data for validation. After individual simulation validation, these models are coupled to predict material property changes resulting from plasma exposure. We focused here on an experimental design to emphasize the separate effects of moderate thermal loads and plasma exposure using tungsten. Annealing of tungsten was performed under a protective environment for temperatures ranging from 500$^o$C to 1500$^o$C. The plasma exposure was completed in the Tritium Plasma Experiment at Idaho National Laboratory under a deuterium flux of 1e22 $\frac{D}{m^2s}$. This incremental approach is employed to build confidence in the modeling capability: separate-effects tests ensure that the models capture key mechanisms from single environmental conditions before predicting PFM property degradation under combined loads. We will show our early results from coupling these simulation models to predict PFM property changes from microstructural evolution. Comparisons of the simulation results with preliminary validation data will be discussed.

36 - MATERIALS SCIENCE↗
DOE OSTI2022

Evaluating wind speed and power forecasts for wind energy applications using an open-source and systematic validation framework

Building on the verification and validation work developed under the Second Wind Forecast Improvement Project, this work exhibits the value of a consistent procedure to evaluate wind power forecasts. We established an open-source Python code base tailored for wind speed and wind power forecast validation, WE-Validate. The code base can evaluate model forecasts with observations in a coherent manner. To demonstrate the systematic validation framework of WE-Validate, we designed and hosted a forecast evaluation benchmark exercise. We invited forecast providers in industry and academia to participate and submit forecasts for two case studies. We then evaluated the submissions with WE-Validate. Our findings suggest that ensemble means have reasonable skills in time series forecasting, whereas they are often inferior to single ensemble members in wind ramp forecasting. Adopting a voting scheme in ramp forecasting that allows ensemble members to detect ramps independently leads to satisfactory skill scores. Throughout this document, we also emphasize the importance of using statistically robust and resistant metrics as well as equitable skill scores in forecast evaluation.

17 WIND ENERGY↗
DOE OSTI2022

Development and validation of a mathematical model of heart rate response to fluid perturbation

Physiological closed-loop controlled (PCLC) medical devices monitor and automatically adjust the patient’s condition by using physiological variables as feedback, ideally with minimal human intervention to achieve the target levels set by a clinician. PCLC devices present a challenge when it comes to evaluating their performance, where conducting large clinical trials can be expensive. Virtual physiological patients simulated by validated mathematical models can be utilized to obtain preclinical evidence of safety and assess the performance of the PCLC medical device during normal and worst-case conditions that are unlikely to happen in a limited clinical trial. A physiological variable that plays a major role during fluid resuscitation is heart rate (HR). For in silico assessment of PCLC medical devices regarding fluid perturbation, there is currently no mathematical model of HR validated in terms of its predictive capability performance. This paper develops and validates a mathematical model of HR response using data collected from sheep subjects undergoing hemorrhage and fluid infusion. The model proved to be accurate in estimating the HR response to fluid perturbation, where averaged between 21 calibration datasets, the fitting performance showed a normalized root mean square error (NRMSE) of 7.41 ± 2.8%. The model was also evaluated in terms of model predictive capability performance via a leave-one-out procedure (21 subjects) and an independent validation dataset (6 subjects). Two different virtual cohort generation tools were used in each validation analysis. The generated envelope of virtual subjects robustly met the defined acceptance criteria, in which 95% of the testing datasets presented simulated HR patterns that were within a deviation of 50% from the observed data. In addition, out of 16000 and 18522 simulated subjects for the leave one-out and independent datasets, the model was able to generate at least one virtual subject that was close to the real subject within an error margin of 9.56 ± 3.15% and 11.1 ± 1.22% NRMSE, respectively. In conclusion, the model can generate valid virtual HR physiological responses to fluid perturbation and be incorporated into future non-clinical simulated testing setups for assessing PCLC devices intended for fluid resuscitation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2015

High-resolution multi-band imaging for validation and characterization of small Kepler planets

High-resolution ground-based optical speckle and near-infrared adaptive optics images are taken to search for stars in close angular proximity to host stars of candidate planets identified by the NASA Kepler Mission. Neighboring stars are a potential source of false positive signals. These stars also blend into Kepler light curves, affecting estimated planet properties, and are important for an understanding of planets in multiple star systems. Deep images with high angular resolution help to validate candidate planets by excluding potential background eclipsing binaries as the source of the transit signals. A study of 18 Kepler Object of Interest stars hosting a total of 28 candidate and validated planets is presented. Validation levels are determined for 18 planets against the likelihood of a false positive from a background eclipsing binary. Most of these are validated at the 99% level or higher, including five newly validated planets in two systems: Kepler-430 and Kepler-431. The stellar properties of the candidate host stars are determined by supplementing existing literature values with new spectroscopic characterizations. Close neighbors of seven of these stars are examined using multi-wavelength photometry to determine their nature and influence on the candidate planet properties. Most of the close neighbors appear to be gravitationally bound secondaries, while a few are best explained as closely co-aligned field stars. Revised planet properties are derived for each candidate and validated planet, including cases where the close neighbors are the potential host stars.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Applicability Analysis of Validation Evidence for Biomedical Computational Models

Computational modeling has the potential to revolutionize medicine the way it transformed engineering. However, despite decades of work, there has only been limited progress to successfully translate modeling research to patient care. One major difficulty which often occurs with biomedical computational models is an inability to perform validation in a setting that closely resembles how the model will be used. For example, for a biomedical model that makes in vivo clinically relevant predictions, direct validation of predictions may be impossible for ethical, technological, or financial reasons. Unavoidable limitations inherent to the validation process lead to challenges in evaluating the credibility of biomedical model predictions. Therefore, when evaluating biomedical models, it is critical to rigorously assess applicability, that is, the relevance of the computational model, and its validation evidence to the proposed context of use (COU). However, there are no well-established methods for assessing applicability. In this paper, we present a novel framework for performing applicability analysis and demonstrate its use with a medical device computational model. The framework provides a systematic, step-by-step method for breaking down the broad question of applicability into a series of focused questions, which may be addressed using supporting evidence and subject matter expertise. The framework can be used for model justification, model assessment, and validation planning. While motivated by biomedical models, it is relevant to a broad range of disciplines and underlying physics. Finally, the proposed applicability framework could help overcome some of the barriers inherent to validation of, and aid clinical implementation of, biomedical models.

97 MATHEMATICS AND COMPUTING↗