V&V/UQ Activities to Help Support Computational Simulation Credibility.
Abstract not provided.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Abstract not provided.
Abstract not provided.
Abstract not provided.
The musculoskeletal system of astronauts is subject to physiological changes, potentially leading to injuries due to the exposure to different gravitational environments experienced during spaceflight. These injuries can occur while an astronaut is performing an extravehicular activity (EVA) in space, on lunar or planetary surfaces or while wearing a spacesuit during terrestrial training for an EVA. One area of concern are injuries occurring at the feet and ankles due to a poor fit of the spacesuit boot. These injuries can occur if the boot does not support the ankle sufficiently and the astronaut steps awkwardly, twisting the ankle or foot, during an EVA. The OpenSimTM software can assess EVA induced musculoskeletal injuries for muscle strains, ligament injuries and joint injuries. A modeling and simulation credibility assessment of the OpenSim musculoskeletal modeling and simulation procedures, applied to the boot/ankle injury scenario injury mechanisms that may occur while carrying out tasks in these environments, is performed. These modeling and simulation credibility assessments give information on the credibility of the model’s use in simulating foot/ankle related injury mechanisms.
Abstract not provided.
Abstract not provided.
The utility of software primarily depends on its reliability and performance; whereas, its significance depends solely on its credibility for intended use. The credibility of simulations confirms the credibility of software. The level of veracity and the level of validity of simulations determine the degree of credibility of simulations. The process of assessing this credibility in fields such as computational mechanics (CM) differs from that followed by the Defense Modeling and Simulation Office in operations research. Verification and validation (V&V) of CM simulations is not the same as V&V of CM software. Uncertainty is the measure of simulation credibility. Designers who use software are concerned with management of simulation uncertainty. Terminology and concepts are presented with a few examples from computational fluid dynamics.
The importance of credible, trustworthy numerical simulations is obvious especially when using the results for making high-consequence decisions. Determining the credibility of such numerical predictions is much more difficult and requires a systematic approach to assessing predictive capability, associated uncertainties and overall confidence in the computational simulation process for the intended use of the model. This process begins with an evaluation of the computational modeling of the identified, important physics of the simulation for its intended use. This is commonly done through a Phenomena Identification Ranking Table (PIRT). Then an assessment of the evidence basis supporting the ability to computationally simulate these physics can be performed using various frameworks such as the Predictive Capability Maturity Model (PCMM). There were several critical activities that follow in the areas of code and solution verification, validation and uncertainty quantification, which will be described in detail in the following sections. Here, we introduce the subject matter for general applications but specifics are given for the failure prediction project. In addition, the first task that must be completed in the verification & validation procedure is to perform a credibility assessment to fully understand the requirements and limitations of the current computational simulation capability for the specific application intended use. The PIRT and PCMM are tools used at Sandia National Laboratories (SNL) to provide a consistent manner to perform such an assessment. Ideally, all stakeholders should be represented and contribute to perform an accurate credibility assessment. PIRTs and PCMMs are both described in brief detail below and the resulting assessments for an example project are given.
This paper investigates the changes of the Southern Westerly Winds (SWW) and Southern Ocean (SO) upwelling between the Last Glacial Maximum (LGM) and preindustrial (PI) in the PMIP3/CMIP5 simulations, highlighting the role of the Antarctic sea ice in modulating the wind stress effect on the ocean. Particularly, a discrepancy may occur between the changes in SWW and westerly wind stress, caused primarily by an equatorward expansion of winter Antarctic sea ice that undermines the wind stress in driving the liquid ocean. Such discrepancy may reflect the LGM condition in reality, in view of that the model simulates this condition has most credible simulation of modern SWW and Antarctic sea ice. The effect of wind stress on the SO upwelling is further explored via the wind-induced Ekman pumping, which is reduced under the LGM condition in all models, in part by the sea-ice “capping” effect present in the models.
This paper documents coupled simulations of two developmental versions of the Community Atmosphere Model (CAM) towards CAM6. The configuration called CAM5.4 introduces new microphysics, aerosol, and ice nucleation changes, among others to CAM. The CAM5.5 configuration represents a more radical departure, as it uses an assumed probability density function (PDF)-based unified cloud parameterization to replace the turbulence, shallow convection, and warm cloud macrophysics in CAM. This assumed PDF method has been widely used in the last decade in atmosphere-only climate simulations but has never been documented in coupled mode. Here, we compare the simulated coupled climates of CAM5.4 and CAM5.5 and compare them to the control coupled simulation produced by CAM5.3. We find that CAM5.5 has lower cloud forcing biases when compared to the control simulations. Improvements are also seen in the simulated amplitude of the Niño-3.4 index, an improved representation of the diurnal cycle of precipitation, subtropical surface wind stresses, and double Intertropical Convergence Zone biases. Degradations are seen in Amazon precipitation as well as slightly colder sea surface temperatures and thinner Arctic sea ice. Simulation of the 20th century results in a credible simulation that ends slightly colder than the control coupled simulation. The authors find this is due to aerosol indirect effects that are slightly stronger in the new version of the model and propose a solution to ameliorate this. Altogether, in these early coupled simulations, CAM5.5 produces a credible climate that is appropriate for science applications and is ready for integration into the National Center for Atmospheric Research's (NCAR's) next-generation climate model.
Licensing a new nuclear reactor involves evaluating credible accident scenarios by developing models that simulate accident progression to predict possible outcomes and their impact on safety. The formation of radionuclide-bearing aerosols in the respirable size range can significantly affect dose consequence and threaten human health and is therefore a focus of nuclear reactor safety assessments. Recent reviews of the literature identified a lack of existing experimental data describing the mechanisms of formation and properties (size, concentration, and composition) of radionuclide-bearing aerosols that may be produced from molten salt reactor (MSR) facilities during postulated accident scenarios. Experiments on aerosol formation from molten salt systems are a high priority need that will support MSR licensing by indicating the radiological consequences of aerosol formation and providing the data required for model development and validation. The evolving conditions that occur during a MSR accident, such as a spill of molten fuel salt, may affect aerosol formation mechanisms and aerosol properties over time. There is a need to conduct experiments that simulate credible MSR accidents in a laboratory to generate aerosols with realistic characteristics and behaviors and a need to measure these aerosols in real time under accident-relevant conditions. This report describes the development of a method that can be used to quantify the size and concentration of salt aerosol particles that form from molten salt systems in real time. This method will be employed in future integral effects tests that are conducted at an engineering scale to simulate realistic MSR accidents and in future separate effects tests that will provide mechanistic insight into aerosol formation and properties to support process model development.
Sandia's Dakota software (available at http://dakota.sandia.gov) supports science and engineering transformation through advanced exploration of simulations. Specifically it manages and analyzes ensembles of simulations to provide broader and deeper perspective for analysts and decision makers. This enables them to enhance understanding of risk, improve products, and assess simulation credibility. This manual offers Consortium for Advanced Simulation of Light Water Reactors (LWRs) (CASL) partners a guide to conducting Dakota-based VUQ studies for CASL problems. It motivates various classes of Dakota methods and includes examples of their use on representative application problems. On reading, a CASL analyst should understand why and how to apply Dakota to a simulation problem. This SAND report constitutes the product of CASL milestone L3:VUQ.V&V.P8.01 and is also being released as a CASL unlimited release report with number CASL-U-2014-0038-000.
Sandia's Dakota software (available at http://dakota.sandia.gov) supports science and engineering transformation through advanced exploration of simulations. Specifically, it manages and analyzes ensembles of simulations to provide broader and deeper perspective for analysts and decision makers. This enables them to enhance understanding of risk, improve products, and assess simulation credibility. This manual offers Consortium for Advanced Simulation of Light Water Reactors (LWRs) (CASL) partners a guide to conducting Dakota-based VUQ studies for CASL problems. It motivates various classes of Dakota methods and includes examples of their use on representative application problems. On reading, a CASL analyst should understand why and how to apply Dakota to a simulation problem.
This effort expands upon a previous NASA activity that developed flight simulation benchmark check-cases to include new check-cases for the Cislunar domain, comparing multiple NASA simulation tools. The results of this effort describe the benefits of standardizing inputs, simulation comparisons and describe an interactive website that enables comparison of externally provided simulation data. Participating simulations improved their software and identified implementation errors. This activity elevated simulation credibility and provided a measure of validation for the simulations actively in use for NASA’s Human Landing Systems (HLS).
This is the Appendix containing figures of simulation output data plots for comparison from the assessment, “Expansion of Check-Cases for 6DOF Simulation”. This effort expands upon a previous NASA activity that developed flight simulation benchmark check-cases to include new check-cases for the Cislunar domain, comparing multiple NASA simulation tools. The results of this effort describe the benefits of standardizing inputs, simulation comparisons and describe an interactive website that enables comparison of externally provided simulation data. Participating simulations improved their software and identified implementation errors. This activity elevated simulation credibility and provided a measure of validation for the simulations actively in use for NASA’s Human Landing Systems (HLS).
Computational fluid dynamics (CFD) and other advanced modeling and simulation (M&S) methods are increasingly relied on for predictive performance, reliability and safety of engineering systems. Analysts, designers, decision makers, and project managers, who must depend on simulation, need practical techniques and methods for assessing simulation credibility. The AIAA Guide for Verification and Validation of Computational Fluid Dynamics Simulations (AIAA G-077-1998 (2002)), originally published in 1998, was the first engineering standards document available to the engineering community for verification and validation (V&V) of simulations. Much progress has been made in these areas since 1998. The AIAA Committee on Standards for CFD is currently updating this Guide to incorporate in it the important developments that have taken place in V&V concepts, methods, and practices, particularly with regard to the broader context of predictive capability and uncertainty quantification (UQ) methods and approaches. This paper will provide an overview of the changes and extensions currently underway to update the AIAA Guide. Specifically, a framework for predictive capability will be described for incorporating a wide range of error and uncertainty sources identified during the modeling, verification, and validation processes, with the goal of estimating the total prediction uncertainty of the simulation. The Guide's goal is to provide a foundation for understanding and addressing major issues and concepts in predictive CFD. However, this Guide will not recommend specific approaches in these areas as the field is rapidly evolving. It is hoped that the guidelines provided in this paper, and explained in more detail in the Guide, will aid in the research, development, and use of CFD in engineering decision-making.
Abstract Image-based simulation, the use of 3D images to calculate physical quantities, relies on image segmentation for geometry creation. However, this process introduces image segmentation uncertainty because different segmentation tools (both manual and machine-learning-based) will each produce a unique and valid segmentation. First, we demonstrate that these variations propagate into the physics simulations, compromising the resulting physics quantities. Second, we propose a general framework for rapidly quantifying segmentation uncertainty. Through the creation and sampling of segmentation uncertainty probability maps, we systematically and objectively create uncertainty distributions of the physics quantities. We show that physics quantity uncertainty distributions can follow a Normal distribution, but, in more complicated physics simulations, the resulting uncertainty distribution can be surprisingly nontrivial. We establish that bounding segmentation uncertainty can fail in these nontrivial situations. While our work does not eliminate segmentation uncertainty, it improves simulation credibility by making visible the previously unrecognized segmentation uncertainty plaguing image-based simulation.
Power system frequency needs to be maintained close to its nominal value at all times to successfully balance load and generation and maintain system reliability. Adequate primary frequency response and secondary frequency response are the primary forces to correct an energy imbalance at the second-to-minute level. As wind energy becomes a larger portion of the world's energy portfolio, there is an increased need for wind to provide frequency response. This paper addresses one of the major concerns about using wind for frequency regulation: the unknown factor of the interaction between primary and secondary reserves. The lack of a commercially available tool to model this has limited the energy industry's understanding of when the depletion of primary reserves will impact the performance of secondary response or vice versa. This paper investigates the issue by developing a multi-area frequency response integration tool with combined primary and secondary capabilities. The simulation is conducted in close coordination with economical energy scheduling scenarios to ensure credible simulation results.