Search NASA⌕ Search

SEARCH · Search NASA

Results for “Simulation Credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2008

Dimensions of Credibility in Models and Simulations

Based on the National Aeronautics and Space Administration's (NASA's) work in developing a standard for models and simulations (M&S), the subject of credibility in M&S became a distinct focus. This is an indirect result from the Space Shuttle Columbia Accident Investigation Board (CAIB), which eventually resulted in an action, among others, to improve the rigor in NASA's M&S practices. The focus of this action came to mean a standardized method for assessing and reporting results from any type of M&S. As is typical in the standards development process, this necessarily developed into defming a common terminology base, common documentation requirements (especially for M&S used in critical decision making), and a method for assessing the credibility of M&S results. What surfaced in the development of the NASA Standard was the various dimensions credibility to consider when accepting the results from any model or simulation analysis. The eight generally relevant factors of credibility chosen in the NASA Standard proved only one aspect in the dimensionality of M&S credibility. At the next level of detail, the full comprehension of some of the factors requires an understanding along a couple of dimensions as well. Included in this discussion are the prerequisites for the appropriate use of a given M&S, the choice of factors in credibility assessment with their inherent dimensionality, and minimum requirements for fully reporting M&S results.

Steele, Martin J.↗
NASA NTRS2007

An Approach to Improved Credibility of CFD Simulations for Rocket Injector Design

Computational fluid dynamics (CFD) has the potential to improve the historical rocket injector design process by simulating the sensitivity of performance and injector-driven thermal environments to. the details of the injector geometry and key operational parameters. Methodical verification and validation efforts on a range of coaxial injector elements have shown the current production CFD capability must be improved in order to quantitatively impact the injector design process.. This paper documents the status of an effort to understand and compare the predictive capabilities and resource requirements of a range of CFD methodologies on a set of model problem injectors. Preliminary results from a steady Reynolds-Average Navier-Stokes (RANS), an unsteady Reynolds-Average Navier Stokes (URANS) and three different Large Eddy Simulation (LES) techniques used to model a single element coaxial injector using gaseous oxygen and gaseous hydrogen propellants are presented. Initial observations are made comparing instantaneous results, corresponding time-averaged and steady-state solutions in the near -injector flow field. Significant differences in the flow fields exist, as expected, and are discussed. An important preliminary result is the identification of a fundamental mixing mechanism, accounted for by URANS and LES, but missing in the steady BANS methodology. Since propellant mixing is the core injector function, this mixing process may prove to have a profound effect on the ability to more correctly simulate injector performance and resulting thermal environments. Issues important to unifying the basis for future comparison such as solution initialization, required run time and grid resolution are addressed.

Tucker, Paul K.↗
NASA NTRSDate not supplied

Development and Future of the Ten Simple Rules for Credible Practice of Models and Simulations in HealthCare

In this session, we will review the development and implementation history of the Ten Simple Rules (TSR) for credible practice in models and simulations that support healthcare. Our focus will be initially on the need and background for the Committee on Credible Practice of Models and Simulations in Healthcare (CPMS) as well as the TSR development process and implementation. This will flow into a demonstration of how the TSR has evolved as means of communicating and understanding the model development process, likely successful application and where the model could fail. We will end this presentation with a discussion on future development avenue, the CPMS efforts to develop a comprehensive rubric supporting current TSR implementation, and efforts to support further adoption of the TSR. The audience will be encouraged to comment on the current state of the TSR, share their TSR application experiences, and provide their insights on future development.

model↗
NASA NTRS2020

Credible practice of modeling and simulation in healthcare: ten rules from a multidisciplinary perspective

The complexities of modern biomedicine are rapidly increasing. Thus, modeling and simulation have become increasingly important as a strategy to understand and predict the trajectory of pathophysiology, disease genesis, and disease spread in support of clinical and policy decisions. In such cases, inappropriate or ill-placed trust in the model and simulation outcomes may result in negative outcomes, and hence illustrate the need to formalize the execution and communication of modeling and simulation practices. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the devel-opment and reuse of the models. For the past several years, the Committee on Credible Practice of Modeling and Simulation in Healthcare, an interdisciplinary group seeded from a U.S. interagency initiative, has worked to codify best practices. Here, we provide Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative analysis by the Committee’s multidisciplinary membership, followed by a large stakeholder com-munity survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. While biomedical science and clinical care domains have somewhat different requirements and expectations for credible practice, our study converged on rules that would be useful across a broad swath of model types. In brief, the rules are: (1) Define context clearly. (2) Use contextually appropriate data. (3) Evaluate within context. (4) List limitations explicitly. (5) Use version control. (6) Document appropriately. (7) Disseminate broadly. (8) Get independent reviews. (9) Test competing imple-mentations. (10) Conform to standards. Although some of these are common sense guidelines, we have found that many are often missed or misconstrued, even by seasoned practitioners. Computational models are already widely used in basic science to generate new biomedical knowledge. As they penetrate clinical care and healthcare policy, contributing to personalized and precision medicine, clinical safety will require established guidelines for the credible practice of modeling and simulation in healthcare.

Credibility↗
NASA NTRS2008

Towards a Credibility Assessment of Models and Simulations

A scale is presented to evaluate the rigor of modeling and simulation (M&S) practices for the purpose of supporting a credibility assessment of the M&S results. The scale distinguishes required and achieved levels of rigor for a set of M&S elements that contribute to credibility including both technical and process measures. The work has its origins in an interest within NASA to include a Credibility Assessment Scale in development of a NASA standard for models and simulations.

Blattnig, Steve R.↗
NASA NTRSDate not supplied

Credible Practice of Models and Simulations for Healthcare

As computational models penetrate clinical care and drive healthcare policy decisions, there is increasing need to communicate that model creation and testing follows reasonable credible practice. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the development and reuse of the models. This presentation introduces the Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative multidisciplinary analysis and practitioner survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. Although some of these are common sense guidelines, many are often missed or misconstrued, even by seasoned practitioners.

credibility↗
NASA NTRS1995

Verification and Validation of Simulation Models for High-Speed Flow Fields

This paper is largely based on the paper titled "Guide to Credible Computational Fluid Dynamics Simulations (Invited)" (AIAA Paper 95-2225). The significance of computational fluid dynamics (CFD) simulations depends solely on their credibility. A customer of CFD products -- simulations and software -- expects that these products are credible for the intended use. Simulation model verification and validation are critical in establishing the credibility of simulations and in certifying simulation software. The only thing that matters in establishing the credibility is uncertainty, not veracity or validity. The sensitivity-uncertainty analysis is the key to the establishment of this credibility. Assessing the credibility of complex simulation results poses a significant challenge. Terminology, concepts, framework, and guidelines are presented for addressing this challenge. Verification assesses whether the problem is solved correctly and estimates the level of computational accuracy of simulations; validation determines whether the right problem is solved and assesses the level of validity of the simulation model by estimating the degree to which simulations accurately represent reality. These concepts and the related guidelines are discussed with examples from high-speed flow fields.

Mehta, Unmeel B.↗
NASA NTRS2010

The Continual Intercomparison of Radiation Codes (CIRC) Assessing Anew the Quality of GCM Radiation Algorithms

The simulation of changes in the Earth's climate due to solar and thermal radiative processes with global climate models (GCMs) is highly complex, depending on the parameterization of a multitude of nonlinearly coupled physical processes. In contrast, the germ of global climate change, the radiative forcing from enhanced abundances of greenhouse gases, is relatively well understood. The impressive agreement between detailed radiation calculations and highly resolved spectral radiation measurements in the thermal infrared under cloudless conditions (see, for example, Fig. 1) instills confidence in our knowledge of the sources of gaseous absorption. That the agreement spans a broad range of temperature and humidity regimes using instruments mounted on surface, aircraft, and satellite platforms not only attests to our capability to accurately calculate radiative fluxes under present conditions, but also provides confidence in the spectroscopic basis for computation of fluxes under conditions that might characterize future global climate (e.g., radiative forcing). Alas, the computational costs of highly resolved spectral radiation calculations cannot be afforded presently in GCMs. Such calculations have instead been used as the foundation for approximations implemented in fast but generally less accurate algorithms performing the needed radiative transfer (RT) calculations in GCMs. Credible climate simulations by GCMs cannot be ensured without accurate solar and thermal radiative flux calculations under all types of sky conditions: pristine cloudless, aerosol-laden, and cloudy. The need for accuracy in RT calculations is not only important for greenhouse gas forcing scenarios, but is also profoundly needed for the robust simulation of many other atmospheric phenomena, such as convective processes.

Oreopoulos, Lazaros↗
NASA NTRS2005

Impact of Lunar Dust on the Exploration Initiative

From the Apollo era it is known that dust on the Moon can cause serious problems for exploration activities. Such problems include adhering to clothing and equipment, reducing external visibility on landings, and causing difficulty to breathing and vision within the spacecraft. An important step in dealing with dust-related problems is to understand how dust grains behave in the lunar environment. All astronauts who walked on the Moon reported difficulties with lunar dust. Eugene Cernan, commander of Apollo 17, stated that one of the most aggravating, restricting facets of lunar surface exploration is the dust and its adherence to everything no matter what kind of material, whether it be skin, suit material, metal, no matter what it be and it's restrictive friction-like action to everything it gets on. Dust has also been highlighted as a priority by the Mars Exploration Program Assessment Group (MEPAG): 1A. Characterize both aeolian dust and particulates that would be kicked up from the martian regolith by surface operations of a human mission with fidelity sufficient to establish credible engineering simulation labs and/or software codes on Earth. We shall briefly describe the properties of lunar dust and its impact on the Apollo astronauts, and then summarize three main problems areas for understanding its behavior: Dust Adhesion and Abrasion, Surface Electric Fields and Dust Transport. These issues are all inter-related and must be well understood in order to minimize the impact of dust on lunar surface exploration.

Stubbs, T. J.↗
NASA NTRS2004

Evaluating the Credibility of Transport Processes in Simulations of Ozone Recovery using the Global Modeling Initiative Three-dimensional Model

The Global Modeling Initiative (GMI) has integrated two 36-year simulations of an ozone recovery scenario with an offline chemistry and tra nsport model using two different meteorological inputs. Physically ba sed diagnostics, derived from satellite and aircraft data sets, are d escribed and then used to evaluate the realism of temperature and transport processes in the simulations. Processes evaluated include barri er formation in the subtropics and polar regions, and extratropical w ave-driven transport. Some diagnostics are especially relevant to sim ulation of lower stratospheric ozone, but most are applicable to any stratospheric simulation. The global temperature evaluation, which is relevant to gas phase chemical reactions, showed that both sets of me teorological fields have near climatological values at all latitudes and seasons at 30 hPa and below. Both simulations showed weakness in upper stratospheric wave driving. The simulation using input from a g eneral circulation model (GMI(GCM)) showed a very good residual circulation in the tropics and Northern Hemisphere. The simulation with inp ut from a data assimilation system (GMI(DAS)) performed better in the midlatitudes than it did at high latitudes. Neither simulation forms a realistic barrier at the vortex edge, leading to uncertainty in the fate of ozone-depleted vortex air. Overall, tracer transport in the offline GML(GCM) has greater fidelity throughout the stratosphere tha n it does in the GMI(DAS)

Strahan, Susan E.↗
NASA NTRS2015

Credibility Assessment of Deterministic Computational Models and Simulations for Space Biomedical Research and Operations

Human missions beyond low earth orbit to destinations, such as to Mars and asteroids will expose astronauts to novel operational conditions that may pose health risks that are currently not well understood and perhaps unanticipated. In addition, there are limited clinical and research data to inform development and implementation of health risk countermeasures for these missions. Consequently, NASA's Digital Astronaut Project (DAP) is working to develop and implement computational models and simulations (M&S) to help predict and assess spaceflight health and performance risks, and enhance countermeasure development. In order to effectively accomplish these goals, the DAP evaluates its models and simulations via a rigorous verification, validation and credibility assessment process to ensure that the computational tools are sufficiently reliable to both inform research intended to mitigate potential risk as well as guide countermeasure development. In doing so, DAP works closely with end-users, such as space life science researchers, to establish appropriate M&S credibility thresholds. We will present and demonstrate the process the DAP uses to vet computational M&S for space biomedical analysis using real M&S examples. We will also provide recommendations on how the larger space biomedical community can employ these concepts to enhance the credibility of their M&S codes.

Mulugeta, Lealem↗
NASA NTRSDate not supplied

Evaluating the Feasibility of Data Simulation for Radiation Research: Estimating RBE Values for HZE Radiation

As the drive for deep space exploration intensifies, a comprehensive understanding of the health effects of radiation exposure becomes paramount to the future of human space flight. However, epidemiological data for radiation exposure, particularly to high-energy (HZE) ions, is limited, partially due to the financial and logistical costs of radiation studies. As an alternative, this study aims to assess the viability of data simulation strategies to accurately model potential study parameters prior to utilizing laboratory conditions. This study estimates a relative biological effectiveness (RBE) factor based on the solid tumor data for outbred mice provided by Edmundson et al. 2020. Excess relative risk (ERR) models for HZE-Fe ions and gamma radiation were estimated using Poisson regression with Weibull models to represent the background solid tumor hazard without radiation. RBE values were calculated from the ratio of the heavy ion linear slope to the gamma linear slope. The parameters from these models were then applied to simulate iterations of 300 datasets across HZE-Fe doses of 0.05, 0.2, 0.4, and 0.75 Gy; gamma radiation doses of 0.75, 2, and 3 Gy, as well as an unirradiated control group. Sample size per dose varied from 100-500 mice across simulations (800-4000 mice total per dataset). 1500 datasets were generated total (300 each for sample sizes 800, 1600, 3200, and 4000). For each dataset, ERR per radiation type and RBE for HZE-Fe were calculated. The RBE from the Edmondson data was calculated to be 5.55. The RBEs from the simulated data converged around this value across the increasing sample sizes. For n = 4000, the mean RBE across the 300 datasets was 5.71 (95% CI: 5.47-5.95). These results suggest that RBEs calculated from simulated data are credible. Based on this exploration, data simulation is a viable method of testing radiation studies. It provides a method of testing study assumptions and refining research questions prior to embarking on costly laboratory experiments.

Sneha Mehta↗
NASA NTRS1978

Numerical derivation of linear perturbation model from nonlinear real-time vehicle simulations - A means of validation and increased credibility

A numerical technique has been developed which generates linear perturbation models from nonlinear real-time aircraft vehicle simulations at Langley Research Center. The technique is very general and can be applied to simulations of any system described by nonlinear differential equations. Linear models about various trim conditions have been obtained from simulations of the Boeing 737, Northrop F5-E, Cessna 180, Sikorsky CH-54, and NASA/Army Rotor Systems Research Aircraft (RSRA). One use of the derived linear models is as a validation and verification tool for the nonlinear simulation.

Dieudonne, J. E.↗
NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Across engineering there is an evolving need to increase reliance on physics-based simulation to develop, design and optimize engineering systems. This increased reliance on modeling and simulation has highlighted a growing need to transform the confidence that modeling and simulation analysts have in their results into credibility for systems engineers to design and field systems more quickly and with less physical testing. For isolated components of a complex system, where a single discipline may drive product design, this is less of a concern as the relationship is often straightforward and easy to explain. However, when these isolated components are integrated, and are expected to operate in a multi-disciplinary context in which safety critical systems are involved, new concepts and model assurance standards are required. In this paper we address this challenge by showing how a model validation hierarchy can be exploited to identify those model validation experiments that will contribute most to increasing confidence and credibility of modeling and simulation predictions. The approach that is adopted contains four main steps. The first step is the construction of a model validation hierarchy that links subsystems, assemblies, and components to a hierarchy of physical experiments that can be used support model validation. This hierarchy connects the concerns of systems engineers to those of the modeling and simulation analyst in a clear and logical way. The structure and content of this hierarchy is then used in a second step to establish which physical phenomena have the greatest impact on overall system performance metrics. A gap analysis technique, based upon modeling and simulation concerns, is then used to prioritize the important physical phenomenon. Unfortunately, a common outcome of such gap analyses is the identification of many important gaps and so, in the final step of our process, we advocate the use of a global sensitivity analysis as a means to complete the prioritization.

Verification and Validation↗
NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Across engineering there is an evolving need to increase reliance on physics-based simulation to develop, design and optimize engineering systems. This increased reliance on modeling and simulation has highlighted a growing need to transform the confidence that modeling and simulation analysts have in their results into credibility for systems engineers to design and field systems more quickly and with less physical testing. For isolated components of a complex system, where a single discipline may drive product design, this is less of a concern as the relationship is often straightforward and easy to explain. However, when these isolated components are integrated, and are expected to operate in a multi-disciplinary context in which safety critical systems are involved, new concepts and model assurance standards are required. In this paper we address this challenge by showing how a model validation hierarchy can be exploited to identify those model validation experiments that will contribute most to increasing confidence and credibility of modeling and simulation predictions. The approach that is adopted contains four main steps. The first step is the construction of a model validation hierarchy that links subsystems, assemblies, and components to a hierarchy of physical experiments that can be used support model validation. This hierarchy connects the concerns of systems engineers to those of the modeling and simulation analyst in a clear and logical way. The structure and content of this hierarchy is then used in a second step to establish which physical phenomena have the greatest impact on overall system performance metrics. A gap analysis technique, based upon modeling and simulation concerns, is then used to prioritize the important physical phenomenon. Unfortunately, a common outcome of such gap analyses is the identification of many important gaps and so, in the final step of our process, we advocate the use of a global sensitivity analysis as a means to complete the prioritization.

Verification and Validation↗
NASA NTRS2014

NASA-STD-7009 Guidance Document for Human Health and Performance Models and Simulations

Rigorous verification, validation, and credibility (VVC) processes are imperative to ensure that models and simulations (MS) are sufficiently reliable to address issues within their intended scope. The NASA standard for MS, NASA-STD-7009 (7009) [1] was a resultant outcome of the Columbia Accident Investigation Board (CAIB) to ensure MS are developed, applied, and interpreted appropriately for making decisions that may impact crew or mission safety. Because the 7009 focus is engineering systems, a NASA-STD-7009 Guidance Document is being developed to augment the 7009 and provide information, tools, and techniques applicable to the probabilistic and deterministic biological MS more prevalent in human health and performance (HHP) and space biomedical research and operations.

numerical analysis↗
NASA NTRS2017

The Cloud Feedback Model Intercomparison Project (CFMIP) contribution to CMIP6.

The primary objective of CFMIP is to inform future assessments of cloud feedbacks through improved understanding of cloud-climate feedback mechanisms and better evaluation of cloud processes and cloud feedbacks in climate models. However, the CFMIP approach is also increasingly being used to understand other aspects of climate change, and so a second objective has now been introduced, to improve understanding of circulation, regional-scale precipitation, and non-linear changes. CFMIP is supporting ongoing model inter-comparison activities by coordinating a hierarchy of targeted experiments for CMIP6, along with a set of cloud-related output diagnostics. CFMIP contributes primarily to addressing the CMIP6 questions 'How does the Earth system respond to forcing?' and 'What are the origins and consequences of systematic model biases?' and supports the activities of the WCRP Grand Challenge on Clouds, Circulation and Climate Sensitivity. A compact set of Tier 1 experiments is proposed for CMIP6 to address this question: (1) what are the physical mechanisms underlying the range of cloud feedbacks and cloud adjustments predicted by climate models, and which models have the most credible cloud feedbacks? Additional Tier 2 experiments are proposed to address the following questions. (2) Are cloud feedbacks consistent for climate cooling and warming, and if not, why? (3) How do cloud-radiative effects impact the structure, the strength and the variability of the general atmospheric circulation in present and future climates? (4) How do responses in the climate system due to changes in solar forcing differ from changes due to CO2, and is the response sensitive to the sign of the forcing? (5) To what extent is regional climate change per CO2 doubling state-dependent (non-linear), and why? (6) Are climate feedbacks during the 20th century different to those acting on long-term climate change and climate sensitivity? (7) How do regional climate responses (e.g. in precipitation) and their uncertainties in coupled models arise from the combination of different aspects of CO2 forcing and sea surface warming? CFMIP also proposes a number of additional model outputs in the CMIP DECK, CMIP6 Historical and CMIP6 CFMIP experiments, including COSP simulator outputs and process diagnostics to address the following questions. 1. How well do clouds and other relevant variables simulated by models agree with observations? 2. What physical processes and mechanisms are important for a credible simulation of clouds, cloud feedbacks and cloud adjustments in climate models? 3. Which models have the most credible representations of processes relevant to the simulation of clouds? 4. How do clouds and their changes interact with other elements of the climate system?

carbon dioxide↗