SEARCH · Search NASA

Results for “model credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Modeling and Simulation Credibility Assessment of Finite Element Models for Fall From Heights Injury Scenarios

Finite element (FE) computational models of the whole human body are being used to investigate the injury modalities and mechanisms arising from suited astronauts’ ExtraVehicular Activities (EVAs). These FE whole body models – simplified Elemance and THUMS pedestrian versions – have undergone a series of verification and validation (V&V) assessments for applications in automotive, domestic, and contact sports arenas. However, the FE models’ Modeling and Simulation (M&S) credibility assessments for simulating suited astronaut injuries due to fall from heights in altered gravity environments are limited. The current study evaluates the M&S credibility of simplified Elemance and THUMS pedestrian FE models based on eight credibility factors – data pedigree, input pedigree, code verification, solution verification, conceptual validation, referent validation, results uncertainty, and results robustness – as per NASA-STD-7009A [1]. These credibility factors were assigned ordinal scores ranging between 0 to 4, with a score of 0 in the case of insufficient evidence and 4 when all data for the model is known, used appropriately, and traceable to the Real-World System (RWS) with acceptable accuracy, precision, and uncertainty. The credibility factor sufficiency thresholds were assigned upon guidance from NASA EVA subject matter experts. The factor scores were ascertained through the evaluation of M&S credibility assessment procedures described in the published literature and FE model reports. The M&S credibility scores for the Elemance and THUMS FE models vary between 1 and 3 for data pedigree, input pedigree, and referent validation credibility factors, and between 0 and 1 for code verification, solution verification, conceptual validation, results uncertainty and results robustness. The results show that these FE models are credible for simulating fall from heights injury scenarios in specific kinematic and kinetic ranges that correlate with highway and contact sports-related external conditions. Multiple credibility elevation strategies are prescribed to improve the Elemance and THUMS models’ credibility for use in the EVA fall from height injury scenarios.

R A Perkins↗

NASA NTRS2025

Modeling and Simulation Credibility Assessments of Musculoskeletal Computational Models for Simulating Astronaut Injuries Due to a Poor Spacesuit Fit

The musculoskeletal (MS) system of astronauts is subject to physiological changes, potentially leading to injuries due to the exposure to different gravitational environments experienced during spaceflight. These injuries can occur while an astronaut is performing an Extravehicular Activity (EVA) in space, on lunar or planetary surfaces or while wearing a spacesuit during terrestrial training for an EVA. The OpenSim MS modeling software can assess EVA induced MS injury mechanisms such as muscle strains, ligament injuries and joint injuries. One area of concern, since there are only a few different spacesuit sizes with limited adjustability, is the possibility of a poorly fitting spacesuit. This can cause unnatural joint motions and torques resulting in various MS injuries. A credibility assessment of the OpenSim modeling and simulation procedures is performed per NASA-STD-7009A to provide information on the credibility of the model’s use in simulating EVA related injury mechanisms. The credibility assessment evaluated various OpenSim models against the following eight credibility factors: data pedigree, input pedigree, code verification, solution verification, conceptual validation, referent validation, results uncertainty and results robustness (sensitivity). The models evaluated for EVA injuries will require additional credibility factor analysis and upgrades to the model features, such as adding ligaments to a whole-body model, to reliably predict and analyze the EVA injuries expected to occur due to a poor spacesuit fit. The degree of elevation strategy required to increase the credibility assessment scores will depend on the model complexity and the injury mechanism.

Christopher A. Gallo↗

NASA NTRS2024

Modeling and Simulation Credibility Assessments of Musculoskeletal Computational Models for Simulating Astronaut Injuries Due to a Poor Spacesuit Fit

Christopher A Gallo↗

NASA NTRS2020

Credible practice of modeling and simulation in healthcare: ten rules from a multidisciplinary perspective

The complexities of modern biomedicine are rapidly increasing. Thus, modeling and simulation have become increasingly important as a strategy to understand and predict the trajectory of pathophysiology, disease genesis, and disease spread in support of clinical and policy decisions. In such cases, inappropriate or ill-placed trust in the model and simulation outcomes may result in negative outcomes, and hence illustrate the need to formalize the execution and communication of modeling and simulation practices. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the devel-opment and reuse of the models. For the past several years, the Committee on Credible Practice of Modeling and Simulation in Healthcare, an interdisciplinary group seeded from a U.S. interagency initiative, has worked to codify best practices. Here, we provide Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative analysis by the Committee’s multidisciplinary membership, followed by a large stakeholder com-munity survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. While biomedical science and clinical care domains have somewhat different requirements and expectations for credible practice, our study converged on rules that would be useful across a broad swath of model types. In brief, the rules are: (1) Define context clearly. (2) Use contextually appropriate data. (3) Evaluate within context. (4) List limitations explicitly. (5) Use version control. (6) Document appropriately. (7) Disseminate broadly. (8) Get independent reviews. (9) Test competing imple-mentations. (10) Conform to standards. Although some of these are common sense guidelines, we have found that many are often missed or misconstrued, even by seasoned practitioners. Computational models are already widely used in basic science to generate new biomedical knowledge. As they penetrate clinical care and healthcare policy, contributing to personalized and precision medicine, clinical safety will require established guidelines for the credible practice of modeling and simulation in healthcare.

Credibility↗

NASA NTRS2008

Dimensions of Credibility in Models and Simulations

Based on the National Aeronautics and Space Administration's (NASA's) work in developing a standard for models and simulations (M&S), the subject of credibility in M&S became a distinct focus. This is an indirect result from the Space Shuttle Columbia Accident Investigation Board (CAIB), which eventually resulted in an action, among others, to improve the rigor in NASA's M&S practices. The focus of this action came to mean a standardized method for assessing and reporting results from any type of M&S. As is typical in the standards development process, this necessarily developed into defming a common terminology base, common documentation requirements (especially for M&S used in critical decision making), and a method for assessing the credibility of M&S results. What surfaced in the development of the NASA Standard was the various dimensions credibility to consider when accepting the results from any model or simulation analysis. The eight generally relevant factors of credibility chosen in the NASA Standard proved only one aspect in the dimensionality of M&S credibility. At the next level of detail, the full comprehension of some of the factors requires an understanding along a couple of dimensions as well. Included in this discussion are the prerequisites for the appropriate use of a given M&S, the choice of factors in credibility assessment with their inherent dimensionality, and minimum requirements for fully reporting M&S results.

Steele, Martin J.↗

NASA NTRS2023

Modeling and Simulation Credibility Assessments of Whole-Body Finite Element Computational Models for Use in NASA Extravehicular Activity Applications

Computational finite element (FE) models are used in suited astronaut injury risk assessments; however, these models’ verification, validation, and credibility (VV&C) procedures for simulating injuries in altered gravity environments are limited. Our study conducts VV&C assessments of THUMS and Elemance whole-body FE models for predicting suited astronaut injury biomechanics using eight credibility factors, as per NASA-STD-7009A. Credibility factor ordinal scores are assigned by reviewing existing documentation describing VV&C practices, and credibility sufficiency thresholds are assigned based on input from subject matter experts. Our results show the FE models are credible for suited astronaut injury investigation in specific ranges of kinematic and kinetic conditions correlating to highway and contact sports events. Nevertheless, these models are deficient when applied outside these ranges. Several credibility elevation strategies are prescribed to improve models’ credibility for the NASA-centric application domain.

Finite Element↗

NASA NTRS2023

Modeling and simulation credibility assessments of whole-body finite element computational models for use in NASA extravehicular activity applications

Finite Element↗

NASA NTRSDate not supplied

Credible Practice of Models and Simulations for Healthcare

As computational models penetrate clinical care and drive healthcare policy decisions, there is increasing need to communicate that model creation and testing follows reasonable credible practice. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the development and reuse of the models. This presentation introduces the Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative multidisciplinary analysis and practitioner survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. Although some of these are common sense guidelines, many are often missed or misconstrued, even by seasoned practitioners.

credibility↗

NASA NTRSDate not supplied

Development and Future of the Ten Simple Rules for Credible Practice of Models and Simulations in HealthCare

In this session, we will review the development and implementation history of the Ten Simple Rules (TSR) for credible practice in models and simulations that support healthcare. Our focus will be initially on the need and background for the Committee on Credible Practice of Models and Simulations in Healthcare (CPMS) as well as the TSR development process and implementation. This will flow into a demonstration of how the TSR has evolved as means of communicating and understanding the model development process, likely successful application and where the model could fail. We will end this presentation with a discussion on future development avenue, the CPMS efforts to develop a comprehensive rubric supporting current TSR implementation, and efforts to support further adoption of the TSR. The audience will be encouraged to comment on the current state of the TSR, share their TSR application experiences, and provide their insights on future development.

model↗

NASA NTRS2010

Composite Overwrapped Pressure Vessels (COPV): Developing Flight Rationale for the Space Shuttle Program

Introducing composite vessels into the Space Shuttle Program represented a significant technical achievement. Each Orbiter vehicle contains 24 (nominally) Kevlar tanks for storage of pressurized helium (for propulsion) and nitrogen (for life support). The use of composite cylinders saved 752 pounds per Orbiter vehicle compared with all-metal tanks. The weight savings is significant considering each Shuttle flight can deliver 54,000 pounds of payload to the International Space Station. In the wake of the Columbia accident and the ensuing Return to Flight activities, the Space Shuttle Program, in 2005, re-examined COPV hardware certification. Incorporating COPV data that had been generated over the last 30 years and recognizing differences between initial Shuttle Program requirements and current operation, a new failure mode was identified, as composite stress rupture was deemed credible. The Orbiter Project undertook a comprehensive investigation to quantify and mitigate this risk. First, the engineering team considered and later deemed as unfeasible the option to replace existing all flight tanks. Second, operational improvements to flight procedures were instituted to reduce the flight risk and the danger to personnel. Third, an Orbiter reliability model was developed to quantify flight risk. Laser profilometry inspection of several flight COPVs identified deep (up to 20 mil) depressions on the tank interior. A comprehensive analysis was performed and it confirmed that these observed depressions were far less than the criterion which was established as necessary to lead to liner buckling. Existing fleet vessels were exonerated from this failure mechanism. Because full validation of the Orbiter Reliability Model was not possible given limited hardware resources, an Accelerated Stress Rupture Test of a flown flight vessel was performed to provide increased confidence. A Bayesian statistical approach was developed to evaluate possible test results with respect to the model credibility and thus flight rationale for continued operation of the Space Shuttle with existing flight hardware. A non-destructive evaluation (NDE) technique utilizing Raman Spectroscopy was developed to directly measure the overwrap residual stress state. Preliminary results provide optimistic results that patterns of fluctuation in fiber elastic strains over the outside vessel surface could be directly correlated with increased fiber stress ratios and thus reduced reliability.

Kezirian, Michael T.↗

NASA NTRS2003

Investigations of Mechanisms Associated with Nucleate Boiling Under Microgravity Conditions

In this work a building block type of approach is used so that a basic understanding of the processes that contribute to nucleate boiling heat fluxes under microgravity conditions can be developed. This understanding will lead to development of a mechanistic model for nucleate boiling heat transfer which could eventually be used as a design tool in space applications. Task Description Task 1: Fabrication of the Experimental Setup. Under this task, the test section and liquid holding and viewing chambers will be fabricated. Artificial cylinder cavities will be formed on silicon wafers. A single cavity and two or four cavities, with a prescribed spacing and size, will be formed. The desired nucleation wall superheat will be used to determine the size of the mouth of the cavities. Task 2: Experiments. The basic experiments for flow and temperature field around single and multiple (2 or 4 separated or merged bubbles growing on downward facing or inclined surfaces) will be carried out under normal gravity conditions. The experiments will be conducted at one atmosphere pressure, but liquid subcooling will be varied from 0 to 30C. Water and PF-5050 will be used as test liquids. Task 3: Analytical/Numerical Models. In this task, transient temperature and flow field in vapor and liquid will be determined during growth of a single bubble. Analysis will include the evolution of the vapor-liquid interface and development of microlayer underneath the bubbles. For merged bubbles, detailed calculations of flow and temperature field will be carried out for transient shapes of vapor stems supporting a large bubble and the corresponding evaporation rate. Flow and temperature field for a bubble sliding along a heated wall will also be determined. Microgravity conditions will be simulated and a framework of a numerical tool for prediction of nucleate boiling heat fluxes under microgravity conditions will be developed. Task 4: Experiments in a KC-135. To understand bubble growth and detachment behavior of single or large merged bubbles, boiling experiments will be conducted under low gravity (10-2 g) conditions of the aircraft. In these experiments, 'designed' surfaces will be used. Visual observations and heat transfer data will be taken, but holography will not be used. The apparatus used for laboratory experiments will also be employed for experiments in the aircraft. Task 5: Experiments in the Space Shuttle. Effort will be devoted for defining a boiling experiment to be conducted on a 'designed' surface. The experiment will provide microgravity data on bubble growth and departure. These data are needed for development of a credible model for nucleate boiling heat fluxes under microgravity conditions. The heat transfer data will also be obtained and will be used to validate the models.

Dhir, Vijay K.↗

NASA NTRS1991

CFD code validation for nozzle flowfields

Renewed interest in hypersonic airbreathing flight has resulted in heavy reliance on computational fluid dynamics for all aspects of aerodynamic and scramjet component modeling. Credible numerical performance predictions require calibration to unit type flowfields. This paper presents results of one such calibration to nonreacting simulated scramjet nozzle flowfields. The experimental test program was performed on a static thrust stand. Computed results are in excellent agreement with measured wall pressure, pitot pressure profiles, and thrust. Agreement with thrust is shown to be within the reported accuracy of the experimental measurements.

Jentink, Thomas N.↗

NASA NTRS2014

Integrated Medical Model Verification, Validation, and Credibility

The Integrated Medical Model (IMM) was designed to forecast relative changes for a specified set of crew health and mission success risk metrics by using a probabilistic (stochastic process) model based on historical data, cohort data, and subject matter expert opinion. A probabilistic approach is taken since exact (deterministic) results would not appropriately reflect the uncertainty in the IMM inputs. Once the IMM was conceptualized, a plan was needed to rigorously assess input information, framework and code, and output results of the IMM, and ensure that end user requests and requirements were considered during all stages of model development and implementation. METHODS: In 2008, the IMM team developed a comprehensive verification and validation (VV) plan, which specified internal and external review criteria encompassing 1) verification of data and IMM structure to ensure proper implementation of the IMM, 2) several validation techniques to confirm that the simulation capability of the IMM appropriately represents occurrences and consequences of medical conditions during space missions, and 3) credibility processes to develop user confidence in the information derived from the IMM. When the NASA-STD-7009 (7009) was published, the IMM team updated their verification, validation, and credibility (VVC) project plan to meet 7009 requirements and include 7009 tools in reporting VVC status of the IMM. RESULTS: IMM VVC updates are compiled recurrently and include 7009 Compliance and Credibility matrices, IMM VV Plan status, and a synopsis of any changes or updates to the IMM during the reporting period. Reporting tools have evolved over the lifetime of the IMM project to better communicate VVC status. This has included refining original 7009 methodology with augmentation from the NASA-STD-7009 Guidance Document. End user requests and requirements are being satisfied as evidenced by ISS Program acceptance of IMM risk forecasts, transition to an operational model and simulation tool, and completion of service requests from a broad end user consortium including Operations, Science and Technology Planning, and Exploration Planning. CONCLUSIONS: The VVC approach established by the IMM project of combining the IMM VV Plan with 7009 requirements is comprehensive and includes the involvement of end users at every stage in IMM evolution. Methods and techniques used to quantify the VVC status of the IMM have not only received approval from the local NASA community but have also garnered recognition by other federal agencies seeking to develop similar guidelines in the medical modeling community.

Validation↗

NASA NTRS2014

How to Develop and Interpret a Credibility Assessment of Numerical Models for Human Research: NASA-STD-7009 Demystified

In the wake of the Columbia accident, the NASA-STD-7009 [1] credibility assessment was developed as a unifying platform to describe model credibility and the uncertainties in its modeling predictions. This standard is now being adapted by NASAs Human Research Program to cover a wide range of numerical models for human research. When used properly, the standard can improve the process of code development by encouraging the use of best practices. It can also give management more insight in making informed decisions through a better understanding of the models capabilities and limitations.To a newcomer, the abstractions presented in NASA-STD-7009 and the sheer volume of information that must be absorbed can be overwhelming. This talk is aimed at describing the credibility assessment, which is the heart of the standard, in plain terms. It will outline how to develop a credibility assessment under the standard. It will also show how to quickly interpret the graphs and tables that result from the assessment and how to drill down from the top-level view to the foundation of the assessment. Finally, it will highlight some of the resources that are available for further study.

numerical analysis↗

NASA NTRS2014

Verification, Validation and Credibility Assessment of a Computational Model of the Advanced Resistive Exercise Device (ARED)

The Advanced Resistive Exercise Device (ARED) is the resistive exercise device used by astronauts on the International Space Station (ISS) to mitigate bone loss and muscle atrophy due to extended exposure to microgravity (micro g). The Digital Astronaut Project (DAP) has developed a multi-body dynamics model of biomechanics models for use in spaceflight exercise physiology research and operations. In an effort to advance model maturity and credibility of the ARED model, the DAP performed verification, validation and credibility (VV and C) assessment of the analyses of the model in accordance to NASA-STD-7009 'Standards for Models and Simulations'.

Werner, C. R.↗

NASA NTRS2008

Towards a Credibility Assessment of Models and Simulations

A scale is presented to evaluate the rigor of modeling and simulation (M&S) practices for the purpose of supporting a credibility assessment of the M&S results. The scale distinguishes required and achieved levels of rigor for a set of M&S elements that contribute to credibility including both technical and process measures. The work has its origins in an interest within NASA to include a Credibility Assessment Scale in development of a NASA standard for models and simulations.

Blattnig, Steve R.↗

NASA NTRS2019

2018-2019 Mid-Term Credibility Plan Review

This presentation summarizes the Interagency Modeling and Analysis Group (IMAG) / Multiscale Modeling (MSM)'s Committee on Credible Practice in Modeling and Simulation in Healthcare's (CPMS) mid-term review of each MSM U01 project's progress toward fulfilling their Model Credibility plan. Generally, the model credibility reporting and update process utilized several pieces of information formatted to communicate to the uninformed reviewer/user an assessment of steps taken toward model credibility with respect to the intended use of each U01 model, as well as communicate where the more interested reviewer/user can go to find more information. The reviewers, consisting of the members of the CPMS Executive Committee, focused on the documentation and communication of each PI's credibility efforts and did not perform an assessment of the implemented credible practice given the evolving nature of the projects. In this inaugural activity, the CPMS members sought insight on how PIs may view the importance of communicating credibility, as well as how PIs interpret establishing evidence for credibility within the scope of CPMS Tens simple rules. To that end, reviewers provided feedback on documentation and communication adequacy, and, only in few rare cases, on the credible practice itself.

Myers, Jerry↗

NASA NTRS2009

NASA Standard for Models and Simulations: Credibility Assessment Scale

As one of its many responses to the 2003 Space Shuttle Columbia accident, NASA decided to develop a formal standard for models and simulations (M&S). Work commenced in May 2005. An interim version was issued in late 2006. This interim version underwent considerable revision following an extensive Agency-wide review in 2007 along with some additional revisions as a result of the review by the NASA Engineering Management Board (EMB) in the first half of 2008. Issuance of the revised, permanent version, hereafter referred to as the M&S Standard or just the Standard, occurred in July 2008. Bertch, Zang and Steeleiv provided a summary review of the development process of this standard up through the start of the review by the EMB. A thorough recount of the entire development process, major issues, key decisions, and all review processes are available in Ref. v. This is the second of a pair of papers providing a summary of the final version of the Standard. Its focus is the Credibility Assessment Scale, a key feature of the Standard, including an example of its application to a real-world M&S problem for the James Webb Space Telescope. The companion paper summarizes the overall philosophy of the Standard and an overview of the requirements. Verbatim quotes from the Standard are integrated into the text of this paper, and are indicated by quotation marks.

Babula, Maria↗