SEARCH · Search NASA
Results for “model credibility”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Establishing model credibility for process-microstructure-property relationships in additive manufacturing using exascale computing
Additive Manufacturing (AM) of alloys holds significant promise as a disruptive technology in various industries, yet its adoption is often hindered by challenges in achieving consistent part quality. These issues are primarily due to the complex process-microstructure-property (PSP) relationships inherent to AM. Computational models can greatly aid in understanding these relationships, but their widespread impact and adoption has been limited by a lack of validated, open-source, and computationally efficient PSP modeling frameworks and hardware limitations. Here, this study leverages the ExaAM software suite and data from the AMBench-2018 series of laser powder bed fusion (LPBF) benchmark experiments to perform a comprehensive model assessment, including verification, validation, sensitivity analysis, and uncertainty quantification. The RADICAL-EnTK workflow manager was used to perform an ensemble of heat transport, solidification, and mechanical response simulations on the exascale computer Frontier, considering uncertainties in critical model inputs such as laser spot size and nucleation parameters, and consisting of 125 explicit grain structure simulations and 7875 crystal plasticity simulations. For a selected location within the Inconel 625 AMBench-2018 test artifact, sensitivity analysis and uncertainty quantification were performed using the predicted distributions of grain structure and mechanical properties. Qualitative agreement was found between the predicted grain size and texture and the observed AMBench-2018 microstructure, the mean predicted yield stress was within 5% of the experimental measurement mean, and the mean predicted engineering stress at 5% strain was within 10% of the experimental measurement mean. The insights gained from development and validation of the ExaAM PSP modeling framework will help guide future directions for enhancing the credibility and reliability of PSP models in AM, thereby accelerating the adoption of AM technologies in various industries.
An Extension to the Predictive Capability Maturity Model to Assess Model Credibility Using Dempster-Shafer Theory
Explore the source record for details and available documents.
A Demonstration of the Quantitative Approach for Model Credibility on an Electromagnetic Application
Explore the source record for details and available documents.
Model credibility and risk quantification
Explore the source record for details and available documents.
Assessing the Accuracy of ML Explanations for Model Credibility.
Abstract not provided.
Robust Assessment of Surrogate Model Credibility
Explore the source record for details and available documents.
What Questions Would a Systems Engineer Ask to Assess Systems Engineering Models as Credible
Digital Systems Engineering strategies typically call for digital Systems Engineering models to be retained in repositories and certified as an authoritative source of truth (enabling model reuse, qualification, and collaboration). In order for digital Systems Engineering models to be certified as authoritative (credible), they need to be assessed - verified and validated - and with the amount of uncertainty in the model quantified (consider reusing someone else's model without knowing the author). Due to this increasing model complexity, the authors assert that traditional human-based methods for validating, verifying, and uncertainty quantification - such as human-based peer-review sessions - cannot sufficiently establish that a digital Systems Engineering model of a complex system is credible. Digital Systems Engineering models of complex systems can contain millions of nodes and edges. The authors assert that this level of detail is beyond the ability of any group of humans - even working for weeks at a time - to discern and catch every minor model infraction. In contrast, computers are highly effective at discerning infractions with massive amounts of information. The authors suggest that a better approach might be to focus the humans at what model patterns should be assessed and enable the computer to assess the massive details in accordance with those patterns - by running through perhaps 100,000 test loops. In anticipation of future projects to implement and automate the assessment of models at Sandia National Laboratories, a study was initiated to elicit input from a group of 25 Systems Engineering experts. The authors positioning query began with - 'What questions would a Systems Engineer ask to assess Systems Engineering models for credibility?" This report documents the results of that survey.
Modeling and Simulation Credibility Assessment of Finite Element Models for Fall From Heights Injury Scenarios
Finite element (FE) computational models of the whole human body are being used to investigate the injury modalities and mechanisms arising from suited astronauts’ ExtraVehicular Activities (EVAs). These FE whole body models – simplified Elemance and THUMS pedestrian versions – have undergone a series of verification and validation (V&V) assessments for applications in automotive, domestic, and contact sports arenas. However, the FE models’ Modeling and Simulation (M&S) credibility assessments for simulating suited astronaut injuries due to fall from heights in altered gravity environments are limited. The current study evaluates the M&S credibility of simplified Elemance and THUMS pedestrian FE models based on eight credibility factors – data pedigree, input pedigree, code verification, solution verification, conceptual validation, referent validation, results uncertainty, and results robustness – as per NASA-STD-7009A [1]. These credibility factors were assigned ordinal scores ranging between 0 to 4, with a score of 0 in the case of insufficient evidence and 4 when all data for the model is known, used appropriately, and traceable to the Real-World System (RWS) with acceptable accuracy, precision, and uncertainty. The credibility factor sufficiency thresholds were assigned upon guidance from NASA EVA subject matter experts. The factor scores were ascertained through the evaluation of M&S credibility assessment procedures described in the published literature and FE model reports. The M&S credibility scores for the Elemance and THUMS FE models vary between 1 and 3 for data pedigree, input pedigree, and referent validation credibility factors, and between 0 and 1 for code verification, solution verification, conceptual validation, results uncertainty and results robustness. The results show that these FE models are credible for simulating fall from heights injury scenarios in specific kinematic and kinetic ranges that correlate with highway and contact sports-related external conditions. Multiple credibility elevation strategies are prescribed to improve the Elemance and THUMS models’ credibility for use in the EVA fall from height injury scenarios.
Modeling and Simulation Credibility Assessments of Musculoskeletal Computational Models for Simulating Astronaut Injuries Due to a Poor Spacesuit Fit
The musculoskeletal (MS) system of astronauts is subject to physiological changes, potentially leading to injuries due to the exposure to different gravitational environments experienced during spaceflight. These injuries can occur while an astronaut is performing an Extravehicular Activity (EVA) in space, on lunar or planetary surfaces or while wearing a spacesuit during terrestrial training for an EVA. The OpenSim MS modeling software can assess EVA induced MS injury mechanisms such as muscle strains, ligament injuries and joint injuries. One area of concern, since there are only a few different spacesuit sizes with limited adjustability, is the possibility of a poorly fitting spacesuit. This can cause unnatural joint motions and torques resulting in various MS injuries. A credibility assessment of the OpenSim modeling and simulation procedures is performed per NASA-STD-7009A to provide information on the credibility of the model’s use in simulating EVA related injury mechanisms. The credibility assessment evaluated various OpenSim models against the following eight credibility factors: data pedigree, input pedigree, code verification, solution verification, conceptual validation, referent validation, results uncertainty and results robustness (sensitivity). The models evaluated for EVA injuries will require additional credibility factor analysis and upgrades to the model features, such as adding ligaments to a whole-body model, to reliably predict and analyze the EVA injuries expected to occur due to a poor spacesuit fit. The degree of elevation strategy required to increase the credibility assessment scores will depend on the model complexity and the injury mechanism.
Modeling and Simulation Credibility Assessments of Musculoskeletal Computational Models for Simulating Astronaut Injuries Due to a Poor Spacesuit Fit
The musculoskeletal (MS) system of astronauts is subject to physiological changes, potentially leading to injuries due to the exposure to different gravitational environments experienced during spaceflight. These injuries can occur while an astronaut is performing an Extravehicular Activity (EVA) in space, on lunar or planetary surfaces or while wearing a spacesuit during terrestrial training for an EVA. The OpenSim MS modeling software can assess EVA induced MS injury mechanisms such as muscle strains, ligament injuries and joint injuries. One area of concern, since there are only a few different spacesuit sizes with limited adjustability, is the possibility of a poorly fitting spacesuit. This can cause unnatural joint motions and torques resulting in various MS injuries. A credibility assessment of the OpenSim modeling and simulation procedures is performed per NASA-STD-7009A to provide information on the credibility of the model’s use in simulating EVA related injury mechanisms. The credibility assessment evaluated various OpenSim models against the following eight credibility factors: data pedigree, input pedigree, code verification, solution verification, conceptual validation, referent validation, results uncertainty and results robustness (sensitivity). The models evaluated for EVA injuries will require additional credibility factor analysis and upgrades to the model features, such as adding ligaments to a whole-body model, to reliably predict and analyze the EVA injuries expected to occur due to a poor spacesuit fit. The degree of elevation strategy required to increase the credibility assessment scores will depend on the model complexity and the injury mechanism.
Credible practice of modeling and simulation in healthcare: ten rules from a multidisciplinary perspective
The complexities of modern biomedicine are rapidly increasing. Thus, modeling and simulation have become increasingly important as a strategy to understand and predict the trajectory of pathophysiology, disease genesis, and disease spread in support of clinical and policy decisions. In such cases, inappropriate or ill-placed trust in the model and simulation outcomes may result in negative outcomes, and hence illustrate the need to formalize the execution and communication of modeling and simulation practices. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the devel-opment and reuse of the models. For the past several years, the Committee on Credible Practice of Modeling and Simulation in Healthcare, an interdisciplinary group seeded from a U.S. interagency initiative, has worked to codify best practices. Here, we provide Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative analysis by the Committee’s multidisciplinary membership, followed by a large stakeholder com-munity survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. While biomedical science and clinical care domains have somewhat different requirements and expectations for credible practice, our study converged on rules that would be useful across a broad swath of model types. In brief, the rules are: (1) Define context clearly. (2) Use contextually appropriate data. (3) Evaluate within context. (4) List limitations explicitly. (5) Use version control. (6) Document appropriately. (7) Disseminate broadly. (8) Get independent reviews. (9) Test competing imple-mentations. (10) Conform to standards. Although some of these are common sense guidelines, we have found that many are often missed or misconstrued, even by seasoned practitioners. Computational models are already widely used in basic science to generate new biomedical knowledge. As they penetrate clinical care and healthcare policy, contributing to personalized and precision medicine, clinical safety will require established guidelines for the credible practice of modeling and simulation in healthcare.
Dimensions of Credibility in Models and Simulations
Based on the National Aeronautics and Space Administration's (NASA's) work in developing a standard for models and simulations (M&S), the subject of credibility in M&S became a distinct focus. This is an indirect result from the Space Shuttle Columbia Accident Investigation Board (CAIB), which eventually resulted in an action, among others, to improve the rigor in NASA's M&S practices. The focus of this action came to mean a standardized method for assessing and reporting results from any type of M&S. As is typical in the standards development process, this necessarily developed into defming a common terminology base, common documentation requirements (especially for M&S used in critical decision making), and a method for assessing the credibility of M&S results. What surfaced in the development of the NASA Standard was the various dimensions credibility to consider when accepting the results from any model or simulation analysis. The eight generally relevant factors of credibility chosen in the NASA Standard proved only one aspect in the dimensionality of M&S credibility. At the next level of detail, the full comprehension of some of the factors requires an understanding along a couple of dimensions as well. Included in this discussion are the prerequisites for the appropriate use of a given M&S, the choice of factors in credibility assessment with their inherent dimensionality, and minimum requirements for fully reporting M&S results.
Material Models and Credibility for System Level Abnormal Mechanical ModSim Applications
The purpose of this document is to provide evidence for assessing the adequacy of parameterized material models for a collection of materials used in a finite element analyses setting. “Adequacy” is relative to the intended use of the material in particular analyses. The intended application of the material models covered within this document is for system level abnormal mechanical solid mechanics analyses. Generally, material model parameterizations should be valid from temperatures of approximately -50 to 70° C, across a range of strain rates, and (depending on details of the parts involved) large deformations. Each material covered in this document is presented in its own chapter with a common format across materials. Model assumptions, limitations, existing validation results, readiness for use with uncertainty quantification, general usage guidance, and failure considerations are all provided along with specific parameterization inputs suitable for the finite element analysis code Sierra/Solid Mechanics.
Modeling and Simulation Credibility Assessments of Whole-Body Finite Element Computational Models for Use in NASA Extravehicular Activity Applications
Computational finite element (FE) models are used in suited astronaut injury risk assessments; however, these models’ verification, validation, and credibility (VV&C) procedures for simulating injuries in altered gravity environments are limited. Our study conducts VV&C assessments of THUMS and Elemance whole-body FE models for predicting suited astronaut injury biomechanics using eight credibility factors, as per NASA-STD-7009A. Credibility factor ordinal scores are assigned by reviewing existing documentation describing VV&C practices, and credibility sufficiency thresholds are assigned based on input from subject matter experts. Our results show the FE models are credible for suited astronaut injury investigation in specific ranges of kinematic and kinetic conditions correlating to highway and contact sports events. Nevertheless, these models are deficient when applied outside these ranges. Several credibility elevation strategies are prescribed to improve models’ credibility for the NASA-centric application domain.
Modeling and simulation credibility assessments of whole-body finite element computational models for use in NASA extravehicular activity applications
Computational finite element (FE) models are used in suited astronaut injury risk assessments; however, these models’ verification, validation, and credibility (VV&C) procedures for simulating injuries in altered gravity environments are limited. Our study conducts VV&C assessments of THUMS and Elemance whole-body FE models for predicting suited astronaut injury biomechanics using eight credibility factors, as per NASA-STD-7009A. Credibility factor ordinal scores are assigned by reviewing existing documentation describing VV&C practices, and credibility sufficiency thresholds are assigned based on input from subject matter experts. Our results show the FE models are credible for suited astronaut injury investigation in specific ranges of kinematic and kinetic conditions correlating to highway and contact sports events. Nevertheless, these models are deficient when applied outside these ranges. Several credibility elevation strategies are prescribed to improve models’ credibility for the NASA-centric application domain.
What Questions Would a Systems Engineer Ask to Assess MBSE Models as Credible?.
Abstract not provided.