Search NASA⌕ Search

SEARCH · Search NASA

Results for “model credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2008

NASA Standard for Models and Simulations: Credibility Assessment Scale

As one of its many responses to the 2003 Space Shuttle Columbia accident, NASA decided to develop a formal standard for models and simulations (M and S)ii. Work commenced in May 2005. An interim version was issued in late 2006. This interim version underwent considerable revision following an extensive Agency-wide review in 2007 along with some additional revisions as a result of the review by the NASA Engineering Management Board (EMB) in the first half of 2008. Issuance of the revised, permanent version,hereafter referred to as the M and S Standard or just the Standard, occurred in July 2008.

verification↗
NASA NTRS2015

Can Simulation Credibility Be Improved Using Sensitivity Analysis to Understand Input Data Effects on Model Outcome?

Model and simulation (MS) credibility is defined as, the quality to elicit belief or trust in MS results. NASA-STD-7009 [1] delineates eight components (Verification, Validation, Input Pedigree, Results Uncertainty, Results Robustness, Use History, MS Management, People Qualifications) that address quantifying model credibility, and provides guidance to the model developers, analysts, and end users for assessing the MS credibility. Of the eight characteristics, input pedigree, or the quality of the data used to develop model input parameters, governing functions, or initial conditions, can vary significantly. These data quality differences have varying consequences across the range of MS application. NASA-STD-7009 requires that the lowest input data quality be used to represent the entire set of input data when scoring the input pedigree credibility of the model. This requirement provides a conservative assessment of model inputs, and maximizes the communication of the potential level of risk of using model outputs. Unfortunately, in practice, this may result in overly pessimistic communication of the MS output, undermining the credibility of simulation predictions to decision makers. This presentation proposes an alternative assessment mechanism, utilizing results parameter robustness, also known as model input sensitivity, to improve the credibility scoring process for specific simulations.

Credibility↗
NASA NTRS2015

Integrated Medical Model (IMM) Project Verification, Validation, and Credibility (VVandC)

The Integrated Medical Model (IMM) Project supports end user requests by employing the Integrated Medical Evidence Database (iMED) and IMM tools as well as subject matter expertise within the Project. The iMED houses data used by the IMM. The IMM is designed to forecast relative changes for a specified set of crew health and mission success risk metrics by using a probabilistic model based on historical data, cohort data, and subject matter expert opinion. A stochastic approach is taken because deterministic results would not appropriately reflect the uncertainty in the IMM inputs. Once the IMM was conceptualized, a plan was needed to rigorously assess input information, framework and code, and output results of the IMM, and ensure that end user requests and requirements were considered during all stages of model development and implementation, as well as lay the foundation for external review and application. METHODS: In 2008, the Project team developed a comprehensive verification and validation (VV) plan, which specified internal and external review criteria encompassing 1) verification of data and IMM structure to ensure proper implementation of the IMM, 2) several validation techniques to confirm that the simulation capability of the IMM appropriately represents occurrences and consequences of medical conditions during space missions, and 3) credibility processes to develop user confidence in the information derived from the IMM. When the NASA-STD-7009 (7009) [1] was published, the Project team updated their verification, validation, and credibility (VVC) project plan to meet 7009 requirements and include 7009 tools in reporting VVC status of the IMM. Construction of these tools included meeting documentation and evidence requirements sufficient to meet external review success criteria. RESULTS: IMM Project VVC updates are compiled recurrently and include updates to the 7009 Compliance and Credibility matrices. Reporting tools have evolved over the lifetime of the IMM Project to better communicate VVC status. This has included refining original 7009 methodology with augmentation from the HRP NASA-STD-7009 Guidance Document working group and the NASA-HDBK-7009 [2]. End user requests and requirements are being satisfied as evidenced by ISS Program acceptance of IMM risk forecasts, transition to an operational model and simulation tool, and completion of service requests from a broad end user consortium including operations, science and technology planning, and exploration planning. IMM v4.0 is slated for operational release in the FY015 and current VVC assessments illustrate the expected VVC status prior to the completion of customer lead external review efforts. CONCLUSIONS: The VVC approach established by the IMM Project of incorporating Project-specific recommended practices and guidelines for implementing the 7009 requirements is comprehensive and includes the involvement of end users at every stage in IMM evolution. Methods and techniques used to quantify the VVC status of the IMM Project represented a critical communication tool in providing clear and concise suitability assessments to IMM customers. These processes have not only received approval from the local NASA community but have also garnered recognition by other federal agencies seeking to develop similar guidelines in the medical modeling community.

Walton, M.↗
NASA NTRS2013

Model Verification and Validation Concepts for a Probabilistic Fracture Assessment Model to Predict Cracking of Knife Edge Seals in the Space Shuttle Main Engine High Pressure Oxidizer

Physics-based models are routinely used to predict the performance of engineered systems to make decisions such as when to retire system components, how to extend the life of an aging system, or if a new design will be safe or available. Model verification and validation (V&V) is a process to establish credibility in model predictions. Ideally, carefully controlled validation experiments will be designed and performed to validate models or submodels. In reality, time and cost constraints limit experiments and even model development. This paper describes elements of model V&V during the development and application of a probabilistic fracture assessment model to predict cracking in space shuttle main engine high-pressure oxidizer turbopump knife-edge seals. The objective of this effort was to assess the probability of initiating and growing a crack to a specified failure length in specific flight units for different usage and inspection scenarios. The probabilistic fracture assessment model developed in this investigation combined a series of submodels describing the usage, temperature history, flutter tendencies, tooth stresses and numbers of cycles, fatigue cracking, nondestructive inspection, and finally the probability of failure. The analysis accounted for unit-to-unit variations in temperature, flutter limit state, flutter stress magnitude, and fatigue life properties. The investigation focused on the calculation of relative risk rather than absolute risk between the usage scenarios. Verification predictions were first performed for three units with known usage and cracking histories to establish credibility in the model predictions. Then, numerous predictions were performed for an assortment of operating units that had flown recently or that were projected for future flights. Calculations were performed using two NASA-developed software tools: NESSUS(Registered Trademark) for the probabilistic analysis, and NASGRO(Registered Trademark) for the fracture mechanics analysis. The goal of these predictions was to provide additional information to guide decisions on the potential of reusing existing and installed units prior to the new design certification.

Pai, Shantaram S.↗
NASA NTRSDate not supplied

Credibility Assessment and Evaluation of the Astronaut Musculoskeletal Boot/Ankle Injury Scenario Using Open SIM(TM) Computational Modeling

The musculoskeletal system of astronauts is subject to physiological changes, potentially leading to injuries due to the exposure to different gravitational environments experienced during spaceflight. These injuries can occur while an astronaut is performing an extravehicular activity (EVA) in space, on lunar or planetary surfaces or while wearing a spacesuit during terrestrial training for an EVA. One area of concern are injuries occurring at the feet and ankles due to a poor fit of the spacesuit boot. These injuries can occur if the boot does not support the ankle sufficiently and the astronaut steps awkwardly, twisting the ankle or foot, during an EVA. The OpenSimTM software can assess EVA induced musculoskeletal injuries for muscle strains, ligament injuries and joint injuries. A modeling and simulation credibility assessment of the OpenSim musculoskeletal modeling and simulation procedures, applied to the boot/ankle injury scenario injury mechanisms that may occur while carrying out tasks in these environments, is performed. These modeling and simulation credibility assessments give information on the credibility of the model’s use in simulating foot/ankle related injury mechanisms.

C A Gallo↗
NASA NTRS2008

Notional Scoring for Technical Review Weighting As Applied to Simulation Credibility Assessment

NASA's Modeling and Simulation Standard requires a credibility assessment for critical engineering data produced by models and simulations. Credibility assessment is thus a "qualifyingfactor" in reporting results from simulation-based analysis. The degree to which assessors should be independent of the simulation developers, users and decision makers is a recurring question. This paper provides alternative "weighting algorithms" for calculating the value-added for independence of the levels of technical review defined for the NASA Modeling and Simulation Standard.

Hale, Joseph Peter↗
NASA NTRS2015

Credibility Assessment of Deterministic Computational Models and Simulations for Space Biomedical Research and Operations

Human missions beyond low earth orbit to destinations, such as to Mars and asteroids will expose astronauts to novel operational conditions that may pose health risks that are currently not well understood and perhaps unanticipated. In addition, there are limited clinical and research data to inform development and implementation of health risk countermeasures for these missions. Consequently, NASA's Digital Astronaut Project (DAP) is working to develop and implement computational models and simulations (M&S) to help predict and assess spaceflight health and performance risks, and enhance countermeasure development. In order to effectively accomplish these goals, the DAP evaluates its models and simulations via a rigorous verification, validation and credibility assessment process to ensure that the computational tools are sufficiently reliable to both inform research intended to mitigate potential risk as well as guide countermeasure development. In doing so, DAP works closely with end-users, such as space life science researchers, to establish appropriate M&S credibility thresholds. We will present and demonstrate the process the DAP uses to vet computational M&S for space biomedical analysis using real M&S examples. We will also provide recommendations on how the larger space biomedical community can employ these concepts to enhance the credibility of their M&S codes.

Mulugeta, Lealem↗
NASA NTRS2008

Development of NASA's Models and Simulations Standard

From the Space Shuttle Columbia Accident Investigation, there were several NASA-wide actions that were initiated. One of these actions was to develop a standard for development, documentation, and operation of Models and Simulations. Over the course of two-and-a-half years, a team of NASA engineers, representing nine of the ten NASA Centers developed a Models and Simulation Standard to address this action. The standard consists of two parts. The first is the traditional requirements section addressing programmatics, development, documentation, verification, validation, and the reporting of results from both the M&S analysis and the examination of compliance with this standard. The second part is a scale for evaluating the credibility of model and simulation results using levels of merit associated with 8 key factors. This paper provides an historical account of the challenges faced by and the processes used in this committee-based development effort. This account provides insights into how other agencies might approach similar developments. Furthermore, we discuss some specific applications of models and simulations used to assess the impact of this standard on future model and simulation activities.

simulation↗
NASA NTRS2006

An Overview of NASA's Program of Future M&S VV&A Outreach and Training Activities

NASA's Exploration Systems Mission Directorate (ESMD) is implementing a management approach for modeling and simulation (M&S) that will provide decision-makers information on the model s fidelity, credibility, and quality. The Integrated Modeling & Simulation Verification, Validation and Accreditation (IM&S W&A) process will allow the decision-maker to understand the risks involved in using a model s results for mission-critical decisions. The W&A Technical Working Group (W&A TWG) has been identified to communicate this process throughout the agency. As the W&A experts, the W&A NVG will be the central resource for support of W&A policy, procedures, training and templates for documentation. This presentation will discuss the W&A Technical Working Group s outreach approach aimed at educating M&S program managers, developers, users and proponents on the W&A process, beginning at MSFC with the CLV program.

Caine, Lisa↗
NASA NTRS1993

Wind-forced variability of the deep eastern north Pacific: Observations of seafloor pressure and abyssal currents

Data from an array of bottom pressure gauges and a string of current meters in the vicinity of 47 deg N, 139 deg W, are used to examine the deep-ocean variability forced by ocean surface wind stress curl from August 1987 to June 1988. Bottom geostrophic currents are computed from the pressure gauge array, and these correspond well to the long-period directly measured currents at 3000 m. The supratidal-period bottom pressure variations are coherent at 95% confidence with the wind stress curl in period bands of 3-4 days and 15-60 days but removed in distances of 400 and 700 km to the northwest and the southeast, respectively. A linear, two-layer hydrodynamic model is used to examine the theoretical forcing produced by random-phased surface wind fields for the conditions of the eastern north Pacific and the 15- to 60-day-period observed response is reproduced credibly. To model 3- to 15-day variations, more realistic models are required.

Niller, P. P.↗
NASA NTRS1992

Component performance requirements

Component data are essential for modelling the overall performance of gas turbines. The component characteristics are not easily obtained, and much of the data are proprietary and not available in the open literature. Several methods are available for estimating component characteristics and are briefly described. The requirements of users and manufacturers are quite different, but both can produce fully credible performance models.

H I H Saravanamuttoo↗
NASA NTRS1996

The Effect of Wake Passing on Turbine Blade Film Cooling

The effect of upstream blade row wake passing on the showerhead film cooling performance of a downstream turbine blade has been investigated through a combination of experimental and computational studies. The experiments were performed in a steady-flow annular turbine cascade facility equipped with an upstream rotating row of cylindrical rods to produce a periodic wake field similar to that found in an actual turbine. Spanwise, chordwise, and temporal resolution of the blade surface temperature were achieved through the use of an array of nickel thin-film surface gauges covering one unit cell of showerhead film hole pattern. Film effectiveness and Nusselt number values were determined for a test matrix of various injectants, injectant blowing ratios, and wake Strouhal numbers. Results indicated a demonstratable reduction in film effectiveness with increasing Strouhal number, as well as the expected increase in film effectiveness with blowing ratio. An equation was developed to correlate the span-average film effectiveness data. The primary effect of wake unsteadiness was found to be correlated well by a chordwise-constant decrement of 0.094-St. Measurable spanwise film effectiveness variations were found near the showerhead region, but meaningful unsteady variations and downstream spanwise variations were not found. Nusselt numbers were less sensitive to wake and injection changes. Computations were performed using a three-dimensional turbulent Navier-Stokes code which was modified to model wake passing and film cooling. Unsteady computations were found to agree well with steady computations provided the proper time-average blowing ratio and pressure/suction surface flow split are matched. The remaining differences were isolated to be due to the enhanced mixing in the unsteady solution caused by the wake sweeping normally on the pressure surface. Steady computations were found to be in excellent agreement with experimental Nusselt numbers, but to overpredict experimental film effectiveness values. This is likely due to the inability to match actual hole exit velocity profiles and the absence of a credible turbulence model for film cooling.

Heidmann, James David↗
NASA NTRS1994

Retrieval of sizes and total masses of particles in volcanic clouds using AVHRR bands 4 and 5

The advanced very high resolution radiometer (AVHRR) sensor on polar orbiting NOAA satellites can discriminate between volcanic clouds and meteorological clouds using two-band data in the thermal infrared. This paper is aimed at developing a retrieval of the particle sizes, optical depth, and total masses of particles from AVHRR two-band data of volcanic clouds. Radiative transfer calculations are used with a semi-transparent cloud model that is based on assumptions of spherical particle shape, a homogeneous underlying surface, and a simple thin cloud parallel to the surface. The model is applied to observed AVHRR data from a 13-hour old drifting cloud from the August 19, 1992, eruption of Crater Peak/Spurr Volcano, Alaska. The AVHRR data fit in the range of results calculated by the model, which supports its credibility. According to the model results, the average of effective particle radius in the test frame of this cloud is in the range of 2 to 2.5 micrometer, the optical depth at 12 micrometer is about 0.60 - 0.65. The total estimated mass of ash in the air amounts to 0.24 - 0.31 x 10(exp 6) tons, which is about 0.7-0.9% of the mass measured in the ashfall blanket. Sensitivity tests show that the mass estimate is more sensitive to the assumed ash size distribution than it is to the ash composition.

Wen, Shiming↗
NASA NTRS1997

An Experimental Study of the Effect of Wake Passing on Turbine Blade Film Cooling

The effect of wake passing on the showerhead film cooling performance of a turbine blade has been investigated experimentally. The experiments were performed in an annular turbine cascade with an upstream rotating row of cylindrical rods. Nickel thin-film gauges were used to determine local film effectiveness and Nusselt number values for various injectants, blowing ratios, and Strouhal numbers. Results indicated a reduction in film effectiveness with increasing Strouhal number, as well as the expected increase in film effectiveness with blowing ratio. An equation was developed to correlate the span-average film effectiveness data. The primary effect of wake unsteadiness was found to be correlated by a streamwise-constant decrement of 0.094.St. Steady computations were found to be in excellent agreement with experimental Nusselt numbers, but to overpredict experimental film effectiveness values. This is likely due to the inability to match actual hole exit velocity profiles and the absence of a credible turbulence model for film cooling.

Heidmann, James D.↗
NASA NTRS2014

Comparison of the Results of MISSE 6 Atomic Oxygen Erosion Yields of Layered Kapton H Films with Monte Carlo Computational Predictions

A space experiment flown as part of the Materials International Space Station Experiment 6B (MISSE 6B) was designed to compare the atomic oxygen erosion yield (Ey) of layers of Kapton H polyimide with no spacers between layers with that of layers of Kapton H with spacers between layers. The results were compared to a solid Kapton H (DuPont, Wilmington, DE) sample. Monte Carlo computational modeling was performed to optimize atomic oxygen interaction parameter values to match the results of both the MISSE 6B multilayer experiment and the undercut erosion profile from a crack defect in an aluminized Kapton H sample flown on the Long Duration Exposure Facility (LDEF). The Monte Carlo modeling produced credible agreement with space results of increased Ey for all samples with spacers as well as predicting the space-observed enhancement in erosion near the edges of samples due to scattering from the beveled edges of the sample holders.

Oxygen atoms↗
NASA NTRS2022

Unique Observational Constraints on the Seasonal and Longitudinal Variability of the Earth’s Planetary Albedo and Cloud Distribution Inferred From EPIC Measurements

Thorough comparison to observations is key to developing a credible climate model forecasting capability. Deep Space Climate Observatory (DSCOVR) measurements of Earth’s reflected solar and emitted thermal radiation provide a unique observational perspective that permits a more reliable model/data comparison than is possible with the otherwise available satellite data. The uniqueness is in the DSCOVR satellite’s viewing geometry, which enables continuous viewing of the Earth’s sunlit hemisphere from its Lissajous orbit around the Lagrangian L1 point. The key instrument is the Earth Polychromatic Imaging Camera (EPIC), which views the Earth’s sunlit hemisphere with 1024-by-1024-pixel imagery in 10 narrow spectral bands from 317 to 780 nm, acquiring up to 22 high spatial resolution images per day. The additional feature is that the frequency of EPIC image acquisition is nearly identical to that of the climate GCM data generation scheme where climate data for the entire globe are ‘instantaneously’ calculated at 1-h radiation time-step intervals. Implementation of the SHS (Sunlit Hemisphere Sampling) EPIC-view geometry for the in-line GCM output data sampling establishes a precise self-consistency in the space-time data sampling between EPIC observational and GCM output data generation and sampling. The remaining problem is that the GCM generated data are radiative fluxes, while the EPIC measurements are backscatter-dependent radiances. Radiance to flux conversion is a complex problem with no simple way to convert GCM radiative fluxes into spectral radiances. The more expedient approach is to convert the EPIC spectral radiances into broadband radiances by MODIS/CERES-based regression relationships and then into solar radiative fluxes using the CERES angular distribution models. Averaging over the sunlit hemisphere suppresses the meteorological weather noise, but preserves the intra-seasonal larger scale variability. Longitudinal slicing by the Earth’s rotation permits a self-consistent model/data comparison of the longitudinal model/data differences in the variability of the reflected solar radiation. Ancillary EPIC Composite data provide additional cloud property information for climate model diagnostics. Comparison of EPIC-derived seasonal and longitudinal variability of the Earth’s planetary albedo with the GISS ModelE2 results shows systematic overestimate of cloud reflectivity over the Pacific Ocean with corresponding underestimates over continental land areas.

DSCOVR↗
NASA NTRSDate not supplied

Quantifying the Sensitivity of Condition Incidence Parameters in the Evidence Library

One approach to quantifying spaceflight risk at NASA makes use event driven probabilistic techniques. The Medical Extensible Dynamic Probabilistic Risk Assessment Tool (MEDPRAT) is such a tool that estimates medical risk metrics via simulation and enables optimization of medical resources subject to mission constraints [1]. Previous analyses have informed medical set composition, exercise countermeasures, and water intake, where each analysis quantifies the risk associated with proposed variations in system design. As future mission profiles extend beyond Low-Earth Orbit (LEO) and lengthen in duration, understanding these risks and contributing factors is critical. MEDPRAT employs Monte Carlo sampling techniques to simulate missions and track the occurrence of medical events. These events follow fault-tree-like progressions through levels of severity and mitigation via medical treatment to many possible outcomes and these are reported throughout the mission. Making this possible, are the medical databases that contain evidence gathered by the Human Research Program (HRP). Quantifying the impact of uncertainty or variability in the input data is an important step in evaluating the credibility of modeling and simulation results. In this work, we investigate the sensitivity of medical risk metrics with respect to the condition incidence parameters within the Evidence Library (EL) [2] as the medical database input for MEDPRAT. The medical conditions, contained in the EL, are equipped with incidence rates that describe the likelihood that the condition will occur. These incidence rates reflect historical spaceflight data or when appropriate, terrestrial data. In this presentation, we will explore how uncertainty in these rates propagate to the medical risk described by MEDPRAT. These results identify the conditions and parameters with the largest contribution to medical risks.

Ian Lim↗