Results for “Simulation Credibility” — Page 5

NASA NTRS2015

Integrated Medical Model (IMM) Project Verification, Validation, and Credibility (VVandC)

The Integrated Medical Model (IMM) Project supports end user requests by employing the Integrated Medical Evidence Database (iMED) and IMM tools as well as subject matter expertise within the Project. The iMED houses data used by the IMM. The IMM is designed to forecast relative changes for a specified set of crew health and mission success risk metrics by using a probabilistic model based on historical data, cohort data, and subject matter expert opinion. A stochastic approach is taken because deterministic results would not appropriately reflect the uncertainty in the IMM inputs. Once the IMM was conceptualized, a plan was needed to rigorously assess input information, framework and code, and output results of the IMM, and ensure that end user requests and requirements were considered during all stages of model development and implementation, as well as lay the foundation for external review and application. METHODS: In 2008, the Project team developed a comprehensive verification and validation (VV) plan, which specified internal and external review criteria encompassing 1) verification of data and IMM structure to ensure proper implementation of the IMM, 2) several validation techniques to confirm that the simulation capability of the IMM appropriately represents occurrences and consequences of medical conditions during space missions, and 3) credibility processes to develop user confidence in the information derived from the IMM. When the NASA-STD-7009 (7009) [1] was published, the Project team updated their verification, validation, and credibility (VVC) project plan to meet 7009 requirements and include 7009 tools in reporting VVC status of the IMM. Construction of these tools included meeting documentation and evidence requirements sufficient to meet external review success criteria. RESULTS: IMM Project VVC updates are compiled recurrently and include updates to the 7009 Compliance and Credibility matrices. Reporting tools have evolved over the lifetime of the IMM Project to better communicate VVC status. This has included refining original 7009 methodology with augmentation from the HRP NASA-STD-7009 Guidance Document working group and the NASA-HDBK-7009 [2]. End user requests and requirements are being satisfied as evidenced by ISS Program acceptance of IMM risk forecasts, transition to an operational model and simulation tool, and completion of service requests from a broad end user consortium including operations, science and technology planning, and exploration planning. IMM v4.0 is slated for operational release in the FY015 and current VVC assessments illustrate the expected VVC status prior to the completion of customer lead external review efforts. CONCLUSIONS: The VVC approach established by the IMM Project of incorporating Project-specific recommended practices and guidelines for implementing the 7009 requirements is comprehensive and includes the involvement of end users at every stage in IMM evolution. Methods and techniques used to quantify the VVC status of the IMM Project represented a critical communication tool in providing clear and concise suitability assessments to IMM customers. These processes have not only received approval from the local NASA community but have also garnered recognition by other federal agencies seeking to develop similar guidelines in the medical modeling community.

Walton, M.↗

NASA NTRS1984

Reliability and maintainability assessment factors for reliable fault-tolerant systems

A long term goal of the NASA Langley Research Center is the development of a reliability assessment methodology of sufficient power to enable the credible comparison of the stochastic attributes of one ultrareliable system design against others. This methodology, developed over a 10 year period, is a combined analytic and simulative technique. An analytic component is the Computer Aided Reliability Estimation capability, third generation, or simply CARE III. A simulative component is the Gate Logic Software Simulator capability, or GLOSS. The numerous factors that potentially have a degrading effect on system reliability and the ways in which these factors that are peculiar to highly reliable fault tolerant systems are accounted for in credible reliability assessments. Also presented are the modeling difficulties that result from their inclusion and the ways in which CARE III and GLOSS mitigate the intractability of the heretofore unworkable mathematics.

Bavuso, S. J.↗

NASA NTRS1990

Progress in hypersonic combustion technology with computation and experiment

Design of successful airbreathing engines for operation at near-orbital speeds presents significant challenges in all the disciplines involved, including propulsion. This paper presents a discussion of the important physics of hypersonic combustion and an assessment of the state of the art of ground simulations with pulse facilities and with computational techniques. Recent examples of experimental and computational simulations are presented and discussed. The need for continued application of these tools to establish the credibility and fidelity of engineering design methods for practical hypersonic combustors is emphasized along with the critical need for improved diagnostic methods for hypervelocity reacting flows.

Anderson, Griffin Y.↗

NASA NTRS2015

Conclusion: Major Findings and Future Activities.

The Agricultural Model Intercomparison and Improvement Project (AgMIP) incorporates a number of major advances in the way that climate impacts, vulnerability, and adaptation are being simulated. At its core is a protocol approach that results in impact assessments being more scientifically credible and thus ultimately having greater value to the wide range of agricultural stakeholders. Moreover, the use of the protocol approach enables closer scrutiny and intercomparison of models and methods so that they can be improved over time. By creating a truly trans-disciplinary, systems-based approach, AgMIP impact assessments and evaluation of adaptations become useful to agricultural decision-makers at multiple scales. The chapters in this two-part set demonstrate the use of this approach and represent early steps towards the full realization of these new methods and their application.

damage assessment↗

NASA NTRS2021

Evolution of Tropical Cyclone Properties Across the Development Cycle of the GISS-E3 Global Climate Model

The next-generation global climate model from the NASA Goddard Institute for Space Studies, GISS-E3, contains many improvements to resolution and physics that allow for improved representation of tropical cyclones (TCs) in the model. This study examines the properties of TCs in two different versions of E3 at different points in its development cycle, run for 20 years at 0.5 degree resolution, and compares these TCs with observations, the previous generation GISS model, E2, and other climate models. E3 shares many TC biases common to global climate models, such as having too few tropical cyclones, but is much improved from E2. E3 produces strong enough TCs that observation-based wind speed thresholds can now be used to detect and track them, and some storms now reach hurricane intensity; neither of these was true of E2. Model development between the first and second versions of E3 further increased the number and intensity of TCs and reduced TC count biases globally and in most regions. One-year sensitivity tests to changes in various microphysical and dynamical tuning parameters are also examined. Increasing the entrainment rate for the more strongly-entraining plume in the convection scheme increases the number of TCs (though also affecting other climate variables, and in some cases increasing biases). Variations in divergence damping did not have a strong effect on simulated TC properties, contrary to expectations based on previous studies. Overall, the improvements in E3 make it more credible for studies of TC activity and its relationship to climate.

Tropical Cyclones↗

NASA NTRS2018

Validation of the Nasa Integrated Medical Model: a Space Flight Medical Risk Prediction Tool

The Human Research Program funded the development of the Integrated Medical Model (IMM) to quantify the medical component of overall mission risk. The IMM uses Monte Carlo simulation methodology, incorporating space flight and ground medical data, to estimate the probability of mission medical outcomes and resource utilization. To determine the credibility of IMM output, the IMM project team completed two validation studies that compared IMM predicted output to observed medical events from a selection of Shuttle Transportation System (STS) and International Space Station (ISS) missions. The validation study results showed that the IMM underpredicted the occurrence of ~10% of the modeled medical conditions for the STS missions and overpredicted ~20% of the modeled medical conditions for the ISS missions. These findings imply that the strength of IMM predictions to inform decisions depends on simulated mission specifications including length. This discrepancy could result from medical recording differences between ISS and STS that possibly influence observed incidence rates, IMM combining all "mission type" data as constant occurrence rate or fixed proportion across both mission types, misspecification of symptoms to conditions, and gaps in the literature informing the model. Some of these issues will be alleviated by updating the IMM source data through incorporation of the observed validation data.

medical equipment↗

NASA NTRS2018

Validation of the NASA Integrated Medical Model: a Space Flight Medical Risk Prediction Tool

The Human Research Program funded the development of the Integrated Medical Model (IMM) to quantify the medical component of overall mission risk. The IMM uses Monte Carlo simulation methodology, incorporating space flight and ground medical data, to estimate the probability of mission medical outcomes and resource utilization. To determine the credibility of IMM output, the IMM project team completed two validation studies that compared IMM predicted output to observed medical events from a selection of Shuttle Transportation System (STS) and International Space Station (ISS) missions. The validation study results showed that the IMM underpredicted the occurrence of ~10% of the modeled medical conditions for the STS missions and overpredicted ~20% of the modeled medical conditions for the ISS missions. These findings imply that the strength of IMM predictions to inform decisions depends on simulated mission specifications including length. This discrepancy could result from medical recording differences between ISS and STS that possibly influence observed incidence rates, IMM combining all "mission type" data as constant occurrence rate or fixed proportion across both mission types, misspecification of symptoms to conditions, and gaps in the literature informing the model. Some of these issues will be alleviated by updating the IMM source data through incorporation of the observed validation data.

risk assessment↗

NASA NTRS1991

CFD code validation for nozzle flowfields

Renewed interest in hypersonic airbreathing flight has resulted in heavy reliance on computational fluid dynamics for all aspects of aerodynamic and scramjet component modeling. Credible numerical performance predictions require calibration to unit type flowfields. This paper presents results of one such calibration to nonreacting simulated scramjet nozzle flowfields. The experimental test program was performed on a static thrust stand. Computed results are in excellent agreement with measured wall pressure, pitot pressure profiles, and thrust. Agreement with thrust is shown to be within the reported accuracy of the experimental measurements.

Jentink, Thomas N.↗

NASA NTRS1992

The Atmospheric Effects of Stratospheric Aircraft: a First Program Report

Studies have indicated that, with sufficient technology development, high speed civil transport aircraft could be economically competitive with long haul subsonic aircraft. However, uncertainty about atmospheric pollution, along with community noise and sonic boom, continues to be a major concern; and this is addressed in the planned 6 yr HSRP begun in 1990. Building on NASA's research in atmospheric science and emissions reduction, the AESA studies particularly emphasizing stratospheric ozone effects. Because it will not be possible to directly measure the impact of an HSCT aircraft fleet on the atmosphere, the only means of assessment will be prediction. The process of establishing credibility for the predicted effects will likely be complex and involve continued model development and testing against climatological patterns. Lab simulation of heterogeneous chemistry and other effects will continue to be used to improve the current models.

Prather, Michael J.↗

NASA NTRS1997

Computational Infrastructure for Engine Structural Performance Simulation

Select computer codes developed over the years to simulate specific aspects of engine structures are described. These codes include blade impact integrated multidisciplinary analysis and optimization, progressive structural fracture, quantification of uncertainties for structural reliability and risk, benefits estimation of new technology insertion and hierarchical simulation of engine structures made from metal matrix and ceramic matrix composites. Collectively these codes constitute a unique infrastructure readiness to credibly evaluate new and future engine structural concepts throughout the development cycle from initial concept, to design and fabrication, to service performance and maintenance and repairs, and to retirement for cause and even to possible recycling. Stated differently, they provide 'virtual' concurrent engineering for engine structures total-life-cycle-cost.

Chamis, Christos C.↗

NASA NTRS2007

Use of Remote Sensing and Dust Modelling to Evaluate Ecosystem Phenology and Pollen Dispersal

The impact of pollen release and downwind concentrations can be evaluated utilizing remote sensing. Previous NASA studies have addressed airborne dust prediction systems PHAiRS (Public Health Applications in Remote Sensing) which have determined that pollen forecasts and simulations are possible. By adapting the deterministic dust model (as an in-line system with the National Weather Service operational forecast model) used in PHAiRS to simulate downwind dispersal of pollen, initializing the model with pollen source regions from MODIS, assessing the results a rapid prototype concept can be produced. We will present the results of our effort to develop a deterministic model for predicting and simulating pollen emission and downwind concentration to study details or phenology and meteorology and their dependencies, and the promise of a credible real time forecast system to support public health and agricultural science and service. Previous studies have been done with PHAiRS research, the use of NASA data, the dust model and the PHAiRS potential to improve public health and environmental services long into the future.

Luvall, Jeffrey C.↗

NASA NTRS1991

Atmospheric effects of stratospheric aircraft - A status report from NASA's High-Speed Research Program

Studies have indicated that, with sufficient technology development, future high-speed civil transport aircraft could be economically competitive with long-haul subsonic aircraft. However, uncertainty about atmospheric pollution, along with community noise and sonic boom, continues to be a major concern which is being addressed in the planned six-year High-Speed Research Program begun in 1990. Building on NASA's research in atmospheric science and emissions reduction, current analytical predictions indicate that an operating range may exist at altitudes below 20 km (i.e., corresponding to a cruise Mach number of approximately 2.4) where the goal level of 5 gm equivalent NO2 emissions/kg fuel will deplete less than one percent of column ozone. Because it will not be possible to directly measure the impact of an aircraft fleet on the atmosphere, the only means of assessment will be prediction. The process of establishing credibility for the predicted effects will likely be complex and involve continued model development and testing against climatological patterns. In particular, laboratory simulation of heterogeneous chemistry and other effects, and direct measurements of well understood tracers in the troposphere and stratosphere are being used to improve the current models.

Wesoky, Howard L.↗

NASA NTRS2014

Verification, Validation and Credibility Assessment of a Computational Model of the Advanced Resistive Exercise Device (ARED)

The Advanced Resistive Exercise Device (ARED) is the resistive exercise device used by astronauts on the International Space Station (ISS) to mitigate bone loss and muscle atrophy due to extended exposure to microgravity (micro g). The Digital Astronaut Project (DAP) has developed a multi-body dynamics model of biomechanics models for use in spaceflight exercise physiology research and operations. In an effort to advance model maturity and credibility of the ARED model, the DAP performed verification, validation and credibility (VV and C) assessment of the analyses of the model in accordance to NASA-STD-7009 'Standards for Models and Simulations'.

Werner, C. R.↗

NASA NTRS2012

NASA Integrated Model Centric Architecture (NIMA) Model Use and Re-Use

This whitepaper accepts the goals, needs and objectives of NASA's Integrated Model-centric Architecture (NIMA); adds experience and expertise from the Constellation program as well as NASA's architecture development efforts; and provides suggested concepts, practices and norms that nurture and enable model use and re-use across programs, projects and other complex endeavors. Key components include the ability to effectively move relevant information through a large community, process patterns that support model reuse and the identification of the necessary meta-information (ex. history, credibility, and provenance) to safely use and re-use that information. In order to successfully Use and Re-Use Models and Simulations we must define and meet key organizational and structural needs: 1. We must understand and acknowledge all the roles and players involved from the initial need identification through to the final product, as well as how they change across the lifecycle. 2. We must create the necessary structural elements to store and share NIMA-enabled information throughout the Program or Project lifecycle. 3. We must create the necessary organizational processes to stand up and execute a NIMA-enabled Program or Project throughout its lifecycle. NASA must meet all three of these needs to successfully use and re-use models. The ability to Reuse Models a key component of NIMA and the capabilities inherent in NIMA are key to accomplishing NASA's space exploration goals. 11

Conroy, Mike↗

NASA NTRS2015

Can We Trust Computational Modeling for Medical Applications?

Operations in extreme environments such as spaceflight pose human health risks that are currently not well understood and potentially unanticipated. In addition, there are limited clinical and research data to inform development and implementation of therapeutics for these unique health risks. In this light, NASA's Human Research Program (HRP) is leveraging biomedical computational models and simulations (M&S) to help inform, predict, assess and mitigate spaceflight health and performance risks, and enhance countermeasure development. To ensure that these M&S can be applied with confidence to the space environment, it is imperative to incorporate a rigorous verification, validation and credibility assessment (VV&C) processes to ensure that the computational tools are sufficiently reliable to answer questions within their intended use domain. In this presentation, we will discuss how NASA's Integrated Medical Model (IMM) and Digital Astronaut Project (DAP) have successfully adapted NASA's Standard for Models and Simulations, NASA-STD-7009 (7009) to achieve this goal. These VV&C methods are also being leveraged by organization such as the Food and Drug Administration (FDA), National Institute of Health (NIH) and the American Society of Mechanical Engineers (ASME) to establish new M&S VV&C standards and guidelines for healthcare applications. Similarly, we hope to provide some insight to the greater aerospace medicine community on how to develop and implement M&S with sufficient confidence to augment medical research and operations.

Mulugeta, Lealem↗

NASA NTRSDate not supplied

Pretest Simulations of a Supersonic Mixing and Combustion Validation Experiment to Assess Sensitivities

The reliance on CFD simulations to develop, design, and optimize scramjet systems (or components) has become commonplace. This reliance inevitably hinges on the ability of the computational analyst to quantify the level of confidence in their computational results. Unfortunately, nearly all the measured data available for this assessment comes from antiquated experimental datasets, or from tests that focused on the extraction of scramjet system (or component) performance. The objective of a CFD validation experiment is to quantify the predictive accuracy of one or more of the CFD physics submodels, implying that other uncertainties related to replicating the facility flow environment (e.g., knowledge of boundary conditions) must be minimized to the extent possible. This inevitably places stringent requirements on the quality and quantity of measurements taken to accurately specify inflow, outflow, and surface conditions for the CFD simulations; in addition to the measurements taken for the validation of physics submodels. This places additional demands on the experimental process above and beyond those for test article performance assessment. A recent high speed code credibility workshop series sponsored by AFRL identified a gap in existing validation data for fundamental assessments of turbulent mixing and combustion CFD closure models at scramjet engine relevant conditions. To address this gap, engineers at AFRL have designed a coaxial jet flame configuration that will be tested at two facilities (Research Cell 19 at the Air Force Research Lab, and at Purdue University). The effort described here documents pretest simulations of this validation experiment with the goal of fleshing out the extent of the facility flowpath that must be included to adequately reproduce the facility test section flow environment. The findings indicate that the flow around the support structure for the fuel injection centerbody upstream of the facility nozzle generates disturbances that persist throughout the nozzle expansion process; corrupting the azimuthal symmetry that was desired in the fuel/air mixing region of the test section. Simulations without this support structure maintained a high degree of azimuthal symmetry up until the fuel injection plane. However, even in this scenario the azimuthal symmetry was not maintained once the centerbody boundary layer transitioned to a wake flow downstream of the fuel injection plane.

CFD↗

NASA NTRSDate not supplied

Pretest Simulations of a Supersonic Mixing and Combustion Validation Experiment to Assess Sensitivities

The reliance on CFD simulations to develop, design, and optimize scramjet systems (or components) has become commonplace. This reliance inevitably hinges on the ability of the computational analyst to quantify the level of confidence in their computational results. Unfortunately, nearly all the measured data available for this assessment comes from antiquated experimental datasets, or from tests that focused on the extraction of scramjet system (or component) performance. The objective of a CFD validation experiment is to quantify the predictive accuracy of one or more of the CFD physics submodels, implying that other uncertainties related to replicating the facility flow environment (e.g., knowledge of boundary conditions) must be minimized to the extent possible. This inevitably places stringent requirements on the quality and quantity of measurements taken to accurately specify inflow, outflow, and surface conditions for the CFD simulations; in addition to the measurements taken for the validation of physics submodels. This places additional demands on the experimental process above and beyond those for test article performance assessment. A recent high speed code credibility workshop series sponsored by AFRL identified a gap in existing validation data for fundamental assessments of turbulent mixing and combustion CFD closure models at scramjet engine relevant conditions. To address this gap, engineers at AFRL have designed a coaxial jet flame configuration that will be tested at two facilities (Research Cell 19 at the Air Force Research Lab, and at Purdue University). The effort described here documents pretest simulations of this validation experiment with the goal of fleshing out the extent of the facility flowpath that must be included to adequately reproduce the facility test section flow environment. The findings indicate that the flow around the support structure for the fuel injection centerbody upstream of the facility nozzle generates disturbances that persist throughout the nozzle expansion process; corrupting the azimuthal symmetry that was desired in the fuel/air mixing region of the test section. Simulations without this support structure maintained a high degree of azimuthal symmetry up until the fuel injection plane. However, even in this scenario the azimuthal symmetry was not maintained once the centerbody boundary layer transitioned to a wake flow downstream of the fuel injection plane.

CFD↗

NASA NTRS2001

Risk analysis simulation of rover operations for Mars surface exploration

Risk management advocates have long sought to directly influence the early stages of the systems engineering process through a more effective role in system design trade studies. The principal obstacle to this has been the lack of credible ways to represent and quantify mission risk—that is, a combination of the probability of mission success (“system safety”) and science value—for the project manager and the rest of the design team. If it were possible to quantify mission risk, then the effects of proposed mission and system design changes could be calculated, and along with life-cycle costs, could be used to explore the design space more extensively and select better designs. JPL has been working to build the capability to quantify the probability of mission success using a federation of diverse simulations and models, each of which contributes some vital piece of the puzzle. The initial institutional focus has been on Mars surface operations. This ensemble computing framework enables the diverse models and simulations to work together seamlessly. Recent work at JPL has demonstrated the capability to exercise this ensemble from end-to-end using an Oracle-based database to automatically move results from one model/simulation to the next stage in the analysis.

Shishko, Robert↗