SEARCH · Search NASA

Results for “model credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

NASA NTRS2006

Modeling and Simulation Resource Repository (MSRR)(System Engineering/Integrated M&S Management Approach

NASA s Exploration Systems Mission Directorate (ESMD) is implementing a management approach for modeling and simulation (M&S) that will provide decision-makers information on the model s fidelity, credibility, and quality, including the verification, validation and accreditation information. The NASA MSRR will be implemented leveraging M&S industry best practices. This presentation will discuss the requirements that will enable NASA to capture and make available the "meta data" or "simulation biography" data associated with a model. The presentation will also describe the requirements that drive how NASA will collect and document relevant information for models or suites of models in order to facilitate use and reuse of relevant models and provide visibility across NASA organizations and the larger M&S community.

Milroy, Audrey↗

NASA NTRS2000

Towards a Comprehensive Dynamic-chemistry Assimilation for Eos-Chem: Plans and Status in NASA's Data Assimilation Office

In order to support the EOS-Chem project, a comprehensive assimilation package for the coupled chemical-dynamical system is being developed by the Data Assimilation Office at NASA GSFC. This involves development of a coupled chemistry/meteorology model and of data assimilation techniques for trace species and meteorology. The model is being developed using the flux-form semi-Lagrangian dynamical core of Lin and Rood, the physical parameterizations from the NCAR Community Climate Model, and atmospheric chemistry modules from the Atmospheric Chemistry and Dynamics branch at NASA GSFC. To date the following results have been obtained: (i) multi-annual simulations with the dynamics-radiation model show the credibility of the package for atmospheric simulations; (ii) initial simulations including a limited number of middle atmospheric trace gases reveal the realistic nature of transport mechanisms, although there is still a need for some improvements. Samples of these results will be shown. A meteorological assimilation system is currently being constructed using the model; this will form the basis for the proposed meteorological/chemical assimilation package. The latter part of the presentation will focus on areas targeted for development in the near and far terms, with the objective of Providing a comprehensive assimilation package for the EOS-Chem science experiment. The first stage will target ozone assimilation. The plans also encompass a reanalysis (ReSTS) for the 1991-1995 period, which includes the Mt. Pinatubo eruption and the time when a large number of UARS observations were available. One of the most challenging aspects of future developments will be to couple theoretical advances in tracer assimilation with the practical considerations of a real environment and eventually a near-real-time assimilation system.

Pawson, Steven↗

NASA NTRS2022

A Framework for Deep Learning Emulation of Numerical Models With a Case Study in Satellite Remote Sensing

Numerical models based on physics represent the state of the art in Earth system modeling and comprise our best tools for generating insights and predictions. Despite rapid growth in computational power, the perceived need for higher model resolutions overwhelms the latest generation computers, reducing the ability of modelers to generate simulations for understanding parameter sensitivities and characterizing variability and uncertainty. Thus, surrogate models are often developed to capture the essential attributes of the full-blown numerical models. Recent successes of machine learning methods, especially deep learning (DL), across many disciplines offer the possibility that complex nonlinear connectionist representations may be able to capture the underlying complex structures and nonlinear processes in Earth systems. A difficult test for DL-based emulation, which refers to function approximation of numerical models, is to understand whether they can be comparable to traditional forms of surrogate models in terms of computational efficiency while simultaneously reproducing model results in a credible manner. A DL emulation that passes this test may be expected to perform even better than simple models with respect to capturing complex processes and spatiotemporal dependencies. Here, we examine, with a case study in satellite-based remote sensing, the hypothesis that DL approaches can credibly represent the simulations from a surrogate model with comparable computational efficiency. Our results are encouraging in that the DL emulation reproduces the results with acceptable accuracy and often even faster performance. We discuss the broader implications of our results in light of the pace of improvements in high-performance implementations of DL and the growing desire for higher resolution simulations in the Earth sciences.

Bayesian Deep Learning↗

NASA NTRS2009

Motion Cueing Algorithm Modification for Improved Turbulence Simulation

Atmospheric turbulence cueing produced by flight simulator motion systems has been less than satisfactory because the turbulence profiles have been attenuated by the motion cueing algorithms. Cardullo and Ellor initially addressed this problem by directly porting the turbulence model output to the motion system. Reid and Robinson addressed the problem by employing a parallel aircraft model, which is only stimulated by the turbulence inputs and adding a filter specially designed to pass the higher turbulence frequencies. There have been advances in motion cueing algorithm development at the Man-Machine Systems Laboratory, at SUNY Binghamton. In particular, the system used to generate turbulence cues has been studied. The Reid approach, implemented by Telban and Cardullo, was employed to augment the optimal motion cueing algorithm installed at the NASA LaRC Simulation Laboratory, driving the Visual Motion Simulator. In this implementation, the output of the primary flight channel was added to the output of the turbulence channel and then sent through a non-linear cueing filter. The cueing filter is an adaptive filter; therefore, it is not desirable for the output of the turbulence channel to be augmented by this type of filter. The likelihood of the signal becoming divergent was also an issue in this design. After testing on-site it became apparent that the architecture of the turbulence algorithm was generating unacceptable cues. As mentioned above, this cueing algorithm comprised a filter that was designed to operate at low bandwidth. Therefore, the turbulence was also filtered, augmenting the cues generated by the model. If any filtering is to be done to the turbulence, it will utilize a filter with a much higher bandwidth, above the frequencies produced by the aircraft response to turbulence. The authors have developed an implementation wherein only the signal from the primary flight channel passes through the nonlinear cueing filter. This paper discusses three new algorithms. Testing shows that the new methods provide the pilot with a more realistic sensation of turbulence; the cues are not attenuated by algorithm. Results of offline testing show the credibility of the models. Offline test verification was based primarily on the evaluation of the power spectral density of the outputs and the time response.

Ercole, Anthony V.↗

NASA NTRS1999

Ocean Turbulence I: One-Point Closure Model Momentum and Heat Vertical Diffusivities

Since the early forties, one-point turbulence closure models have been the canonical tools used to describe turbulent flows in many fields. In geophysics, Mellor and Yamada applied such models using the 1980 state-of-the art. Since then, no improvements were introduced to alleviate two major difficulties: 1) closure of the pressure correlations, which affects the correct determination of the critical Richardson number Ri(sub cr) above which turbulent mixing is no longer possible and 2) the need to express the non-local third-order moments (TOM) in terms of lower order moments rather than via the down-gradient approximation as done thus far, since the latter seriously underestimates the TOMs. Since 1) and 2) are still being dealt with adjustable parameters which weaken the credibility of the models, alternative models, not based on turbulence modeling, have been suggested. The aim of this paper is to show that new information, partly derived from the newest 2-point closure model discussed, can be used to solve these shortcomings. The new one-point closure model, which in its simplest form is algebraic and thus simple to implement, is first shown to reproduce a variety of data. Then, it is used in a Ocean-General Circulation Model (O-GCM) where it reproduces well a large variety of ocean data. While phenomenological models are specifically tuned to ocean turbulence, the present model is not. It is first tested against laboratory data on stably stratified flows and then used in an O-GCM. It is more general, more predictive and more resilient, e.g., it can incorporate phenomena like wave-breaking at the surface, salinity diffusivity, non-locality, etc. One important feature that naturally comes out of the new model is that the predicted Richardson critical value Ri(sub cr) is Ri (sub cr approx. = 1) in agreement with both Large Eddy Simulations (LES) and empirical evidence while all previous models predicted Ri (sub cr approx. = 0.2) which led to a considerable underestimate of the extent of turbulent mixing and thus to an incorrect mixed layer depth. The predicted temperature and salinity profiles (vs. depth) are presented and compared with those of the Kolmogorov-Petruvsky-Piskunuv (KPP) model and Levitus data.

Canuto, V. M.↗

NASA NTRS1986

F-106 data summary and model results relative to threat criteria and protection design analysis

The NASA F-106 has acquired considerable data on the rates-of-change of electromagnetic parameters on the aircraft surface during 690 direct lightning strikes while penetrating thunderstorms at altitudes ranging from 15,000 to 40,000 feet. These in-situ measurements have provided the basis for the first statistical quantification of the lightning electromagnetic threat to aircrat appropriate for determining lightning indirect effects on aircraft. The data are presently being used in updating previous lightning criteria and standards developed over the years from ground-based measurements. The new lightning standards will, therefore, be the first which reflect actual aircraft responses measured at flight altitudes. The modeling technique developed to interpret and understand the direct strike electromagnetic data acquired on the F-106 provides a means to model the interaction of the lightning channel with the F-106. The reasonable results obtained with the model, compared to measured responses, yield confidence that the model may be credibly applied to other aircraft types and uses in the prediction of internal coupling effects in the design of lightning protection for new aircraft.

Pitts, F. L.↗

NASA NTRS1987

New methods and results for quantification of lightning-aircraft electrodynamics

The NASA F-106 collected data on the rates of change of electromagnetic parameters on the aircraft surface during over 700 direct lightning strikes while penetrating thunderstorms at altitudes from 15,000 t0 40,000 ft (4,570 to 12,190 m). These in situ measurements provided the basis for the first statistical quantification of the lightning electromagnetic threat to aircraft appropriate for determining indirect lightning effects on aircraft. These data are used to update previous lightning criteria and standards developed over the years from ground-based measurements. The proposed standards will be the first which reflect actual aircraft responses measured at flight altitudes. Nonparametric maximum likelihood estimates of the distribution of the peak electromagnetic rates of change for consideration in the new standards are obtained based on peak recorder data for multiple-strike flights. The linear and nonlinear modeling techniques developed provide means to interpret and understand the direct-strike electromagnetic data acquired on the F-106. The reasonable results obtained with the models, compared with measured responses, provide increased confidence that the models may be credibly applied to other aircraft.

Pitts, Felix L.↗

NASA NTRS2008

A Capable and Temporary Test Facility on a Shoestring Budget: The MSL Touchdown Test Facility

The Mars Science Laboratory mission (MSL) has undertaken a developmental Touchdown Test Program that utilizes a full-scale rover vehicle and an overhead winch system to replicate the skycrane landing event. Landing surfaces consisting of flat and sloped granular media, planar, rigid surfaces, and various combinations of rocks and slopes were studied. Information gathered from these tests was vital for validating the rover analytical model, validating certain design or system behavior assumptions, and for exploring events and phenomenon that are either very difficult or too costly to model in a credible way. This paper describes this test program, with a focus on the creation of test facility, daily test operations, and some of the challenges faced and lessons learned along the way.

rover↗

NASA NTRS2014

NASA-STD-7009 Guidance Document for Human Health and Performance Models and Simulations

Rigorous verification, validation, and credibility (VVC) processes are imperative to ensure that models and simulations (MS) are sufficiently reliable to address issues within their intended scope. The NASA standard for MS, NASA-STD-7009 (7009) [1] was a resultant outcome of the Columbia Accident Investigation Board (CAIB) to ensure MS are developed, applied, and interpreted appropriately for making decisions that may impact crew or mission safety. Because the 7009 focus is engineering systems, a NASA-STD-7009 Guidance Document is being developed to augment the 7009 and provide information, tools, and techniques applicable to the probabilistic and deterministic biological MS more prevalent in human health and performance (HHP) and space biomedical research and operations.

numerical analysis↗

NASA NTRS2017

The Cloud Feedback Model Intercomparison Project (CFMIP) contribution to CMIP6.

The primary objective of CFMIP is to inform future assessments of cloud feedbacks through improved understanding of cloud-climate feedback mechanisms and better evaluation of cloud processes and cloud feedbacks in climate models. However, the CFMIP approach is also increasingly being used to understand other aspects of climate change, and so a second objective has now been introduced, to improve understanding of circulation, regional-scale precipitation, and non-linear changes. CFMIP is supporting ongoing model inter-comparison activities by coordinating a hierarchy of targeted experiments for CMIP6, along with a set of cloud-related output diagnostics. CFMIP contributes primarily to addressing the CMIP6 questions 'How does the Earth system respond to forcing?' and 'What are the origins and consequences of systematic model biases?' and supports the activities of the WCRP Grand Challenge on Clouds, Circulation and Climate Sensitivity. A compact set of Tier 1 experiments is proposed for CMIP6 to address this question: (1) what are the physical mechanisms underlying the range of cloud feedbacks and cloud adjustments predicted by climate models, and which models have the most credible cloud feedbacks? Additional Tier 2 experiments are proposed to address the following questions. (2) Are cloud feedbacks consistent for climate cooling and warming, and if not, why? (3) How do cloud-radiative effects impact the structure, the strength and the variability of the general atmospheric circulation in present and future climates? (4) How do responses in the climate system due to changes in solar forcing differ from changes due to CO2, and is the response sensitive to the sign of the forcing? (5) To what extent is regional climate change per CO2 doubling state-dependent (non-linear), and why? (6) Are climate feedbacks during the 20th century different to those acting on long-term climate change and climate sensitivity? (7) How do regional climate responses (e.g. in precipitation) and their uncertainties in coupled models arise from the combination of different aspects of CO2 forcing and sea surface warming? CFMIP also proposes a number of additional model outputs in the CMIP DECK, CMIP6 Historical and CMIP6 CFMIP experiments, including COSP simulator outputs and process diagnostics to address the following questions. 1. How well do clouds and other relevant variables simulated by models agree with observations? 2. What physical processes and mechanisms are important for a credible simulation of clouds, cloud feedbacks and cloud adjustments in climate models? 3. Which models have the most credible representations of processes relevant to the simulation of clouds? 4. How do clouds and their changes interact with other elements of the climate system?

carbon dioxide↗

NASA NTRS1978

Numerical derivation of linear perturbation model from nonlinear real-time vehicle simulations - A means of validation and increased credibility

A numerical technique has been developed which generates linear perturbation models from nonlinear real-time aircraft vehicle simulations at Langley Research Center. The technique is very general and can be applied to simulations of any system described by nonlinear differential equations. Linear models about various trim conditions have been obtained from simulations of the Boeing 737, Northrop F5-E, Cessna 180, Sikorsky CH-54, and NASA/Army Rotor Systems Research Aircraft (RSRA). One use of the derived linear models is as a validation and verification tool for the nonlinear simulation.

Dieudonne, J. E.↗

NASA NTRS2010

An Extreme-Value Approach to Anomaly Vulnerability Identification

The objective of this paper is to present a method for importance analysis in parametric probabilistic modeling where the result of interest is the identification of potential engineering vulnerabilities associated with postulated anomalies in system behavior. In the context of Accident Precursor Analysis (APA), under which this method has been developed, these vulnerabilities, designated as anomaly vulnerabilities, are conditions that produce high risk in the presence of anomalous system behavior. The method defines a parameter-specific Parameter Vulnerability Importance measure (PVI), which identifies anomaly risk-model parameter values that indicate the potential presence of anomaly vulnerabilities, and allows them to be prioritized for further investigation. This entails analyzing each uncertain risk-model parameter over its credible range of values to determine where it produces the maximum risk. A parameter that produces high system risk for a particular range of values suggests that the system is vulnerable to the modeled anomalous conditions, if indeed the true parameter value lies in that range. Thus, PVI analysis provides a means of identifying and prioritizing anomaly-related engineering issues that at the very least warrant improved understanding to reduce uncertainty, such that true vulnerabilities may be identified and proper corrective actions taken.

Everett, Chris↗

NASA NTRS1995

Verification and Validation of Simulation Models for High-Speed Flow Fields

This paper is largely based on the paper titled "Guide to Credible Computational Fluid Dynamics Simulations (Invited)" (AIAA Paper 95-2225). The significance of computational fluid dynamics (CFD) simulations depends solely on their credibility. A customer of CFD products -- simulations and software -- expects that these products are credible for the intended use. Simulation model verification and validation are critical in establishing the credibility of simulations and in certifying simulation software. The only thing that matters in establishing the credibility is uncertainty, not veracity or validity. The sensitivity-uncertainty analysis is the key to the establishment of this credibility. Assessing the credibility of complex simulation results poses a significant challenge. Terminology, concepts, framework, and guidelines are presented for addressing this challenge. Verification assesses whether the problem is solved correctly and estimates the level of computational accuracy of simulations; validation determines whether the right problem is solved and assesses the level of validity of the simulation model by estimating the degree to which simulations accurately represent reality. These concepts and the related guidelines are discussed with examples from high-speed flow fields.

Mehta, Unmeel B.↗

NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Across engineering there is an evolving need to increase reliance on physics-based simulation to develop, design and optimize engineering systems. This increased reliance on modeling and simulation has highlighted a growing need to transform the confidence that modeling and simulation analysts have in their results into credibility for systems engineers to design and field systems more quickly and with less physical testing. For isolated components of a complex system, where a single discipline may drive product design, this is less of a concern as the relationship is often straightforward and easy to explain. However, when these isolated components are integrated, and are expected to operate in a multi-disciplinary context in which safety critical systems are involved, new concepts and model assurance standards are required. In this paper we address this challenge by showing how a model validation hierarchy can be exploited to identify those model validation experiments that will contribute most to increasing confidence and credibility of modeling and simulation predictions. The approach that is adopted contains four main steps. The first step is the construction of a model validation hierarchy that links subsystems, assemblies, and components to a hierarchy of physical experiments that can be used support model validation. This hierarchy connects the concerns of systems engineers to those of the modeling and simulation analyst in a clear and logical way. The structure and content of this hierarchy is then used in a second step to establish which physical phenomena have the greatest impact on overall system performance metrics. A gap analysis technique, based upon modeling and simulation concerns, is then used to prioritize the important physical phenomenon. Unfortunately, a common outcome of such gap analyses is the identification of many important gaps and so, in the final step of our process, we advocate the use of a global sensitivity analysis as a means to complete the prioritization.

Verification and Validation↗

NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Verification and Validation↗

NASA NTRS2022

NASA Orbital Debris Engineering Model (ORDEM) 3.1: Model Verification and Validation

The NASA Orbital Debris Engineering Model (ORDEM) 3.1 Model Verification and Validation (V&V) document accompanies the delivery of the latest ORDEM 3.1 model (Vavrin & Manis, 2019) and provides a detailed description of the V&V activities used to verify that the model was built correctly and validate the model against independent, real world sources of data obtained from sampling the orbital debris environment. This ORDEM 3.1 Model V&V document, along with the related ORDEM 3.1 Model Process document – which covers details of the mathematical, statistical, and physical basis of the model – are intended to inform credibility assessments, risk analyses, uncertainty characterizations, and other applications derived from use of the model by the ORDEM 3.1 user community.

Timothy Kennedy↗

NASA NTRS1994

Magnetohydrodynamic Simulation of a Streamer Beside a Realistic Coronal Hole

Existing models of coronal streamers establish their credibility and act as the initial state for transients. The models have produced satisfactory streamer simulations, but unsatisfactory coronal hole simulations. This is a consequence of the character of the models and the boundary conditions. The models all have higher densities in the magnetically open regions than occur in coronal holes (Noci, et al., 1993).

Suess, S. T.↗

NASA NTRS2007

The Roles of Verification, Validation and Uncertainty Quantification in the NASA Standard for Models and Simulations

The National Aeronautics and Space Administration (NASA) recently issued an interim version of the Standard for Models and Simulations (M&S Standard) [1]. The action to develop the M&S Standard was identified in an internal assessment [2] of agency-wide changes needed in the wake of the Columbia Accident [3]. The primary goal of this standard is to ensure that the credibility of M&S results is properly conveyed to those making decisions affecting human safety or mission success criteria. The secondary goal is to assure that the credibility of the results from models and simulations meets the project requirements (for credibility). This presentation explains the motivation and key aspects of the M&S Standard, with a special focus on the requirements for verification, validation and uncertainty quantification. Some pilot applications of this standard to computational fluid dynamics applications will be provided as illustrations. The authors of this paper are the members of the team that developed the initial three drafts of the standard, the last of which benefited from extensive comments from most of the NASA Centers. The current version (number 4) incorporates modifications made by a team representing 9 of the 10 NASA Centers. A permanent version of the M&S Standard is expected by December 2007. The scope of the M&S Standard is confined to those uses of M&S that support program and project decisions that may affect human safety or mission success criteria. Such decisions occur, in decreasing order of importance, in the operations, the test & evaluation, and the design & analysis phases. Requirements are placed on (1) program and project management, (2) models, (3) simulations and analyses, (4) verification, validation and uncertainty quantification (VV&UQ), (5) recommended practices, (6) training, (7) credibility assessment, and (8) reporting results to decision makers. A key component of (7) and (8) is the use of a Credibility Assessment Scale, some of the details of which were developed in consultation with William Oberkampf, David Peercy and Timothy Trocano of Sandia National Laboratories. The focus of most of the requirements, including those for VV&UQ, is on the documentation of what was done and the reporting, using the Credibility Assessment Scale, of the level of rigor that was followed. The aspects of one option for the Credibilty Assessment Scale are (1) code verification, (2) solution verification, (3) validation, (4) predictive capability, (5) technical review, (6) process control, and (7) operator and analyst qualification.

Zang, Thomas A.↗