Search NASA⌕ Search

SEARCH · Search NASA

Results for “Simulation Credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Executive Summary: Special Section on Credible Computational Fluid Dynamics Simulations

This summary presents the motivation for the Special Section on the credibility of computational fluid dynamics (CFD) simulations, its objective, its background and context, its content, and its major conclusions. Verification and validation (V&V) are the processes for establishing the credibility of CFD simulations. Validation assesses whether correct things are performed and verification assesses whether they are performed correctly. Various aspects of V&V are discussed. Progress is made in verification of simulation models. Considerable effort is still needed for developing a systematic validation method that can assess the credibility of simulated reality.

Mehta, Unmeel B.↗

Dimensions of Credibility in Models and Simulations

Based on the National Aeronautics and Space Administration's (NASA's) work in developing a standard for models and simulations (M&S), the subject of credibility in M&S became a distinct focus. This is an indirect result from the Space Shuttle Columbia Accident Investigation Board (CAIB), which eventually resulted in an action, among others, to improve the rigor in NASA's M&S practices. The focus of this action came to mean a standardized method for assessing and reporting results from any type of M&S. As is typical in the standards development process, this necessarily developed into defming a common terminology base, common documentation requirements (especially for M&S used in critical decision making), and a method for assessing the credibility of M&S results. What surfaced in the development of the NASA Standard was the various dimensions credibility to consider when accepting the results from any model or simulation analysis. The eight generally relevant factors of credibility chosen in the NASA Standard proved only one aspect in the dimensionality of M&S credibility. At the next level of detail, the full comprehension of some of the factors requires an understanding along a couple of dimensions as well. Included in this discussion are the prerequisites for the appropriate use of a given M&S, the choice of factors in credibility assessment with their inherent dimensionality, and minimum requirements for fully reporting M&S results.

Steele, Martin J.↗

An Approach to Improved Credibility of CFD Simulations for Rocket Injector Design

Computational fluid dynamics (CFD) has the potential to improve the historical rocket injector design process by simulating the sensitivity of performance and injector-driven thermal environments to. the details of the injector geometry and key operational parameters. Methodical verification and validation efforts on a range of coaxial injector elements have shown the current production CFD capability must be improved in order to quantitatively impact the injector design process.. This paper documents the status of an effort to understand and compare the predictive capabilities and resource requirements of a range of CFD methodologies on a set of model problem injectors. Preliminary results from a steady Reynolds-Average Navier-Stokes (RANS), an unsteady Reynolds-Average Navier Stokes (URANS) and three different Large Eddy Simulation (LES) techniques used to model a single element coaxial injector using gaseous oxygen and gaseous hydrogen propellants are presented. Initial observations are made comparing instantaneous results, corresponding time-averaged and steady-state solutions in the near -injector flow field. Significant differences in the flow fields exist, as expected, and are discussed. An important preliminary result is the identification of a fundamental mixing mechanism, accounted for by URANS and LES, but missing in the steady BANS methodology. Since propellant mixing is the core injector function, this mixing process may prove to have a profound effect on the ability to more correctly simulate injector performance and resulting thermal environments. Issues important to unifying the basis for future comparison such as solution initialization, required run time and grid resolution are addressed.

Tucker, Paul K.↗

Development and Future of the Ten Simple Rules for Credible Practice of Models and Simulations in HealthCare

In this session, we will review the development and implementation history of the Ten Simple Rules (TSR) for credible practice in models and simulations that support healthcare. Our focus will be initially on the need and background for the Committee on Credible Practice of Models and Simulations in Healthcare (CPMS) as well as the TSR development process and implementation. This will flow into a demonstration of how the TSR has evolved as means of communicating and understanding the model development process, likely successful application and where the model could fail. We will end this presentation with a discussion on future development avenue, the CPMS efforts to develop a comprehensive rubric supporting current TSR implementation, and efforts to support further adoption of the TSR. The audience will be encouraged to comment on the current state of the TSR, share their TSR application experiences, and provide their insights on future development.

model↗

Credible practice of modeling and simulation in healthcare: ten rules from a multidisciplinary perspective

The complexities of modern biomedicine are rapidly increasing. Thus, modeling and simulation have become increasingly important as a strategy to understand and predict the trajectory of pathophysiology, disease genesis, and disease spread in support of clinical and policy decisions. In such cases, inappropriate or ill-placed trust in the model and simulation outcomes may result in negative outcomes, and hence illustrate the need to formalize the execution and communication of modeling and simulation practices. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the devel-opment and reuse of the models. For the past several years, the Committee on Credible Practice of Modeling and Simulation in Healthcare, an interdisciplinary group seeded from a U.S. interagency initiative, has worked to codify best practices. Here, we provide Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative analysis by the Committee’s multidisciplinary membership, followed by a large stakeholder com-munity survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. While biomedical science and clinical care domains have somewhat different requirements and expectations for credible practice, our study converged on rules that would be useful across a broad swath of model types. In brief, the rules are: (1) Define context clearly. (2) Use contextually appropriate data. (3) Evaluate within context. (4) List limitations explicitly. (5) Use version control. (6) Document appropriately. (7) Disseminate broadly. (8) Get independent reviews. (9) Test competing imple-mentations. (10) Conform to standards. Although some of these are common sense guidelines, we have found that many are often missed or misconstrued, even by seasoned practitioners. Computational models are already widely used in basic science to generate new biomedical knowledge. As they penetrate clinical care and healthcare policy, contributing to personalized and precision medicine, clinical safety will require established guidelines for the credible practice of modeling and simulation in healthcare.

Credibility↗

Code Verification of the HIGRAD Computational Fluid Dynamics Solver

The purpose of this report is to outline code and solution verification activities applied to HIGRAD, a Computational Fluid Dynamics (CFD) solver of the compressible Navier-Stokes equations developed at the Los Alamos National Laboratory, and used to simulate various phenomena such as the propagation of wildfires and atmospheric hydrodynamics. Code verification efforts, as described in this report, are an important first step to establish the credibility of numerical simulations. They provide evidence that the mathematical formulation is properly implemented without significant mistakes that would adversely impact the application of interest. Highly accurate analytical solutions are derived for four code verification test problems that exercise different aspects of the code. These test problems are referred to as: (i) the quiet start, (ii) the passive advection, (iii) the passive diffusion, and (iv) the piston-like problem. These problems are simulated using HIGRAD with different levels of mesh discretization and the numerical solutions are compared to their analytical counterparts. In addition, the rates of convergence are estimated to verify the numerical performance of the solver. The first three test problems produce numerical approximations as expected. The fourth test problem (piston-like) indicates the extent to which the code is able to simulate a 'mild' discontinuity, which is a condition that would typically be better handled by a Lagrangian formulation. The current investigation concludes that the numerical implementation of the solver performs as expected. The quality of solutions is sufficient to provide credible simulations of fluid flows around wind turbines. The main caveat associated to these findings is the low coverage provided by these four problems, and somewhat limited verification activities. A more comprehensive evaluation of HIGRAD may be beneficial for future studies.

17 WIND ENERGY↗

Towards a Credibility Assessment of Models and Simulations

A scale is presented to evaluate the rigor of modeling and simulation (M&S) practices for the purpose of supporting a credibility assessment of the M&S results. The scale distinguishes required and achieved levels of rigor for a set of M&S elements that contribute to credibility including both technical and process measures. The work has its origins in an interest within NASA to include a Credibility Assessment Scale in development of a NASA standard for models and simulations.

Blattnig, Steve R.↗

Credible Practice of Models and Simulations for Healthcare

As computational models penetrate clinical care and drive healthcare policy decisions, there is increasing need to communicate that model creation and testing follows reasonable credible practice. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the development and reuse of the models. This presentation introduces the Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative multidisciplinary analysis and practitioner survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. Although some of these are common sense guidelines, many are often missed or misconstrued, even by seasoned practitioners.

credibility↗

From models to reality: a systematic review on simulated and measured residential heat pump energy savings

High-performance HVAC solutions are central to residential energy management. A substantial share of these are electric, reversible-cycle systems, with heat pumps representing the largest portion of current and near-term adoption. This review synthesizes peer-reviewed and grey literature on residential space heating and cooling heat pumps. The academic literature is dominated by modeling (73.8%), with limited field measurement (13.1%). Grey literature from United States serve as a supplemental resource providing measured savings. Conversions from electric-resistance heating consistently show the largest site energy reductions, while oil/propane baselines yield moderate savings, and gas baseline scenario often deliver small and region-dependent savings. This study cross-checks the grey literature measured data with simulation data filtered from the ResStock dataset. The comparison indicates a discrepancy between simulations and measured data: simulated site EUIs are typically lower than measured EUIs, but percentage energy savings fall in similar ranges, implying simulations capture directional effects while underestimating energy use. Factors associated with variability and model–measurement differences include system characterization and control representation (e.g., backup heat engagement, thermostat/setpoint strategies, commissioning/installation quality), occupant behavior, weather normalization, metering scope, and envelope characterization. This paper also outlines the proposed methodology for comparing simulation and measured data for heat pumps. It emphasizes the metrics used for comparison and units harmonization, building characteristics matching, and compact metadata are needed for simulations to match measured data. The proposed methodology is expected to improve the credibility of simulated savings as measured evidence grows.

Yu, Lili↗

Dynamics of Arctic and Sub-Arctic Climate and Atmospheric Circulation: Diagnosis of Mechanisms and Biases Using Data Assimilation

This is the final report for DOE grant DE-FG02-07ER64434 to Eric DeWeaver at the University of Wisconsin-Madison. The overall goal of work performed under this grant is to enhance understanding of simulations of present-day climate and greenhouse gas-induced climate change. Enhanced understanding is desirable 1) as a prerequisite for improving simulations; 2) for assessing the credibility of model simulations and their usefulness as tools for decision support; and 3) as a means to identify robust behaviors which commonly occur over a wide range of models, and may yield insights regarding the dominant physical mechanisms which determine mean climate and produce climate change. A furthe objective is to investigate the use of data assimilation as a means for examining and correcting model biases. Our primary focus is on the Arctic, but the scope of the work was expanded to include the global climate system to the extent that research targets of opportunity present themselves. Research performed under the grant falls into five main research areas: 1) a study of data assimilation using an ensemble filter with the atmospheric circulation model of the National Center for Atmospheric Research, in which both conventional observations and observations of the refraction of radio waves from GPS satellites were used to constrain the atmospheric state of the model; 2) research on the likely future status of polar bears, in which climate model simluations were used to assess the effectiveness of climate change mitigation efforts in preserving the habitat of polar bears, now considered a threatened species under global warming; 3) as assessment of the credibility of Arctic sea ice thickness simulations from climate models; 4) An examination of the persistence and reemergence of Northern Hemisphere sea ice area anomalies in climate model simulations and in observations; 5) An examination of the roles played by changes in net radiation and surface relative humidity in determine the response of the hydrological cycle to global warming.

58 GEOSCIENCES↗

Comparing simulated demand flexibility against actual performance in commercial office buildings

Commercial building energy benchmarking has been used as a mechanism to evaluate energy use of a single building over time, relative to other similar buildings, or to simulations of a reference building conforming to various energy standards. Lack of empirical demand flexibility data and consistent flexibility metrics has limited the ability to compare demand flexibility performance with estimated demand flexibility in buildings. In this study, we collected demand response performance data for a total of 831 demand response events from 192 sites as a first step to build such a building demand flexibility dataset, and propose a standard core data schema to consolidate field data from different sources. We also performed parametric simulations of a control strategy called “global temperature adjustment” using commercial office prototype building models. We then compared the simulated demand flexibility performance against the actual data for offices with global temperature adjustment strategy implemented. During demand response events with an average outside air temperature of 34 °C (range 23 °C–42 °C), the measured demand decrease intensity of the demand flexibility metrics were 6.1 watts per square meter (W/m 2 ), 10.0 W/m 2 , 11.1 W/m 2 , 7.1 W/m 2 , and 4.7 W/m 2 for small, small–medium, medium, medium–large, and large office buildings, respectively. Compared to the measured data in medium- and large-size buildings, the simulated demand decrease intensity was 0.7 W/m 2 (17%) lower on average. The discrepancy between simulated and measured peak demand intensities fell within one standard deviation of the mean measured data. Here, the comparison results validate the credibility of simulations in capturing real building data for assessing the technical potential of building demand flexibility.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Dynamics of Arctic and Sub-Arctic Climate and Atmospheric Circulation: Diagnosis of Mechanisms and Biases Using Data Assimilation

The overall goal of work performed under this grant is to enhance understanding of simulations of present-day climate and greenhouse gas-induced climate change. The examination of present-day climate also includes diagnostic intercomparison of model simulations and observed mean climate and climate variability using reanalysis and satellite datasets. Enhanced understanding is desirable 1) as a prerequisite for improving simulations; 2) for assessing the credibility of model simulations and their usefulness as tools for decision support; and 3) as a means to identify robust behaviors which commonly occur over a wide range of models, and may yield insights regarding the dominant physical mechanisms which determine mean climate and produce climate change. A further objective is to investigate the use of data assimilation as a means for examining and correcting model biases. Our primary focus is on the Arctic, but the scope of the work was expanded to include the global climate system.

58 GEOSCIENCES↗

Verification and Validation of Simulation Models for High-Speed Flow Fields

This paper is largely based on the paper titled "Guide to Credible Computational Fluid Dynamics Simulations (Invited)" (AIAA Paper 95-2225). The significance of computational fluid dynamics (CFD) simulations depends solely on their credibility. A customer of CFD products -- simulations and software -- expects that these products are credible for the intended use. Simulation model verification and validation are critical in establishing the credibility of simulations and in certifying simulation software. The only thing that matters in establishing the credibility is uncertainty, not veracity or validity. The sensitivity-uncertainty analysis is the key to the establishment of this credibility. Assessing the credibility of complex simulation results poses a significant challenge. Terminology, concepts, framework, and guidelines are presented for addressing this challenge. Verification assesses whether the problem is solved correctly and estimates the level of computational accuracy of simulations; validation determines whether the right problem is solved and assesses the level of validity of the simulation model by estimating the degree to which simulations accurately represent reality. These concepts and the related guidelines are discussed with examples from high-speed flow fields.

Mehta, Unmeel B.↗

The Continual Intercomparison of Radiation Codes (CIRC) Assessing Anew the Quality of GCM Radiation Algorithms

The simulation of changes in the Earth's climate due to solar and thermal radiative processes with global climate models (GCMs) is highly complex, depending on the parameterization of a multitude of nonlinearly coupled physical processes. In contrast, the germ of global climate change, the radiative forcing from enhanced abundances of greenhouse gases, is relatively well understood. The impressive agreement between detailed radiation calculations and highly resolved spectral radiation measurements in the thermal infrared under cloudless conditions (see, for example, Fig. 1) instills confidence in our knowledge of the sources of gaseous absorption. That the agreement spans a broad range of temperature and humidity regimes using instruments mounted on surface, aircraft, and satellite platforms not only attests to our capability to accurately calculate radiative fluxes under present conditions, but also provides confidence in the spectroscopic basis for computation of fluxes under conditions that might characterize future global climate (e.g., radiative forcing). Alas, the computational costs of highly resolved spectral radiation calculations cannot be afforded presently in GCMs. Such calculations have instead been used as the foundation for approximations implemented in fast but generally less accurate algorithms performing the needed radiative transfer (RT) calculations in GCMs. Credible climate simulations by GCMs cannot be ensured without accurate solar and thermal radiative flux calculations under all types of sky conditions: pristine cloudless, aerosol-laden, and cloudy. The need for accuracy in RT calculations is not only important for greenhouse gas forcing scenarios, but is also profoundly needed for the robust simulation of many other atmospheric phenomena, such as convective processes.

Oreopoulos, Lazaros↗

Coupled Euler-Lagrange Simulation of the Jolt HE Event and Relevance to the Shock Physics Experiments

The goal of the Shock Physics Experiments is to increase our understanding of the shock phenomenology from an explosive event to aid in modeling these events. A more accurate understanding of the effects of near source geology on that phenomenology is critical. We demonstrated the use of the multi physics Abaqus ® fully coupled Euler-Lagrange code to model the DTRA JOLT experiment. Abaqus/CEL was able to compute the high-deformation explosive loading, the complex rock response with damage, and the large general contact domain. Comparison of results to test data reveals that the simulation performed credibly. These results provide insight into propagati0n of explosive-caused shock in jointed media. This exercise provides evidence that the shock front is not perturbed by the joints. While it is possible that later time, lower amplitude portions of the waveform can exhibit behavior caused by joints, we conclude that the main shock events from the SPE events is not effected by the local jointing.

58 GEOSCIENCES↗

Uncertainty quantification of machine learning models to improve streamflow prediction under changing climate and environmental conditions

Machine learning (ML) models, and Long Short-Term Memory (LSTM) networks in particular, have demonstrated remarkable performance in streamflow prediction and are increasingly being used by the hydrological research community. However, most of these applications do not include uncertainty quantification (UQ). ML models are data driven and can suffer from large extrapolation errors when applied to changing climate/environmental conditions. UQ is required to quantify the influence of data noises on model predictions and avoid overconfident projections in extrapolation. In this work, we integrate a novel UQ method, called PI3NN, with LSTM networks for streamflow prediction. PI3NN calculates Prediction Intervals by training 3 Neural Networks. It can precisely quantify the predictive uncertainty caused by the data noise and identify out-of-distribution (OOD) data in a non-stationary condition to avoid overconfident predictions. We apply the PI3NN-LSTM method in the snow-dominant East River Watershed in the western US and in the rain-driven Walker Branch Watershed in the southeastern US. Results indicate that for the prediction data which have similar features as the training data, PI3NN precisely quantifies the predictive uncertainty with the desired confidence level; and for the OOD data where the LSTM network fails to make accurate predictions, PI3NN produces a reasonably large uncertainty indicating that the results are not trustworthy and should avoid overconfidence. PI3NN is computationally efficient, robust in performance, and generalizable to various network structures and data with no distributional assumptions. It can be broadly applied in ML-based hydrological simulations for credible prediction.

54 ENVIRONMENTAL SCIENCES↗