Search NASA⌕ Search

SEARCH · Search NASA

Results for “Simulation Credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

Validation of a comprehensive first-principles-based framework for predicting the performance of future stellarators

This paper presents the validation of the GENE-KNOSOS-Tango framework for recovering both the steady-state plasma profiles in the considered radial domain and selected turbulence trends in a stellarator. This framework couples the gyrokinetic turbulence code GENE, the neoclassical transport code KNOSOS, and the transport solver Tango in a multi-timescale simulation feedback loop. Ion-scale kinetic-electron and electron-scale adiabatic-ion flux-tube simulations were performed to evolve the density and temperature profiles for four OP1.2b W7-X scenarios. The simulated density and temperature profiles showed good agreement with the experimental data using a reasonable set of boundary conditions. Equally important was the reproduction of observed trends for several turbulence properties, such as density fluctuations and turbulent heat diffusivities. Key effects were also touched upon, such as electron-scale turbulence and the neoclassical radial electric field shear. The validation of the GENE-KNOSOS-Tango framework enables credible predictions of physical phenomena in stellarators and reactor performance based on a given set of edge parameters.

Gyrokinetic simulations↗

Opening new institutional spaces for grappling with uncertainty: A constructivist perspective

In the context of an increasing reliance on predictive computer simulation models to calculate potential project impacts, it has become common practice in impact assessment (IA) to call on proponents to disclose uncertainties in assumptions and conclusions assembled in support of a development project. Understandably, it is assumed that such disclosures lead to greater scrutiny and better policy decisions. This paper questions this assumption. Drawing on constructivist theories of knowledge and an analysis of the role of narratives in managing uncertainty, I argue that the disclosure of uncertainty can obscure as much as it reveals about the impacts of a development project. It is proposed that the opening up of institutional spaces that can facilitate the negotiation and deliberation of foundational assumptions and parameters that feed into predictive models could engender greater legitimacy and credibility for IA outcomes. - Highlights: Black-Right-Pointing-Pointer A reliance on supposedly objective disclosure is unreliable in the predictive model context in which IA is now embedded. Black-Right-Pointing-Pointer A reliance on disclosure runs the risk of reductionism and leaves unexamined the social-interactive aspects of uncertainty. Black-Right-Pointing-Pointer Opening new institutional spaces could facilitate deliberation on foundational predictive model assumptions.

54 ENVIRONMENTAL SCIENCES↗

MFNets: Multifidelity data-driven networks for Bayesian learning and prediction

This paper presents a multifidelity uncertainty quantification framework called MFNets. We seek to address three existing challenges that arise when experimental and simulation data from different sources are used to enhance statistical estimation and prediction with quantified uncertainty. Specifically, we demonstrate that MFNets can (1) fuse heterogeneous data sources arising from simulations with different parameterizations, e.g simulation models with different uncertain parameters or data sets collected under different environmental conditions; (2) encode known relationships among data sources to reduce data requirements; and (3) improve the robustness of existing multi-fidelity approaches to corrupted data. MFNets construct a network of latent variables (LVs) to facilitate the fusion of data from an ensemble of sources of varying credibility and cost. These LVs are posited as explanatory variables that provide the source of correlation in the observed data. Furthermore, MFNets provide a way to encode prior physical knowledge to enable efficient estimation of statistics and/or construction of surrogates via conditional independence relations on the LVs. We highlight the utility of our framework with a number of theoretical results which assess the quality of the posterior mean as a frequentist estimator and compare it to standard sampling approaches that use single fidelity, multilevel, and control variate Monte Carlo estimators. We also use the proposed framework to derive the Monte Carlo-based control variate estimator entirely from the use of Bayes rule and linear-Gaussian models -- to our knowledge the first such derivation. Finally, we demonstrate the ability to work with different uncertain parameters across different models.

97 MATHEMATICS AND COMPUTING↗

Data from: Comparing traditional and Bayesian approaches to ecological meta-analysis

Despite the wide application of meta-analysis in ecology, some of the traditional methods used for meta-analysis may not perform well given the type of data characteristic of ecological meta-analyses. We reviewed published meta-analyses on the ecological impacts of global climate change, evaluating the number of replicates used in the primary studies (ni) and the number of studies or records (k) that were aggregated to calculate a mean effect size. We used the results of the review in a simulation experiment to assess the performance of conventional frequentist and Bayesian meta-analysis methods for estimating a mean effect size and its uncertainty interval. Our literature review showed that ni and k were highly variable, distributions were right-skewed, and were generally small (median ni =5, median k=44). Our simulations show that the choice of method for calculating uncertainty intervals was critical for obtaining appropriate coverage (close to the nominal value of 0.95). When k was low (<40), 95% coverage was achieved by a confidence interval based on the t-distribution that uses an adjusted standard error (the Hartung-Knapp-Sidik-Jonkman, HKSJ), or by a Bayesian credible interval, whereas bootstrap or z-distribution confidence intervals had lower coverage. Despite the importance of the method to calculate the uncertainty interval, 39% of the meta-analyses reviewed did not report the method used, and of the 61% that did, 94% used a potentially problematic method, which may be a consequence of software defaults. In general, for a simple random-effects meta-analysis, the performance of the best frequentist and Bayesian methods were similar for the same combinations of factors (k and mean replication), though the Bayesian approaches had higher than nominal (>95%) coverage for the mean effect when k was very low (k<15). Our literature review suggests that many meta-analyses that used z-distribution or bootstrapping confidence intervals may have over-estimated the statistical significance of their results when the number of studies was low; more appropriate methods need to be adopted in ecological meta-analyses.

54 ENVIRONMENTAL SCIENCES↗

Validation of the AltRAM Physics Models for use with Compressed Natural Gas

The Alternative Fuels Risk Assessment Models (A1tRAM) toolkit combines Quantitative Risk Assessment (QRA) with simulations of unignited dispersion, ignited turbulent diffusion flames, and indoor accumulation with delayed ignition of fuels. The models of the physical phenomena need to be validated for each of the fuels in the toolkit. This report shows the validation for methane which is being used as a surrogate for natural gas. For the unignited dispersion model, seven previously published experiments from credible sources were used to validate. The validation looked at gas concentrations with respect to the distance from the release point. Four of these were underexpanded jets (i.e. release velocity equal to or greater than local speed of sound) and the other three subsonic releases. The methane plume model in AltRAM matched both varieties well, with higher accuracy for the underexpanded releases. For the jet flame model, we compared the heat flux and thermal radiation data reported from five separate turbulent jet flame experiments to the quantities calculated by A1tRAM. Four of the five datasets were for underexpanded diffusion jets flames. While the results still match well enough to give a good estimate of what is occurring, the error is higher than what was seen with the plume model. For the underexpanded flames A1tRAM provided reasonable approximations, which would lead to conservative risk assessments. Some modeling errors can be attributed to environmental effects (i.e. wind) since most large scale flame experiments are conducted outdoors. A1tRAM has been shown to be a reasonably accurate tool for calculating the concentration or flame properties of natural gas releases. Improvements could still be made for the plume of subsonic releases and radiative heat fluxes to reduce the conservative nature of these predictions. These models can provide valuable information for the risk assessment of natural gas infrastructure.

03 NATURAL GAS↗

Comments on "The Sensitivity Study of Radiative-Convective Equilibrium in the Tropics with a Convective Resolving Model"

In general, there are two broad scientific objectives when using cloud resolving models (CRMs or cloud ensemble models-CEMs) to study tropical convection. The first one is to use them as a physics resolving models to understand the dynamic and microphysical processes associated with the tropical water and energy cycles and their role in the climate system. The second approach is to use the CRMs to improve the representation of moist processes and their interaction with radiation in large-scale models. In order to improve the credibility of the CRMs and achieve the above goals, CRMs using identical initial conditions and large-scale influences need to produce very similar results. Two CRMs produced different statistical equilibrium (SE) states even though both used the same initial thermodynamic and wind conditions. Sensitivity tests to identify the major physical processes that determine the SE states for the different CRM simulations were performed. Their results indicated that atmospheric horizontal wind is treated quite differently in these two CRMs. The model that had stronger surface winds and consequently larger latent and sensible heat fluxes from the ocean produced a warmer and more humid modeled thermodynamic SE state. In addition, the domain mean thermodynamic state is more unstable for those experiments that produced a warmer and more humid SE state. Their simulated wet (warm and humid) SE states are thermally more stable in the lower troposphere (from the surface to 4-5 km in altitude). The large-scale horizontal advective effects on temperature and water vapor mixing ratio are needed when using CRMs to perform long-term integrations to study convective feedback under specified large-scale environments. In addition, it is suggested that the dry and cold SE state simulated was caused by enhanced precipitation but not enough surface evaporation. We find some problems with the interpretation of these three phenomena.

Tao, W.-K.↗

Integrated Modeling and Simulation Verification, Validation, and Accreditation Strategy for Exploration Systems Mission Directorate

Models and simulations (M&S) are critical resources in the exploration of space. They support program management, systems engineering, integration, analysis, test, and operations and provide critical information and data supporting key analyses and decisions (technical, cost and schedule). Consequently, there is a clear need to establish a solid understanding of M&S strengths and weaknesses, and the bounds within which they can credibly support decision-making. Their usage requires the implementation of a rigorous approach to verification, validation and accreditation (W&A) and establishment of formal process and practices associated with their application. To ensure decision-making is suitably supported by information (data, models, test beds) from activities (studies, exercises) from M&S applications that are understood and characterized, ESMD is establishing formal, tailored W&A processes and practices. In addition, to ensure the successful application of M&S within ESMD, a formal process for the certification of analysts that use M&S is being implemented. This presentation will highlight NASA's Exploration Systems Mission Directorate (ESMD) management approach for M&S W&A to ensure decision-makers receive timely information on the model's fidelity, credibility, and quality.

Hale, Joseph P.↗

The atmospheric effects of stratospheric aircraft. Report of the 1992 Models and Measurements Workshop. Volume 1: Workshop objectives and summary

This Workshop on Stratospheric Models and Measurements (M&M) marks a significant expansion in the history of model intercomparisons. It provides a foundation for establishing the credibility of stratospheric models used in environmental assessments of chlorofluorocarbons, aircraft emissions, and climate-chemistry interactions. The core of the M&M comparisons involves the selection of observations of the current stratosphere (i.e., within the last 15 years): these data are believed to be accurate and representative of certain aspects of stratospheric chemistry and dynamics that the models should be able to simulate.

Prather, Michael J.↗

The atmospheric effects of stratospheric aircraft. Report of the 1992 Models and Measurements Workshop. Volume 2: Comparisons with global atmospheric measurements

This Workshop on Stratospheric Models and Measurements (M&M) marks a significant expansion in the history of model intercomparisons. It provides a foundation for establishing the credibility of stratospheric models used in environmental assessments of chlorofluorocarbons, aircraft emissions, and climate-chemistry interactions. The core of the M&M comparisons involves the selection of observations of the current stratosphere (i.e., within the last 15 years): these data are believed to be accurate and representative of certain aspects of stratospheric chemistry and dynamics that the models should be able to simulate.

Prather, Michael J.↗

The atmospheric effects of stratospheric aircraft. Report of the 1992 Models and Measurements Workshop. Volume 3: Special diagnostic studies

This Workshop on Stratospheric Models and Measurements (M&M) marks a significant expansion in the history of model intercomparisons. It provides a foundation for establishing the credibility of stratospheric models used in environmental assessments of chlorofluorocarbons, aircraft emissions, and climate-chemistry interactions. The core of the M&M comparisons involves the selection of observations of the current stratosphere (i.e., within the last 15 years): these data are believed to be accurate and representative of certain aspects of stratospheric chemistry and dynamics that the models should be able to simulate.

Prather, Michael J.↗

Sleep Disruption Medical Intervention Forecasting (SDMIF) Module for the Integrated Medical Model

The NASA Integrated Medical Model (IMM) assesses the risk, including likelihood and impact of occurrence, of all credible in-flight medical conditions. Fatigue due to sleep disruption is a condition that could lead to operational errors, potentially resulting in loss of mission or crew. Pharmacological consumables are mitigation strategies used to manage the risks associated with sleep deficits. The likelihood of medical intervention due to sleep disruption was estimated with a well validated sleep model and a Monte Carlo computer simulation in an effort to optimize the quantity of consumables. METHODS: The key components of the model are the mission parameter program, the calculation of sleep intensity and the diagnosis and decision module. The mission parameter program was used to create simulated daily sleep/wake schedules for an ISS increment. The hypothetical schedules included critical events such as dockings and extravehicular activities and included actual sleep time and sleep quality. The schedules were used as inputs to the Sleep, Activity, Fatigue and Task Effectiveness (SAFTE) Model (IBR Inc., Baltimore MD), which calculated sleep intensity. Sleep data from an ISS study was used to relate calculated sleep intensity to the probability of sleep medication use, using a generalized linear model for binomial regression. A human yes/no decision process using a binomial random number was also factored into sleep medication use probability. RESULTS: These probability calculations were repeated 5000 times resulting in an estimate of the most likely amount of sleep aids used during an ISS mission and a 95% confidence interval. CONCLUSIONS: These results were transferred to the parent IMM for further weighting and integration with other medical conditions, to help inform operational decisions. This model is a potential planning tool for ensuring adequate sleep during sleep disrupted periods of a mission.

Lewandowski, Beth↗

SWAT model application for evaluating agricultural conservation practice effectiveness in reducing phosphorous loss from the Western Lake Erie Basin

Lake Erie is threatened by eutrophication and harmful algal blooms due to excess nutrient loading from agricultural sources. Agricultural conservation practices (ACPs) have been developed and implemented to reduce nutrient losses but estimating ACP effectiveness is challenging. The Soil and Water Assessment Tool (SWAT) has been used to investigate ACP effectiveness for water quality improvement. Many SWAT applications have been developed by different investigators to evaluate ACP effectiveness for reducing nutrient, particularly phosphorus (P), loading in the agriculturally-dominated Western Lake Erie Basin (WLEB). Our objective is to establish what has been achieved by past modeling research and make suggestions for future applications and improvements. We synthesized the findings of 28 SWAT modeling studies within the WLEB. Models generally performed satisfactorily against accepted criteria for streamflow and sediment, but performance for P loads, like soluble reactive P, was mostly “unsatisfactory”. The “unsatisfactory” performance maybe due to imperfections and idealizations in model formulations and/or parameterization. Thus, simulations of P transport and transformation processes need improvement. In addition, model parameter selection is the key part of model set-up. Most SWAT modeling studies used default values during initial set-up, then performed calibration and validation. It was found that the calibrated P related parameter values varied widely across different studies, even within the same watershed with some values unrealistic for the study areas. The phenomena of different combinations of model parameters producing similar outputs indicates equifinality. Equifinality in the baseline model may impact results when ACPs are incorporated. Furthermore, the unrealistic values used in ACP assessment undermine the credibility of ACP effectiveness. Future model applications should try to re-examine the calibrated P parameters and make sure they are realistic for the study area as well as reduce equifinality by constraining the model with characterization of watershed conditions, better understanding of hydrologic processes, and parameter values based on real-world observations. As a result, future model applications should focus on improving P transport and transformation processes, using measured watershed characteristics for parameterization, and improving reflections of climate change, which could result in more accurate assessments of ACP effectiveness to meet targeted goals.

54 ENVIRONMENTAL SCIENCES↗

Deep learning model for fast, science-based forecasting of fluid migration along faults in geologic carbon storage scenarios

Effective long-term geologic storage depends on robust site selection and credible, science-based forecasting of subsurface behavior to ensure storage integrity. For this work, we develop a deep learning–based reduced-order model (ROM) to quantify potential carbon dioxide (CO₂) and brine migration through geological faults. The ROM combines a Transformer model for binary classification and a Stacked Ensemble for regression, trained on a comprehensive dataset generated from 1400 physics-based reservoir simulations. Key geologic and operational parameters—including fault geometry, reservoir structure, and injection conditions—were systematically varied to capture a wide range of fluid migration scenarios. The ROM accurately predicts the onset of migration, cumulative migration volumes of both CO₂ and brine, and associated migration rates, as compared to an independent set of validation simulations, while significantly reducing computational cost compared to traditional simulation methods. Model performance was evaluated across diverse fault configurations, revealing that shallow reservoir geometry and fault angle are among the most influential factors governing migration behavior. Sensitivity analysis using SHapley Additive exPlanations (SHAP) provided interpretability, revealing distinct patterns in how geological and operational features drive transient versus cumulative migration outcomes. The ROM’s ability to rapidly simulate fault migration scenarios enables efficient sensitivity analyses, scenario evaluations, and decision support for site selection and monitoring design. This approach enhances the safety, scalability, and long-term operational performance of geologic carbon storage (GCS) systems by providing a robust, interpretable tool for predicting subsurface fluid migration and assessing fault-related migration potential.

42 ENGINEERING↗

Hierarchical Inference of the Lensing Convergence from Photometric Catalogs with Bayesian Graph Neural Networks

Abstract We present a Bayesian graph neural network (BGNN) that can estimate the weak lensing convergence ( κ ) from photometric measurements of galaxies along a given line of sight (LOS). The method is of particular interest in strong gravitational time-delay cosmography (TDC), where characterizing the “external convergence” ( κ ext ) from the lens environment and LOS is necessary for precise Hubble constant ( H 0 ) inference. Starting from a large-scale simulation with a κ resolution of ∼1′, we introduce fluctuations on galaxy–galaxy lensing scales of ∼1″ and extract random sight lines to train our BGNN. We then evaluate the model on test sets with varying degrees of overlap with the training distribution. For each test set of 1000 sight lines, the BGNN infers the individual κ posteriors, which we combine in a hierarchical Bayesian model to yield constraints on the hyperparameters governing the population. For a test field well sampled by the training set, the BGNN recovers the population mean of κ precisely and without bias (within the 2 σ credible interval), resulting in a contribution to the H 0 error budget well under 1%. In the tails of the training set with sparse samples, the BGNN, which can ingest all available information about each sight line, extracts a stronger κ signal compared to a simplified version of the traditional method based on matching galaxy number counts, which is limited by sample variance. Our hierarchical inference pipeline using BGNNs promises to improve the κ ext characterization for precision TDC. The code is available as a public Python package, Node to Joy ⏬ .

79 ASTRONOMY AND ASTROPHYSICS↗

Dynamical and Trace Gas Responses of the Quasi-Biennial Oscillation to Increased CO2

Long-term projections of the Quasi-Biennial Oscillation (QBO) remain uncertain. Using the high-top \Middle Atmosphere" (MA) version of the NASA Goddard Institute for Space Studies (GISS) ModelE (E2.2-AP), we show that increased CO2 reduces the QBO period and weakens its amplitude, consistent with results from recent multimodel studies. The amplitude response is asymmetric, with the easterly phase weakening more than the westerly phase. Results from both coupled atmosphere{ocean and climatologically fixed sea surface temperature simulations link the period response to changes in parameterized convective gravity wave fluxes, whereas the amplitude changes are more closely tied to changes in tropical upwelling. Comparisons of simulations with and without ozone-dynamical feedbacks reveal that ozone feedbacks increase the QBO amplitude and period in pre-industrial control simulations, and reduce the response of the amplitude to increased CO2. The response of stratospheric tracers to projected changes in the QBO is also investigated. Similarity between results using fully interactive (trace gas and aerosol) versus linearized ozone chemical mechanisms suggests that the QBO response to CO2 does not depend on additional chemical feedbacks beyond those associated with ozone, which implies that projected QBO changes may be credibly represented using simpli fied stratospheric ozone schemes.

GISS Model E2.2-AP↗

Metrics for evaluating tropical cyclones in climate data

This manuscript describes a software suite that can be used for objective evaluation of tropical cyclones (TCs) in gridded climate data. Using cyclone trajectories derived from 6-hourly data, a comprehensive set of metrics is defined to systematically compare and contrast products to one another. In addition to annual TC climatologies, attention is paid to spatial and temporal patterns of storm occurrence and intensity. Assessment can be performed either on the global scale or regional domains. Simple to visualize ‘scorecards’ allow for rapid credibility assessment. We showcase three key findings enabled by this suite. First, we compare the representation of TCs in seven current-generation global reanalyses and conclude that higher resolution models and those with TC-specific assimilation contain more accurate storm climatologies. Second, using a free-running Earth system model (ESM) we find that full basin refinement is required in variable-resolution configurations to adequately simulate North Atlantic TC frequency. Furthermore, upstream refinement over northern Africa offers little benefit in simulating storm occurrence but spatial genesis patterns are improved. Finally, we show that TCs simulated by ESMs can be highly sensitive to individual parameterizations in climate models, with North Atlantic TC metrics varying greatly depending on version of the Morrison-Gettelman microphysics package.

54 ENVIRONMENTAL SCIENCES↗

SPARTA: A flux adjustment methodology to interpret complex experiments

For the accurate determination of reactivity from a detector count rate, correction of spatial effects is of prime importance. This spatial correction is often provided using simulation methodologies, but this may introduce a bias if the result of the experiment is also used as input data for the simulation. Here, this work presents a flux adjustment methodology able to infer experimental reactivity and correction of spatial effects without the need for a simulation. It can process the signal from a complex experiment such as a heat balance measurement in the TREAT reactor, where control rods are continuously adjusted to maintain a constant power. In the present work, this methodology successfully computed the reactivity and the local spatial variation of the flux of a generated signal. It also proved to be robust against noise and errors on kinetic parameters and provides a credible interpretation of a heat balance experiment in TREAT. Efficiency of flux adjustment methods for complex experiment enable a better experiment interpretation less reliant on nuclear data evaluation.

73 - NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Characterization of the Space Shuttle Ascent Debris using CFD Methods

After video analysis of space shuttle flight STS-107's ascent showed that an object shed from the bipod-ramp region impacted the left wing, a transport analysis was initiated to determine a credible flight path and impact velocity for the piece of debris. This debris transport analysis was performed both during orbit, and after the subsequent re-entry accident. The analysis provided an accurate prediction of the velocity a large piece of foam bipod ramp would have as it impacted the wing leading edge. This prediction was corroborated by video analysis and fully-coupled CFD/six degree of freedom (DOF) simulations. While the prediction of impact velocity was accurate enough to predict critical damage in this case, one of the recommendations of the Columbia Accident Investigation Board (CAIB) for return-to-flight (RTF) was to analyze the complete debris environment experienced by the shuttle stack on ascent. This includes categorizing all possible debris sources, their probable geometric and aerodynamic characteristics, and their potential for damage. This paper is chiefly concerned with predicting the aerodynamic characteristics of a variety of potential debris sources (insulating foam and cork, nose-cone ablator, ice, ...) for the shuttle ascent configuration using CFD methods. These aerodynamic characteristics are used in the debris transport analysis to predict flight path, impact velocity and angle, and provide statistical variation to perform risk analyses where appropriate. The debris aerodynamic characteristics are difficult to determine using traditional methods, such as static or dynamic test data, due to the scaling requirements of simulating a typical debris event. The use of CFD methods has been a critical element for building confidence in the accuracy of the debris transport code by bridging the gap between existing aerodynamic data and the dynamics of full-scale, in-flight events.

Murman, Scott M.↗