Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model Uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,531 records · Page 85

The uncertainty in ozone calculations by a stratospheric photochemistry model

At present, there is an apparent conflict between one-dimensional stratospheric photochemistry models used in predicting ozone depletion and average data for stratospheric ozone. This conflict is in three particulars - column density of O3 and ozone density in the regions around 30 km and 50 km altitude. A study of the sensitivity of one such model to the values of reaction rates, boundary conditions, solar intensities, photolysis cross sections, and O(1D) yield parameters reveals that even at the 2-sigma uncertainty limit due to these input parameters, the model does not overlap the data for O3 density at 30 km and 50 km. The data is outside the 1-sigma model uncertainty limit for O3 column density. The study also shows the relative contribution of the various parameters studied to the imprecision in these model results.

Butler, D. M.↗

Parametrized uncertainties in the spectral function model of neutrino charged-current quasielastic interactions for oscillation analyses

A substantial fraction of systematic uncertainties in neutrino oscillation experiments stem from the lack of precision in modeling the nuclear target in neutrino-nucleus interactions. Whilst this has driven significant progress in the development of improved nuclear models for neutrino scattering, it is crucial that the models used in neutrino data analyses be accompanied by parameters and associated uncertainties that allow the coverage of plausible nuclear physics. Based on constraints from electron scattering data, we propose such a set of parameters, which can be applied to nuclear shell models, and test their application to the Benhar [] spectral function model. The parametrization is validated through a series of maximum likelihood fits to cross section measurements made by the T2K and MINERvA experiments, which also permit an exploration of the power of near-detector data to provide constraints on the parameters in neutrino oscillation analyses. Published by the American Physical Society 2024

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA Langley's Approach to the Sandia's Structural Dynamics Challenge Problem

The objective of this challenge is to develop a data-based probabilistic model of uncertainty to predict the behavior of subsystems (payloads) by themselves and while coupled to a primary (target) system. Although this type of analysis is routinely performed and representative of issues faced in real-world system design and integration, there are still several key technical challenges that must be addressed when analyzing uncertain interconnected systems. For example, one key technical challenge is related to the fact that there is limited data on target configurations. Moreover, it is typical to have multiple data sets from experiments conducted at the subsystem level, but often samples sizes are not sufficient to compute high confidence statistics. In this challenge problem additional constraints are placed as ground rules for the participants. One such rule is that mathematical models of the subsystem are limited to linear approximations of the nonlinear physics of the problem at hand. Also, participants are constrained to use these models and the multiple data sets to make predictions about the target system response under completely different input conditions. Our approach involved initially the screening of several different methods. Three of the ones considered are presented herein. The first one is based on the transformation of the modal data to an orthogonal space where the mean and covariance of the data are matched by the model. The other two approaches worked solutions in physical space where the uncertain parameter set is made of masses, stiffnesses and damping coefficients; one matches confidence intervals of low order moments of the statistics via optimization while the second one uses a Kernel density estimation approach. The paper will touch on all the approaches, lessons learned, validation 1 metrics and their comparison, data quantity restriction, and assumptions/limitations of each approach. Keywords: Probabilistic modeling, model validation, uncertainty quantification, kernel density

Horta, Lucas G.↗

End-Use Load Profiles for the U.S. Building Stock: Methodology and Results of Model Calibration, Validation, and Uncertainty Quantification

The United States is embarking on an ambitious transition to a 100% clean energy economy by 2050, which will require improving the flexibility of electric grids. One way to achieve grid flexibility is to shed or shift demand to align with changing grid needs. To facilitate this, it is critical to understand how and when energy is used. High- quality end-use load profiles (EULPs) provide this information, and can help cities, states, and utilities understand the time-sensitive value of energy efficiency, demand response, and distributed energy resources. Publicly available EULPs have traditionally had limited application because of age and incomplete geographic representation (Frick, Eckman, and Goldman 2017; Frick 2019). To help fill this gap, the U.S. Department of Energy (DOE) funded a three-year project - End-Use Load Profiles for the U.S. Building Stock - that culminated in the release of a publicly available dataset1 of simulated EULPs representing residential and commercial buildings across the contiguous United States. The motivation for this work is further detailed in a November 2019 report: Market Needs, Use Cases, and Data Gaps (Mims Frick et al. 2019). This Methodology and Results report provides detailed descriptions of how the dataset was developed, intended for an audience of dataset and model users interested in the technical details. These details include descriptions of all of the model improvements made for calibration and the final comparisons to empirical data sources. A companion report, End-Use Load Profiles for the U.S. Building Stock: Applications and Opportunities, will be published subsequently and will describe example applications and considerations for using the dataset, intended for an audience of general dataset users.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Using Ecosystem Experiments to Improve Vegetation Models

Ecosystem responses to rising CO2 concentrations are a major source of uncertainty in climate change projections. Data from ecosystem-scale Free-Air CO2 Enrichment (FACE) experiments provide a unique opportunity to reduce this uncertainty. The recent FACE Model–Data Synthesis project aimed to use the information gathered in two forest FACE experiments to assess and improve land ecosystem models. A new 'assumption-centred' model intercomparison approach was used, in which participating models were evaluated against experimental data based on the ways in which they represent key ecological processes. Identifying and evaluating the main assumptions caused differences among models, and the assumption-centered approach produced a clear roadmap for reducing model uncertainty. We explain this approach and summarize the resulting research agenda. We encourage the application of this approach in other model intercomparison projects to fundamentally improve predictive understanding of the Earth system.

54 ENVIRONMENTAL SCIENCES↗

Correlation and prediction uncertainties in the CyberKnife Synchrony respiratory tracking system

Purpose: The CyberKnife uses an online prediction model to improve radiation delivery when treating lung tumors. This study evaluates the prediction model used by the CyberKnife radiation therapy system in terms of treatment margins about the gross tumor volume (GTV). Methods: From the data log files produced by the CyberKnife synchrony model, the uncertainty in radiation delivery can be calculated. Modeler points indicate the tracked position of the tumor and Predictor points predict the position about 115 ms in the future. The discrepancy between Predictor points and their corresponding Modeler points was analyzed for 100 treatment model data sets from 23 de-identified lung patients. The treatment margins were determined in each anatomic direction to cover an arbitrary volume of the GTV, derived from the Modeler points, when the radiation is targeted at the Predictor points. Each treatment model had about 30 min of motion data, of which about 10 min constituted treatment time; only these 10 min were used in the analysis. The frequencies of margin sizes were analyzed and truncated Gaussian normal functions were fit to each direction's distribution. The standard deviation of each Gaussian distribution was then used to describe the necessary margin expansions in each signed dimension in order to achieve the desired coverage. In this study, 95% modeler point coverage was compared to 99% modeler coverage. Two other error sources were investigated: the correlation error and the targeting error. These were added to the prediction error to give an aggregate error for the CyberKnife during treatment of lung tumors. Results: Considering the magnitude of 2{sigma} from the mean of the Gaussian in each signed dimension, the margin expansions needed for 95% modeler point coverage were 1.2 mm in the lateral (LAT) direction and 1.7 mm in the anterior-posterior (AP) direction. For the superior-inferior (SI) direction, the fit was poor; but empirically, the expansions were 3.5 mm. For 99% modeler point coverage, the AP margin was 3.6 mm and the lateral margin was 2.9 mm. The SI margins for 99% modeler point coverage were highly variable. The aggregate error at 95% was 6.9 mm in the SI direction, 4.6 mm in the AP direction, and 3.5 in the lateral direction. Conclusions: The Predictor points follow the Modeler points closely. Margins were found in each clinical direction that would provide 95% modeler point coverage for 95% of the models reviewed in this study. Similar margins were found in two clinical directions for 99% modeler point coverage in 95% of models. These results can offer guidance in the selection of CTV margins for treatment with the CyberKnife.

61 RADIATION PROTECTION AND DOSIMETRY↗

Sequential Design of Experiments for Pilot Testing of Novel Solvent System

The CCSI2 program is supporting a six-month test campaign at the National Carbon Capture Center (NCCC) for evaluation of a novel water-lean solvent. This presentation describes CCSI2’s efforts in process modeling of the solvent system for both coal and natural gas-based flue gas sources and initial uncertainty quantification (UQ) work to estimate parametric uncertainty in key sub-models of interest (e.g., thermodynamics, mass transfer, reaction kinetics). Moreover, perspective is provided on how UQ and sequential design of experiments (SDoE) tools are used to assess the impact of model uncertainty on projected process performance, use this information to optimize data collection during the campaign, and refine process models through data collection. This framework is expected to reduce the overall model uncertainty, and thus risk associated with scale-up as the process moves towards commercialization.

Morgan, Joshua↗

Energy sector portfolio analysis with uncertainty

Governments are dealing with the challenge of how to efficiently invest in research and development portfolios related to energy technologies. Research and development investment decisions in the energy space are especially difficult due to numerous risks and uncertainties, and due to the complexity of energy's interactions with the broad economy. Historically, much of the U.S. Department of Energy's in-depth research and development analyses focused on assessing the impact of a research and development activity in isolation from other available opportunities and did not substantially consider risk and uncertainty. Endeavoring to combine integrated energy-economy modeling with uncertainty analysis and technology-specific research and development activities, the U.S. Department of Energy commissioned the development of the Stochastic Energy Deployment System to support and improve public energy research and development decision-making. The Stochastic Energy Deployment System draws from expert-elicited probability distributions for research and development-driven improvements in technology cost and performance, and it uses Monte Carlo simulations to evaluate the likelihood of outcomes within a system dynamics energy-economy model. The framework estimates the uncertain benefits and costs of various research and development portfolios and provides insight into the probability of meeting national technology goals, while accounting for interactions with the larger economy and for interactions among research and development investments spanning many energy sectors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Uncertainty Analysis in 3D Equilibrium Reconstruction

Reconstruction is an inverse process where a parameter space is searched to locate a set of parameters with the highest probability of describing experimental observations. Due to systematic errors and uncertainty in experimental measurements, this optimal set of parameters will contain some associated uncertainty. This uncertainty in the optimal parameters leads to uncertainty in models derived using those parameters. V3FIT is a three-dimensional (3D) equilibrium reconstruction code that propagates uncertainty from the input signals, to the reconstructed parameters, and to the final model. Here in this paper, we describe the methods used to propagate uncertainty in V3FIT. Using the results of whole shot 3D equilibrium reconstruction of the Compact Toroidal Hybrid, this propagated uncertainty is validated against the random variation in the resulting parameters. Two different model parameterizations demonstrate how the uncertainty propagation can indicate the quality of a reconstruction. As a proxy for random sampling, the whole shot reconstruction results in a time interval that will be used to validate the propagated uncertainty from a single time slice.

97 MATHEMATICS AND COMPUTING↗

Avian collision risk models for wind energy impact assessments

With the increasing global development of wind energy, collision risk models (CRMs) are routinely used to assess the potential impacts of wind turbines on birds. We reviewed and compared the avian collision risk models currently available in the scientific literature, exploring aspects such as the calculation of a collision probability, inclusion of stationary components e.g. the tower, angle of approach and uncertainty. 10 models were cited in the literature and of these, all included a probability of collision of a single bird colliding with a wind turbine during passage through the rotor swept area, and the majority included a measure of the number of birds at risk. 7 out of the 10 models calculated the probability of birds colliding, whilst the remainder used a constant. We identified four approaches to calculate the probability of collision and these were used by others. 6 of the 10 models were deterministic and included the most frequently used models in the UK, with only 4 including variation or uncertainty in some way, the most recent using Bayesian methods. Despite their appeal, CRMs have their limitations and can be ‘data hungry’ as well as assuming much about bird movement and behaviour. As data become available, these assumptions should be tested to ensure that CRMs are functioning to adequately answer the questions posed by the wind energy sector. - Highlights: • We highlighted ten models available to assess avian collision risk. • Only 4 of the models included variability or uncertainty. • Collision risk models have limitations and can be ‘data hungry’. • It is vital that the most appropriate model is used for a given task.

54 ENVIRONMENTAL SCIENCES↗

Quantitative Feedback Theory (QFT) applied to the design of a rotorcraft flight control system

Quantitative Feedback Theory describes a frequency-domain technique for the design of multi-input, multi-output control systems which meet time or frequency domain performance criteria when specified uncertainty exists in the linear description of the vehicle dynamics. Quantitative Feedback Theory is applied to the design of the longitudinal flight control system for a linear uncertain model of the AH-64 rotorcraft. In this model, the uncertainty is assigned, and is assumed to be attributable to actual uncertainty in the dynamic model and to the changes in the vehicle aerodynamic characteristics which occur near hover. The model includes an approximation to the rotor and actuator dynamics. The design example indicates the manner in which handling qualities criteria may be incorporated into the design of realistic rotorcraft control systems in which significant uncertainty exists in the vehicle model.

Hess, R. A.↗

Sensitivity and uncertainty analysis of PWR spent fuel observables to operational and model parameters

Sensitivity and uncertainty analyses of spent nuclear fuel (SNF) observables - decay heat, neutron and γ-ray emission rate - to operational and model parameters have been performed. A 2D model representing a typical PWR 17x17 UO{sub 2} fuel assembly has been taken as reference. The Serpent code and ENDF/B-VII.1 evaluated nuclear data library have been used for the analyses. Relative uncertainty of decay heat, neutron and γ-ray emission rates due to Monte Carlo counting statistics, as well as the number of fuel pin radial regions required for an accurate characterization of SNF neutron emission, have been estimated. A parametric study has been carried out to investigate the impact of the homogenization to an average value of the fuel and moderator temperature, boron concentration power and burnup on the prediction of the selected observables. In addition, linear sensitivity coefficients have been calculated. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Upscaling Uncertainty with Dynamic Discrepancy for a Multi-Scale Carbon Capture System

Uncertainties from model parameters and model discrepancy from small-scale models impact the accuracy and reliability of predictions of large-scale systems. Inadequate representation of these uncertainties may result in inaccurate and overconfident predictions during scale-up to larger systems. Hence, multiscale modeling efforts must accurately quantify the effect of the propagation of uncertainties during upscaling. Using a Bayesian approach, we calibrate a small-scale solid sorbent model to thermogravimetric (TGA) data on a functional profile using chemistry-based priors. Crucial to this effort is the representation of model discrepancy, which uses a Bayesian smoothing splines (BSS-ANOVA) framework. Our uncertainty quantification (UQ) approach could be considered intrusive as it includes the discrepancy function within the chemical rate expressions; resulting in a set of stochastic differential equations. Such an approach allows for easily propagating uncertainty by propagating the joint model parameter and discrepancy posterior into the larger-scale system of rate expressions. In conclusion, the broad UQ framework presented here could be applicable to virtually all areas of science where multiscale modeling is used. Supplementary materials for this article are available online.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Application of Bayesian model averaging to measurements of the primordial power spectrum

Cosmological parameter uncertainties are often stated assuming a particular model, neglecting the model uncertainty, even when Bayesian model selection is unable to identify a conclusive best model. Bayesian model averaging is a method for assessing parameter uncertainties in situations where there is also uncertainty in the underlying model. We apply model averaging to the estimation of the parameters associated with the primordial power spectra of curvature and tensor perturbations. We use CosmoNest and MultiNest to compute the model evidences and posteriors, using cosmic microwave data from WMAP, ACBAR, BOOMERanG, and CBI, plus large-scale structure data from the SDSS DR7. We find that the model-averaged 95% credible interval for the spectral index using all of the data is 0.940<n{sub s}<1.000, where n{sub s} is specified at a pivot scale 0.015 Mpc{sup -1}. For the tensors model averaging can tighten the credible upper limit, depending on prior assumptions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DEMONSTRATION OF LEACHXS/ORCHESTRA CAPABILITIES BY SIMULATING CONSTITUENT RELEASE FROM A CEMENTITIOUS WASTE FORM IN A REINFORCED CONCRETE VAULT

The objective of the work described in this report is to demonstrate the capabilities of the current version of LeachXS{trademark}/ORCHESTRA for simulating chemical behavior and constituent release processes in a range of applications that are relevant to the CBP. This report illustrates the use of LeachXS{trademark}/ORCHESTRA for the following applications: (1) Comparing model and experimental results for leaching tests for a range of cementitious materials including cement mortars, grout, stabilized waste, and concrete. The leaching test data includes liquid-solid partitioning as a function of pH and release rates based on laboratory column, monolith, and field testing. (2) Modeling chemical speciation of constituents in cementitious materials, including liquid-solid partitioning and release rates. (3) Evaluating uncertainty in model predictions based on uncertainty in underlying composition, thermodynamic, and transport characteristics. (4) Generating predominance diagrams to evaluate predicted chemical changes as a result of material aging using the example of exposure to atmospheric conditions. (5) Modeling coupled geochemical speciation and diffusion in a three layer system consisting of a layer of Saltstone, a concrete barrier, and a layer of soil in contact with air. The simulations show developing concentration fronts over a time period of 1000 years. (6) Modeling sulfate attack and cracking due to ettringite formation. A detailed example for this case is provided in a separate article by the authors (Sarkar et al. 2010). Finally, based on the computed results, the sensitive input parameters for this type of modeling are identified and discussed. The chemical speciation behavior of substances is calculated for a batch system and also in combination with transport and within a three layer system. This includes release from a barrier to the surrounding soil as a function of time. As input for the simulations, the physical and chemical properties of the materials are used. The test cases used in this demonstration are taken from Reference Cases for Use in the Cementitious Barriers Partnership (Langton et al. 2009). Before it is possible to model the release of substances from stabilized waste or radioactive grout through a cement barrier into the engineered soil barrier or natural soil, the relevant characteristics of such materials must be known. Additional chemical characteristics are needed for mechanistic modeling to be undertaken, not just the physical properties relevant for modeling of transport. The minimum required properties for modeling are given in Section 5.0, 'Modeling the chemical speciation of a material'.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Quantifying uncertainty from aerosol and atmospheric parameters and their impact on climate sensitivity

Abstract. Climate sensitivity in Earth system models (ESMs) is an emergentproperty that is affected by structural (missing or inaccurate model physics)and parametric (variations in model parameters) uncertainty. This workprovides the first quantitative assessment of the role of compensationbetween uncertainties in aerosol forcing and atmospheric parameters, andtheir impact on the climate sensitivity of the Community Atmosphere Model,Version 4 (CAM4). Running the model with prescribed ocean and ice conditions,we perturb four parameters related to sulfate and black carbon aerosolradiative forcing and distribution, as well as five atmospheric parametersrelated to clouds, convection, and radiative flux. In this experimental setupwhere aerosols do not affect the properties of clouds, the atmosphericparameters explain the majority of variance in climate sensitivity, with twoparameters being the most important: one controlling low cloud amount, andone controlling the timescale for deep convection. Although the aerosolparameters strongly affect aerosol optical depth, their impacts on climatesensitivity are substantially weaker than the impacts of the atmosphericparameters, but this result may depend on whether aerosol–cloud interactionsare simulated. Based on comparisons to inter-model spread of other ESMs, weconclude that structural uncertainties in this configuration of CAM4 likelycontribute 3 times more to uncertainty in climate sensitivity thanparametric uncertainties. We provide several parameter sets that couldprovide plausible (measured by a skill score) configurations of CAM4, butwith different sulfate aerosol radiative forcing, black carbon radiativeforcing, and climate sensitivity.

54 ENVIRONMENTAL SCIENCES↗

Uncertainty Quantification Analysis of Both Experimental and CFD Simulation Data of a Bench-scale Fluidized Bed Gasifier

Adequate assessment of the uncertainties in modeling and simulation is becoming an integral part of the simulation based engineering design. The goal of this study is to demonstrate the application of non-intrusive Bayesian uncertainty quantification (UQ) methodology in multiphase (gas-solid) flows with experimental and simulation data, as part of our research efforts to determine the most suited approach for UQ of a bench scale fluidized bed gasifier. UQ analysis was first performed on the available experimental data. Global sensitivity analysis performed as part of the UQ analysis shows that among the three operating factors, steam to oxygen ratio has the most influence on syngas composition in the bench-scale gasifier experiments. An analysis for forward propagation of uncertainties was performed and results show that an increase in steam to oxygen ratio leads to an increase in H2 mole fraction and a decrease in CO mole fraction. These findings are in agreement with the ANOVA analysis performed in the reference experimental study. Another contribution in addition to the UQ analysis is the optimization-based approach to guide to identify next best set of additional experimental samples, should the possibility arise for additional experiments. Hence, the surrogate models constructed as part of the UQ analysis is employed to improve the information gain and make incremental recommendation, should the possibility to add more experiments arise. In the second step, series of simulations were carried out with the open-source computational fluid dynamics software MFiX to reproduce the experimental conditions, where three operating factors, i.e., coal flow rate, coal particle diameter, and steam-to-oxygen ratio, were systematically varied to understand their effect on the syngas composition. Bayesian UQ analysis was performed on the numerical results. As part of Bayesian UQ analysis, a global sensitivity analysis was performed based on the simulation results, which shows that the predicted syngas composition is strongly affected not only by the steam-to-oxygen ratio (which was observed in experiments as well) but also by variation in the coal flow rate and particle diameter (which was not observed in experiments). The carbon monoxide mole fraction is underpredicted at lower steam-to-oxygen ratios and overpredicted at higher steam-to-oxygen ratios. The opposite trend is observed for the carbon dioxide mole fraction. These discrepancies are attributed to either excessive segregation of the phases that leads to the fuel-rich or -lean regions or alternatively the selection of reaction models, where different reaction models and kinetics can lead to different syngas compositions throughout the gasifier. To improve quality of numerical models used, the effect that uncertainties in reaction models for gasification, char oxidation, carbon monoxide oxidation, and water gas shift will have on the syngas composition at different grid resolution, along with bed temperature were investigated. The global sensitivity analysis showed that among various reaction models employed for water gas shift, gasification, char oxidation, the choice of reaction model for water gas shift has the greatest influence on syngas composition, with gasification reaction model being second. Syngas composition also shows a small sensitivity to temperature of the bed. The hydrodynamic behavior of the bed did not change beyond grid spacing of 18 times the particle diameter. However, the syngas concentration continued to be affected by the grid resolution as low as 9 times the particle diameter. This is due to a better resolution of the phasic interface between the gases and solid that leads to stronger heterogeneous reactions. This report is a compilation of three manuscripts published in peer-reviewed journals for the series of studies mentioned above.

01 COAL, LIGNITE, AND PEAT↗