Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

SU-F-BRD-07: Empirical Derivation of Site-Specific Margin Formulas

Purpose: To empirically derive margin formulas from existing clinical radiotherapy plans accounting for respiratory motion and setup uncertainties. Methods: We simulated realistic treatment scenarios, including respiratory motion and setup errors. Individual probability density functions (PDF) from respiratory data were used to simulate respiratory motion. Random (σ) and systematic (Σ) setup errors were modeled as Gaussian distributions. One-dimensional dose profiles were extracted from existing radiotherapy plans and convolved with respiratory PDFs and random error distributions to produce blurred dose profiles. Each blurred dose profile was then shifted 1000 times by randomly sampling the simulated systematic error distribution. Margins were determined from the distance between the simulated treatment and the original 95% isodose level. An equation was fit for each (σ, Σ) combination to derive margin formulas for 90% of the population receiving 95% dose. This methodology can be applied to different tumor sites. Here, dose profiles were extracted from partial breast 3DCRT plans in the anterior-posterior (AP) and superior-inferior (SI) directions. Respiratory motion data was from healthy volunteers, and a clinically relevant range of random and systematic setup errors (standard deviations 1 – 4 mm) was determined from the literature. Results: The PBI margin formulas in the AP and SI directions for 95% dose coverage for 90% of the population were very similar: M= 0.68σ + 1.54Σ and M= 0.72σ + 1.50Σ, respectively. Systematic setup errors had the largest influence on required margin size, whereas realistic respiratory amplitude had minimal impact. The derived formulas resulted in a smaller systematic component than commonly-used theoretical margin recipes. Conclusion: We have demonstrated a method to derive empirical margin formulas from existing patient radiotherapy plans, incorporating realistic respiratory motion and appropriate ranges of random and systematic error. Site-specific margin formulas may be better suited to patient populations than theoretically derived, ideal geometry general margin formulas.

60 APPLIED LIFE SCIENCES↗

Impact of numerics on the predictive capabilities of reacting flow LES

Combustors in modern aerospace propulsion systems are highly optimized devices and further enhancements to their performance will require advanced predictive modeling techniques. The LES approach has potential to be such a technique, but demonstration of its true predictive capability remains a challenge. In this paper, a methodical study is performed to assess the impact of numerical error on the predictive capabilities of LES and to distinguish it from error arising from physical modeling. Four codes are employed to simulate the bluff body stabilized flame experiment of Volvo Flygmotor AB. The physical models (governing equations and sub-grid models) are kept identical between the codes so that the physical modeling error is the same. For the same reason, computational grids are also kept identical since the sub-grid models are grid dependent. Results indicate that all codes produce a very similar solution for the non-reacting flow in terms of large scale unsteady vortex shedding behavior, mean profiles and second order turbulence statistics, on a suitably refined grid. This solution is also very close to the experimentally observed behavior of the flow. In contrast, for the reacting case, solutions from the various codes exhibit significant differences when analyzed under the same metrics and with similar or even finer grid resolution than that used for the non-reacting case. These differences are large enough to change the macroscopic behavior of the flow, e.g. some codes predict symmetric vortex shedding while others predict asymmetric shedding and some codes show strong recirculation zones while others show stagnant regions behind the bluff body. It is also shown that none of the solutions truly predicts the experimentally observed flow. Given that the physical sub-grid models and computational grids are the same in all the simulations, the inconsistency between results points to a significant impact of numerical methods and, hence, numerical error. The need for mitigation of this impact is highlighted, along with the need for metrics detailing how LES should be applied to reacting flows. It is suggested that both of these issues should be the focus of future work.

Energy & Fuels↗

Magellan adaptive optics first-light observations of the exoplanet β PIC b. II. 3–5 μm direct imaging with MagAO+Clio, and the empirical bolometric luminosity of a self-luminous giant planet

Young giant exoplanets are a unique laboratory for understanding cool, low-gravity atmospheres. A quintessential example is the massive extrasolar planet β Pic b, which is 9 AU from and embedded in the debris disk of the young nearby A6V star β Pictoris. We observed the system with first light of the Magellan Adaptive Optics (MagAO) system. In Paper I we presented the first CCD detection of this planet with MagAO+VisAO. Here we present four MagAO+Clio images of β Pic b at 3.1 μm, 3.3 μm, L′, and M{sup ′}, including the first observation in the fundamental CH{sub 4} band. To remove systematic errors from the spectral energy distribution (SED), we re-calibrate the literature photometry and combine it with our own data, for a total of 22 independent measurements at 16 passbands from 0.99 to 4.8 μm. Atmosphere models demonstrate the planet is cloudy but are degenerate in effective temperature and radius. The measured SED now covers >80% of the planet’s energy, so we approach the bolometric luminosity empirically. We calculate the luminosity by extending the measured SED with a blackbody and integrating to find log(L{sub bol}/L{sub ⊙}) = −3.78±0.03. From our bolometric luminosity and an age of 23 ± 3 Myr, hot-start evolutionary tracks give a mass of 12.7 ± 0.3 M{sub Jup}, radius of 1.45 ± 0.02 R{sub Jup}, and T{sub eff} of 1708 ± 23 K (model-dependent errors not included). Our empirically determined luminosity is in agreement with values from atmospheric models (typically −3.8 dex), but brighter than values from the field-dwarf bolometric correction (typically −3.9 dex), illustrating the limitations in comparing young exoplanets to old brown dwarfs.

79 ASTRONOMY AND ASTROPHYSICS↗

A Dynamic Approach to Modeling Dependence Between Human Failure Events

In practice, most HRA methods use direct dependence from THERP—the notion that error be- gets error, and one human failure event (HFE) may increase the likelihood of subsequent HFEs. In this paper, we approach dependence from a simulation perspective in which the effects of human errors are dynamically modeled. There are three key concepts that play into this modeling: (1) Errors are driven by performance shaping factors (PSFs). In this context, the error propagation is not a result of the presence of an HFE yielding overall increases in subsequent HFEs. Rather, it is shared PSFs that cause dependence. (2) PSFs have qualities of lag and latency. These two qualities are not currently considered in HRA methods that use PSFs. Yet, to model the effects of PSFs, it is not simply a matter of identifying the discrete effects of a particular PSF on performance. The effects of PSFs must be considered temporally, as the PSFs will have a range of effects across the event sequence. (3) Finally, there is the concept of error spilling. When PSFs are activated, they not only have temporal effects but also lateral effects on other PSFs, leading to emergent errors. This paper presents the framework for tying together these dynamic dependence concepts.

dependence↗

Quantitative validation of carbon-fiber laminate low velocity impact simulations

Simulations of low velocity impact with a flat cylindrical indenter upon a carbon fiber fabric reinforced polymer laminate are rigorously validated. Comparison of the impact energy absorption between the model and experiment is used as the validation metric. Additionally, non-destructive evaluation, including ultrasonic scans and three-dimensional computed tomography, provide qualitative validation of the models. The simulations include delamination, matrix cracks and fiber breaks. An orthotropic damage and failure constitutive model, capable of predicting progressive damage and failure, is developed in conjunction and described. An ensemble of simulations incorporating model parameter uncertainties is used to predict a response distribution which is then compared to experimental output using appropriate statistical methods. Lastly, the model form errors are exposed and corrected for use in an additional blind validation analysis. The result is a quantifiable confidence in material characterization and model physics when simulating low velocity impact in structures of interest.

36 MATERIALS SCIENCE↗

Theoretical modeling of a bottom-raised oscillating surge wave energy converter structural loadings and power performances

Here, this study presents theoretical formulations to evaluate the fundamental parameters and performance characteristics of a bottom-raised oscillating surge wave energy converter (OSWEC) device. Employing a flat plate assumption and potential flow formulation in elliptical coordinates, closed-form equations for the added mass, radiation damping, and excitation forces/torques in the relevant pitch-pitch and surge-pitch directions of motion are developed and used to calculate the system's response amplitude operator and the forces and moments acting on the foundation. The model is benchmarked against numerical simulations using WAMIT and WEC-Sim, showcasing excellent agreement. The sensitivity of plate thickness on the analytical hydrodynamic solutions is investigated over several thickness-to-width ratios ranging from 1:80 to 1:10. The results show that as the thickness of the benchmark OSWEC increases, the deviation of the analytical hydrodynamic coefficients from the numerical solutions grows from 3% to 25%. Differences in the excitation forces and torques, however, are contained within 12%. While the flat plate assumption is a limitation of the proposed analytical model, the error is within a reasonable margin for use in the design space exploration phase before a higher-fidelity (and thus more computationally expensive) model is employed. A parametric study demonstrates the ability of the analytical model to quickly sweep over a domain of OSWEC dimensions, illustrating the analytical model's utility in the early phases of design.

13 HYDRO ENERGY↗

Characterizing Performance of Freshwater Wetland Methane Models Across Time Scales at FLUXNET-CH 4 Sites Using Wavelet Analyses

Process-based land surface models are important tools for estimating global wetland methane (CH 4 ) emissions and projecting their behavior across space and time. So far there are no performance assessments of model responses to drivers at multiple time scales. In this study, we apply wavelet analysis to identify the dominant time scales contributing to model uncertainty in the frequency domain. We evaluate seven wetland models at 23 eddy covariance tower sites. Our study first characterizes site-level patterns of freshwater wetland CH 4 fluxes (FCH 4 ) at different time scales. A Monte Carlo approach was developed to incorporate flux observation error to avoid misidentification of the time scales that dominate model error. Our results suggest that (a) significant model-observation disagreements are mainly at multi-day time scales (<15 days); (b) most of the models can capture the CH 4 variability at monthly and seasonal time scales (>32 days) for the boreal and Arctic tundra wetland sites but have significant bias in variability at seasonal time scales for temperate and tropical/subtropical sites; (c) model errors exhibit increasing power spectrum as time scale increases, indicating that biases at time scales <5 days could contribute to persistent systematic biases on longer time scales; and (d) differences in error pattern are related to model structure (e.g., proxy of CH 4 production). Our evaluation suggests the need to accurately replicate FCH 4 variability, especially at short time scales, in future wetland CH 4 model developments.

58 GEOSCIENCES↗

Impact of point spread function higher moments error on weak gravitational lensing – II. A comprehensive study

ABSTRACT Weak lensing is one of the most powerful probes for dark matter and dark energy science, although it faces increasing challenges in controlling systematic uncertainties as the statistical errors become smaller. The point spread function (PSF) needs to be precisely modeled to avoid systematic error on the weak lensing measurements. The weak lensing biases induced by errors in the PSF model second moments, i.e. its size and shape, are well-studied. However, Zhang et al. showed that errors in the higher moments of the PSF may also be a significant source of systematics for upcoming weak lensing surveys. Therefore, this work comprehensively investigate the modelling quality of PSF moments from the 3rd to 6th order, and propagate the PSFEx higher moments modelling error in the HSC survey data set to the weak lensing shear–shear correlation functions and their cosmological analyses. The overall multiplicative shear bias associated with errors in PSF higher moments can cause an ∼0.1σ shift on the cosmological parameters for LSST Y10, while the associated additive biases can induce 1σ uncertainties in cosmology parameter inference for LSST Y10, if not accounted. We compare the PSFEx model with PSF in Full FOV, and find similar performance in modelling the PSF higher moments. We conclude that PSF higher moment errors of the future PSF models should be reduced from those in current methods, otherwise needed to be explicitly modeled in the weak lensing analysis.

79 ASTRONOMY AND ASTROPHYSICS↗

Embedded Error Bayesian Calibration of Thermal Decomposition of Organic Materials

Organic materials are an attractive choice for structural components due to their light weight and versatility. However, because they decompose at low temperatures relative to tradiational materials they pose a safety risk due to fire and loss of structural integrity. To quantify this risk, analysts use chemical kinetics models to describe the material pyrolysis and oxidation using thermogravimetric analysis. This process requires the calibration of many model parameters to closely match experimental data. Previous efforts in this field have largely been limited to finding a single best-fit set of parameters even though the experimental data may be very noisy. Furthermore the chemical kinetics models are often simplified representations of the true de- composition process. The simplification induces model-form errors that the fitting process cannot capture. In this work we propose a methodology for calibrating decomposition models to thermogravimetric analysis data that accounts for uncertainty in the model-form and experimental data simultaneously. The methodology is applied to the decomposition of a carbon fiber epoxy composite with a three-stage reaction network and Arrhenius kinetics. The results show a good overlap between the model predictions and thermogravimetric analysis data. Uncertainty bounds capture devia- tions of the model from the data. The calibrated parameter distributions are also presented. In conclusion, the distributions may be used in forward propagation of uncertainty in models that leverage this material.

36 MATERIALS SCIENCE↗

Data Assimilation - Advances and Applications

This presentation provides an overview of data assimilation (model calibration) for complex computer experiments. Calibration refers to the process of probabilistically constraining uncertain physics/engineering model inputs to be consistent with observed experimental data. An initial probability distribution for these parameters is updated using the experimental information. Utilization of surrogate models and empirical adjustment for model form error in code calibration form the basis for the statistical methodology considered. The role of probabilistic code calibration in supporting code validation is discussed. Incorporation of model form uncertainty in rigorous uncertainty quantification (UQ) analyses is also addressed. Design criteria used within a batch sequential design algorithm are introduced for efficiently achieving predictive maturity and improved code calibration. Predictive maturity refers to obtaining stable predictive inference with calibrated computer codes. These approaches allow for augmentation of initial experiment designs for collecting new physical data. A standard framework for data assimilation is presented and techniques for updating the posterior distribution of the state variables based on particle filtering and the ensemble Kalman filter are introduced.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Methods for Incorporating Model Uncertainty into Exoplanet Atmospheric Analysis

A key goal of exoplanet spectroscopy is to measure atmospheric properties, such as abundances of chemical species, in order to connect them to our understanding of atmospheric physics and planet formation. In this new era of high-quality JWST data, it is paramount that these measurement methods are robust. When comparing atmospheric models to observations, multiple candidate models may produce reasonable fits to the data. Typically, conclusions are reached by selecting the best-performing model according to some metric. This ignores model uncertainty in favor of specific model assumptions, potentially leading to measured atmospheric properties that are overconfident and/or incorrect. In this paper, we compare three ensemble methods for addressing model uncertainty by combining posterior distributions from multiple analyses: Bayesian model averaging, a variant of Bayesian model averaging using leave-one-out predictive densities, and stacking of predictive distributions. We demonstrate these methods by fitting the Hubble Space Telescope (HST) + Spitzer transmission spectrum of the hot Jupiter HD 209458b using models with different cloud and haze prescriptions. All of our ensemble methods lead to uncertainties on retrieved parameters that are larger but more realistic and consistent with physical and chemical expectations. Since they have not typically accounted for model uncertainty, uncertainties of retrieved parameters from HST spectra have likely been underreported. We recommend stacking as the most robust model combination method. Our methods can be used to combine results from independent retrieval codes and from different models within one code. They are also widely applicable to other exoplanet analysis processes, such as combining results from different data reductions.

79 ASTRONOMY AND ASTROPHYSICS↗

Imperfect slope measurements drive overestimation in a geometric cone model of lake and reservoir depth

Lake and reservoir (waterbody) depth is a critical characteristic that influences many important ecological processes. Unfortunately, depth measurements are labor-intensive to gather and are only available for a small fraction of waterbodies globally. Therefore, scientists have tried to predict depth from characteristics easily obtained for all waterbodies, such as surface area or the slope of the surrounding land. One approach for predicting waterbody depth simulates basins using a geometric cone model where the nearshore land slope and distance to the center of the waterbody are assumed to be representative proxies for in-lake slope and distance to the deepest point respectively. We tested these assumptions using bathymetry data from ~5000 lakes and reservoirs to examine whether differences in waterbody type or shape influenced depth prediction error. Here, we found that nearshore land slope was not representative of in-lake slope, and using it for prediction increases error substantially relative to models using true in-lake slope for all waterbody types and shapes. Predictions were biased toward overprediction in concave waterbodies (i.e., bowl-shaped; up to 18% of the study population) and reservoir waterbodies (up to 30% of the study population). Despite this systematic overprediction, model errors were fewer (in absolute and relative terms, irrespective of any specific slope covariate) for concave than convex waterbodies, suggesting the geometric cone model is an adequate representation of depth for these waterbodies. But because convex waterbodies are far more common (>72% of our study population), minimizing overall depth prediction error remains a challenge.

54 ENVIRONMENTAL SCIENCES↗

Spatio-temporal distribution of Oklahoma earthquakes: Exploring relationships using a nearest-neighbor approach: Nearest-neighbor analysis of Oklahoma

Determining the spatiotemporal characteristics of natural and induced seismic events holds the opportunity to gain new insights into why these events occur. Linking the seismicity characteristics with other geologic, geographic, natural, or anthropogenic factors could help to identify the causes and suggest mitigation strategies that reduce the risk associated with such events. The nearest-neighbor approach utilized in this work represents a practical first step toward identifying statistically correlated clusters of recorded earthquake events. Detailed study of the Oklahoma earthquake catalog’s inherent errors, empirical model parameters, and model assumptions is presented. We found that the cluster analysis results are stable with respect to empirical parameters (e.g., fractal dimension) but were sensitive to epicenter location errors and seismicity rates. Most critically, we show that the patterns in the distribution of earthquake clusters in Oklahoma are primarily defined by spatial relationships between events. This observation is a stark contrast to California (also known for induced seismicity) where a comparable cluster distribution is defined by both spatial and temporal interactions between events. These results highlight the difficulty in understanding the mechanisms and behavior of induced seismicity but provide insights for future work.

58 GEOSCIENCES↗

Adaptive experimental design for multi-fidelity surrogate modeling of multi-disciplinary systems

We present an adaptive algorithm for constructing surrogate models of multi-disciplinary systems composed of a set of coupled components. With this goal we introduce “coupling” variables with a priori unknown distributions that allow surrogates of each component to be built independently. Once built, the surrogates of the components are combined to form an integrated-surrogate that can be used to predict system-level quantities of interest at a fraction of the cost of the original model. The error in the integrated-surrogate is greedily minimized using an experimental design procedure that allocates the amount of training data, used to construct each component-surrogate, based on the contribution of those surrogates to the error of the integrated-surrogate. Here, the multi-fidelity procedure presented is a generalization of multi-index stochastic collocation that can leverage ensembles of models of varying cost and accuracy, for one or more components, to reduce the computational cost of constructing the integrated-surrogate. Extensive numerical results demonstrate that, for a fixed computational budget, our algorithm is able to produce surrogates that are orders of magnitude more accurate than methods that treat the integrated system as a black-box.

42 ENGINEERING↗

Hierarchical Modeling and Statistical Calibration for Photometric Redshifts

The cosmological exploitation of modern photometric galaxy surveys requires both accurate (unbiased) and precise (narrow) redshift probability distributions derived from broadband photometry. Existing methodologies do not meet these requirements. Standard template fitting delivers interpretable models and errors, but lacks the flexibility to learn inaccuracies in the observed photometry or spectral templates. Machine learning addresses those issues, but requires representative training data, and the resulting models and uncertainties cannot be interpreted in the context of a physical model or outside of the training data. We present a hierarchical modeling approach simultaneously addressing the issues of flexibility, interpretability, and generalization. It combines template fitting with flexible (machine-learning-like) models to correct the spectral templates, model their redshift distributions, and recalibrate the photometric observations. By optimizing the full posterior distribution of the model and solving for its (thousands of) parameters, one can perform a global statistical calibration of the data and the spectral energy distribution (SED) model. We apply this approach to the public Dark Energy Survey Science Verification data and show that it provides more accurate and compact redshift posterior distributions than existing methods, as well as insights into residual photometric and SED systematics. Here, the model is causal and makes predictions for future data (e.g., additional photometric bandpasses), and its internal parameters and components are interpretable. This approach does not formally require the training data to be complete or representative; in principle, it can even work in regimes in which few or no spectroscopic redshifts are available.

79 ASTRONOMY AND ASTROPHYSICS↗

A Data-Driven Methodology for Probabilistic Wind Power Ramp Forecasting

With increasing wind penetration, wind power ramps (WPRs) are currently drawing great attention to balancing authorities, since these wind ramps largely affect power system operations. To help better manage and dispatch the wind power, this paper develops a data-driven probabilistic WPR forecasting (p-WPRF) method based on a large number of simulated scenarios. A machine learning technique is first adopted to forecast the basic wind power forecasting scenario and produce the historical forecasting errors. To accurately model the distribution of wind power forecasting errors, a generalized Gaussian mixture model is developed and the cumulative distribution function (CDF) is also analytically deduced. The inverse transform method based on the CDF is used to generate a large number of forecasting error scenarios. An optimized swinging door algorithm is adopted to extract all the WPRs from the complete set of wind power forecasting scenarios. The p-WPRF is generated based on all generated scenarios under different weather and time conditions. Numerical simulations on publicly available wind power data show that the developed p-WPRF method can predict WPRs with a high level of reliability and accuracy.

17 WIND ENERGY↗

Development of an open-source regional data assimilation system in PEcAn v. 1.7.2: application to carbon cycle reanalysis across the contiguous US using SIPNET

Abstract. The ability to monitor, understand, and predict the dynamics of the terrestrial carbon cycle requires the capacity to robustly and coherently synthesize multiple streams of information that each provide partial information about different pools and fluxes. In this study, we introduce a new terrestrial carbon cycle data assimilation system, built on the PEcAn model–data eco-informatics system, and its application for the development of a proof-of-concept carbon “reanalysis” product that harmonizes carbon pools (leaf, wood, soil) and fluxes (GPP, Ra, Rh, NEE) across the contiguous United States from 1986–2019. We first calibrated this system against plant trait and flux tower net ecosystem exchange (NEE) using a novel emulated hierarchical Bayesian approach. Next, we extended the Tobit–Wishart ensemble filter (TWEnF) state data assimilation (SDA) framework, a generalization of the common ensemble Kalman filter which accounts for censored data and provides a fully Bayesian estimate of model process error, to a regional-scale system with a calibrated localization. Combined with additional workflows for propagating parameter, initial condition, and driver uncertainty, this represents the most complete and robust uncertainty accounting available for terrestrial carbon models. Our initial reanalysis was run on an irregular grid of ∼ 500 points selected using a stratified sampling method to efficiently capture environmental heterogeneity. Remotely sensed observations of aboveground biomass (Landsat LandTrendr) and leaf area index (LAI) (MODIS MOD15) were sequentially assimilated into the SIPNET model. Reanalysis soil carbon, which was indirectly constrained based on modeled covariances, showed general agreement with SoilGrids, an independent soil carbon data product. Reanalysis NEE, which was constrained based on posterior ensemble weights, also showed good agreement with eddy flux tower NEE and reduced root mean square error (RMSE) compared to the calibrated forecast. Ultimately, PEcAn's new open-source regional data assimilation framework provides a scalable workflow for harmonizing multiple data constraints and providing a uniform synthetic platform for carbon monitoring, reporting, and verification (MRV) as well as accelerating terrestrial carbon cycle research.

54 ENVIRONMENTAL SCIENCES↗

Domain-aware Control-oriented Neural Models for Autonomous Underwater Vehicles

Conventional physics-based modeling is a time-consuming bottleneck in control design for complex nonlinear systems like autonomous underwater vehicles (AUVs). In contrast, purely data-driven models, require a large number of observations and lack operational guarantees for safety-critical systems. Data-driven models leveraging available partially characterized dynamics have potential to provide reliable systems models in a typical data-limited scenario for high value complex systems, thereby avoiding months of expensive expert modeling time. In this work we explore this middle-ground between expert-modeled and pure data-driven modeling. We present control-oriented parametric models with varying levels of domain-awareness that exploit known system structure and prior physics knowledge to create constrained deep neural dynamical system models. We employ universal differential equations to construct data-driven blackbox and graybox representations of the AUV dynamics. In addition, we explore a hybrid formulation that explicitly models the residual error related to imperfect graybox models. We compare the prediction performance of the learned models for different distributions of initial conditions and control inputs to assess their suitability for control.

Shaw Cortez, Wenceslao E.↗