Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

A new indirect measure of diffusion model error

We define a new indirect measure of the diffusion model error called the diffusion model error source. When this model error source is added to the diffusion equation, the transport solution for the angular-integrated intensity is obtained. This source represents a means by which a transport code can be used to generate information relating to the adequacy of diffusion theory for any given problem without actually solving the diffusion equation. The generation of this source does not relate in any way to acceleration of the iterative convergence of transport solutions. Perhaps the most well-known indirect measure of the diffusion model error is the variable-Eddington tensor. This tensor provides a great deal of information about the angular dependence of the angular intensity solution, but it is not always simple to interpret. In contrast, our diffusion model error source is a scalar that is conceptually easy to understand. In addition to defining the diffusion model error source analytically, we show how to generate this source numerically relative to the S{sub n} radiative transfer equations with linear-discontinuous spatial discretization. This numerical source is computationally tested and shown to reproduce the Sn solution for a Marshak-wave problem. (authors)

97 MATHEMATICS AND COMPUTING↗

Combined selection of the dynamic model and modeling error in nonlinear aeroelastic systems using Bayesian Inference

Here, we report a Bayesian framework for concurrent selection of physics-based models and (modeling) error models. We investigate the use of colored noise to capture the mismatch between the predictions of calibrated models and observational data that cannot be explained by measurement error alone within the context of Bayesian estimation for stochastic ordinary differential equations. Proposed models are characterized by the average data-fit, a measure of how well a model fits the measurements, and the model complexity measured using the Kullback–Leibler divergence. The use of a more complex error models increases the average data-fit but also increases the complexity of the combined model, possibly over-fitting the data. Bayesian model selection is used to find the optimal physical model as well as the optimal error model. The optimal model is defined using the evidence, where the average data-fit is balanced by the complexity of the model. The effect of colored noise process is illustrated using a nonlinear aeroelastic oscillator representing a rigid NACA0012 airfoil undergoing limit cycle oscillations due to complex fluid–structure interactions. Several quasi-steady and unsteady aerodynamic models are proposed with colored noise or white noise for the model error. The use of colored noise improves the predictive capabilities of simpler models.

42 ENGINEERING↗

Impact of transport model errors on the global and regional methane emissions estimated by inverse modelling

A modelling experiment has been conceived to assess the impact of transport model errors on methane emissions estimated in an atmospheric inversion system. Synthetic methane observations, obtained from 10 different model outputs from the international TransCom-CH4 model inter-comparison exercise, are combined with a prior scenario of methane emissions and sinks, and integrated into the three-component PYVAR-LMDZ-SACS (PYthon VARiational-Laboratoire de Météorologie Dynamique model with Zooming capability-Simplified Atmospheric Chemistry System) inversion system to produce 10 different methane emission estimates at the global scale for the year 2005. The same methane sinks, emissions and initial conditions have been applied to produce the 10 synthetic observation datasets. The same inversion set-up (statistical errors, prior emissions, inverse procedure) is then applied to derive flux estimates by inverse modelling. Consequently, only differences in the modelling of atmospheric transport may cause differences in the estimated fluxes. Here in our framework, we show that transport model errors lead to a discrepancy of 27 Tg yr -1 at the global scale, representing 5% of total methane emissions. At continental and annual scales, transport model errors are proportionally larger than at the global scale, with errors ranging from 36 Tg yr -1 in North America to 7 Tg yr -1 in Boreal Eurasia (from 23 to 48%, respectively). At the model grid-scale, the spread of inverse estimates can reach 150% of the prior flux. Therefore, transport model errors contribute significantly to overall uncertainties in emission estimates by inverse modelling, especially when small spatial scales are examined. Sensitivity tests have been carried out to estimate the impact of the measurement network and the advantage of higher horizontal resolution in transport models. The large differences found between methane flux estimates inferred in these different configurations highly question the consistency of transport model errors in current inverse systems.

54 ENVIRONMENTAL SCIENCES↗

Characterizing the impact of model error in hydrologic time series recovery inverse problems

Hydrologic models are commonly over-smoothed relative to reality, owing to computational limitations and to the difficulty of obtaining accurate high-resolution information. When used in an inversion context, such models may introduce systematic biases which cannot be encapsulated by an unbiased “observation noise” term of the type assumed by standard regularization theory and typical Bayesian formulations. Despite its importance, model error is difficult to encapsulate systematically and is often neglected. In this paper, model error is considered for an important class of inverse problems that includes interpretation of hydraulic transients and contaminant source history inference: reconstruction of a time series that has been convolved against a transfer function (i.e., impulse response) that is only approximately known. Using established harmonic theory along with two results established here regarding triangular Toeplitz matrices, upper and lower error bounds are derived for the effect of systematic model error on time series recovery for both well-determined and over-determined inverse problems. It is seen that use of additional measurement locations does not improve expected performance in the face of model error. A Monte Carlo study of a realistic hydraulic reconstruction problem is presented, and the lower error bound is seen informative about expected behavior. Finally, a possible diagnostic criterion for blind transfer function characterization is also uncovered.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence

Abstract Computational models of the Earth System are critical tools for modern scientific inquiry. Efforts toward evaluating and improving errors in representations of physical and chemical processes in these large computational systems are commonly stymied by highly nonlinear and complex error behavior. Recent work has shown that these errors can be effectively predicted using modern Artificial Intelligence (A.I.) techniques. In this work, we go beyond these previous studies to apply an explainable A.I. technique to not only predict model errors but also move toward understanding the underlying reasons for successful error prediction. We use XGBoost classification trees and SHapley Additive exPlanations analysis to explore the errors in the prediction of lightning occurrence in the NASA Goddard Earth Observing System model, a widely used Earth System Model. This explainable error prediction system can effectively predict the model error and indicates that the errors are strongly related to convective processes and the characteristics of the land surface.

54 ENVIRONMENTAL SCIENCES↗

Stochastic modeling and statistical calibration with model error and scarce data

This paper introduces a procedure to assess the predictive accuracy of stochastic models subject to model error and sparse data. Model error is introduced as uncertainty on the coefficients of appropriate polynomial chaos expansions (PCE). The error associated with finite sample size allows us to conceive of these coefficients as statistics of the data that we describe as random variables whose influence on output quantities of interest is evaluated through the extended polynomial chaos expansion (EPCE). A Bayesian data assimilation scheme is introduced to update these expansions by considering the resulting nested chaos expansion as a hierarchical probabilistic model. Stochastic models of quantities of interest (QoI) are thus constructed and efficiently evaluated. Here, the Metropolis–Hastings Markov chain Monte Carlo procedure is used to sample the posterior. Two illustrative analytical and numerical problems are used to demonstrate the proposed approach.

Bayesian inference↗

Stochastic multiscale modeling for quantifying statistical and model errors with application to composite materials

This paper provides a coherent and efficient computational framework for stochastic multiscale analysis of material systems in the presence of parametric uncertainties and modeling errors. Uncertainty in those model parameters that are not deduced as upscaled quantities is attributed to an uncertainty “germ”. While such parameters can appear at any scale, they are predominant at the finest analysis scale. Additional uncertainties stemming from statistical estimation, attributed to lack of data and model error, are associated with each submodel contributing to the multiscale system. Here, a robust and efficient framework based on a generalized extended polynomial chaos expansion (gEPCE) is proposed to simultaneously propagate all these uncertainties in order to provide a probabilistic representation of specific quantities of interest (QoI). We characterize the full probability distribution of the QoI and the uncertainty in the failure probability pertaining to its tails. By combining gEPCE with kernel density estimation (KDE) and directional derivatives, we construct sensitivity measures that connect these statistical metrics of QoI to the various sources of uncertainty to assess their individual and combined impacts. An illustrative problem featuring three-point bending of a composite beam is investigated to demonstrate the presented approach.

36 MATERIALS SCIENCE↗

Spatial Variability of Winds and HRRR–NCEP Model Error Statistics at Three Doppler-Lidar Sites in the Wind-Energy Generation Region of the Columbia River Basin

Annually and seasonally averaged wind profiles from three Doppler lidars were gathered from sites in the Columbia River basin of east-central Oregon and Washington, a major region of wind-energy production, for the Second Wind Forecast Improvement Project (WFIP2) experiment. The profile data are used to quantify the spatial variability of wind flows in this area of complex terrain, to assess the HRRR-NCEP model's ability to capture spatial and temporal variability of wind profiles, and to evaluate model errors. Annually averaged measured wind speed differences over the 70-km extent of the lidar measurements reached 1 m s -1 within the wind-turbine rotor layer, and 2 m s -1 for 200-500 m AGL. Stronger wind speeds in the lowest 500 m occurred at sites higher in elevation, farther from the river, and farther west - closer to the Cascade Mountain barrier. Validating against the lidar data, the HRRR model underestimated strong wind speeds (>12 m s -1 ) and, consequently, their frequency of occurrence, especially at the two lowest-elevation sites, producing annual low biases in rotor-layer wind speed of 0.5 m s -1 . The RMSE between measured and modeled winds at all sites was about 3 m s -1 and did not degrade significantly with forecast lead time. The nature of the model errors was different for different seasons. Furthermore, although the three sites were located in the same basin terrain, the nature of the model errors was different at each site. Thus, if only one of the sites had been instrumented, different conclusions would have been drawn as to the major sources of model error, depending on where the measurements were made.

17 WIND ENERGY↗

Test models for improving filtering with model errors through stochastic parameter estimation

The filtering skill for turbulent signals from nature is often limited by model errors created by utilizing an imperfect model for filtering. Updating the parameters in the imperfect model through stochastic parameter estimation is one way to increase filtering skill and model performance. Here a suite of stringent test models for filtering with stochastic parameter estimation is developed based on the Stochastic Parameterization Extended Kalman Filter (SPEKF). These new SPEKF-algorithms systematically correct both multiplicative and additive biases and involve exact formulas for propagating the mean and covariance including the parameters in the test model. A comprehensive study is presented of robust parameter regimes for increasing filtering skill through stochastic parameter estimation for turbulent signals as the observation time and observation noise are varied and even when the forcing is incorrectly specified. The results here provide useful guidelines for filtering turbulent signals in more complex systems with significant model errors.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Efficient flexible characterization of quantum processors with nested error models

We present a simple and powerful technique for finding a good error model for a quantum processor. The technique iteratively tests a nested sequence of models against data obtained from the processor, and keeps track of the best-fit model and its wildcard error (a metric of the amount of unmodeled error) at each step. Each best-fit model, along with a quantification of its unmodeled error, constitutes a characterization of the processor. We explain how quantum processor models can be compared with experimental data and to each other. We demonstrate the technique by using it to characterize a simulated noisy two-qubit processor.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Accuracy of Some Approximate Gaussian Filters for the Navier--Stokes Equation in the Presence of Model Error

Bayesian state estimation of a dynamical system from a stream of noisy measurements is important in many geophysical and engineering applications where high dimensionality of the state space, sparse observations, and model error pose key challenges. In this work, three computationally feasible, approximate Gaussian data assimilation/filtering algorithms are considered in various regimes of turbulent 2D Navier--Stokes dynamics in the presence of model error. The first source of error arises from the necessary use of reduced models for the forward dynamics of the filters, while a particular type of representation error arises from the finite resolution of observations which mix up information about resolved and unresolved dynamics. Two stochastically parameterized filtering algorithms, referred to as cSPEKF and GCF, are compared with 3DVAR---a prototypical time-sequential algorithm known to be accurate for filtering dissipative systems for a suitably inflated “background” covariance. We provide the first evidence that the stochastically parameterized algorithms, which do not rely on detailed knowledge of the underlying dynamics and do not require covariance inflation, can compete with or outperform an optimally tuned 3DVAR algorithm, and they can overcome competing sources of error in a range of dynamical scenarios.

97 MATHEMATICS AND COMPUTING↗

A robust dynamic state estimation approach against model errors caused by load changes

Dynamic state estimation (DSE) plays an important role in power system security monitoring and online control. In practice, there are two approaches to implementing DSE. The first approach is distributed DSE, which is based on the assumption that the terminal bus of each generator can be measured by PMUs (phasor measurement units). The assumption cannot be satisfied currently, however, because PMUs usually are installed at important high-voltage buses such as 500-kV buses installed in portions of the grid overseen by the Western Electricity Coordinating Council. Another issue of this approach is that performance of DSE is vulnerable to bad measurement data. The reason for this vulnerability is that DSE is performed separately through measurements at each terminal bus, and measurements at terminal buses are the only measurement upon which DSE can rely. Therefore, important redundant measurements are not included in this approach. The second approach is centralized DSE. This approach does not have the requirement for PMU location, and redundant measurements can be considered fully. However, load changes and grid topology changes impact centralized DSE. In this paper, we propose a new approach for handling the impact of load changes on DSE. We have developed a new algorithm that includes two sequential steps. In the first step, errors caused by load changes are detected by analyzing the difference between prediction results and measured results. In the second step, once model error is detected, a model optimization procedure is run to correct the error so the state estimation error can be mitigated. Simulation results from the IEEE 68 bus system show that the proposed approach can effectively handle model errors caused by load changes.

robust dynamic state estimation, load change, powe↗

Measurements and Model Improvement: Insight into NWP Model Error Using Doppler Lidar and Other WFIP2 Measurement Systems

Abstract Doppler-lidar wind-profile measurements at three sites were used to evaluate NWP model errors from two versions of NOAA’s 3-km-grid HRRR model, to see whether updates in the latest version 4 reduced errors when compared against the original version 1. Nested (750-m grid) versions of each were also tested to see how grid spacing affected forecast skill. The measurements were part of the field phase of the Second Wind Forecasting Improvement Project (WFIP2), an 18-month deployment into central Oregon–Washington, a major wind-energy-producing region. This study focuses on errors in simulating marine intrusions, a summertime, 600–800-m-deep, regional sea-breeze flow found to generate large errors. HRRR errors proved to be complex and site dependent. The most prominent error resulted from a premature drop in modeled marine-intrusion wind speeds after local midnight, when lidar-measured winds of greater than 8 m s −1 persisted through the next morning. These large negative errors were offset at low levels by positive errors due to excessive mixing, complicating the interpretation of model “improvement,” such that the updates to the full-scale versions produced mixed results, sometimes enhancing but sometimes degrading model skill. Nesting consistently improved model performance, with version 1’s nest producing the smallest errors overall. HRRR’s ability to represent the stages of sea-breeze forcing was evaluated using radiation budget, surface-energy balance, and near-surface temperature measurements available during WFIP2. The significant site-to-site differences in model error and the complex nature of these errors mean that field-measurement campaigns having dense arrays of profiling sensors are necessary to properly diagnose and characterize model errors, as part of a systematic approach to NWP model improvement. Significance Statement Dramatic increases in NWP model skill will be required over the coming decades. This paper describes the role of major deployments of accurate profiling sensors in achieving that goal and presents an example from the Second Wind Forecast Improvement Program (WFIP2). Wind-profile data from scanning Doppler lidars were used to evaluate two versions of HRRR, the original and an updated version, and nested versions of each. This study focuses on the ability of updated HRRR versions to improve upon predicting a regional sea-breeze flow, which was found to generate large errors by the original HRRR. Updates to the full-scale HRRR versions produced mixed results, but the finer-mesh versions consistently reduced model errors.

Meteorology & Atmospheric Sciences↗