Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Characterizing NWP Model Errors Using Doppler-Lidar Measurements of Recurrent Regional Diurnal Flows: Marine-Air Intrusions into the Columbia-River Basin

Ground-based Doppler-lidar instrumentation provides atmospheric wind data at dramatically improved accuracies and spatial/temporal resolutions. These capabilities have provided new insights into atmospheric flow phenomena, but they also should have a strong role in NWP model improvement. Insight into the nature of model errors can be gained by studying recurrent atmospheric flows, here a regional summertime diurnal sea breeze and subsequent marine-air intrusion into the arid interior of Oregon-Washington, where these winds are an important windenergy resource. These marine intrusions were sampled by three scanning Doppler lidars in the Columbia-River Basin as part of the Second Wind Forecast Improvement Project (WFIP2), using data from summer 2016. Lidar time-height cross sections of wind speed identified eight days when the diurnal-flow cycle, (peak wind speeds at midnight, afternoon minima) was obvious and strong. Eight-day composite time-height cross sections of lidar wind speeds are used to validate those generated by the operational NCEP-HRRR model. HRRR simulated the diurnal wind cycle, but produced errors in the timing of onset and significant errors due to a premature nighttime demise of the intrusion flow, producing low-bias errors of 6 m s-1 . Day-to-day and in the composite, whenever a marine intrusion occurred, HRRR made these same errors. The errors occurred under a range of gradient wind conditions indicating that they resulted from the misrepresentation of physical processes within a limited region around the measurement locations. Because of their generation within a limited geographical area, field-measurement programs can be designed to find and address the sources of these NWP errors.

Banta, Robert M.↗

Assessment of Numerical and Modeling Errors of RANS based Transition Models for Low-Reynolds Numbers 2-D Flows

In this paper we report the outcome of selected workshops organized as part of the NATO Applied Vehicle Technology (AVT)-313 activity Incompressible Laminar-to-Turbulent Flow Transition Study that focused on assessing the numerical and modeling accuracy of the γ−Reθ and γ transition models coupled to the k−ω Shear-Stress Transport (SST) two-equation eddy-viscosity model. Three different test cases involving nominally 2D flow configurations were selected: flow over a flat plate with two different levels of turbulence intensity at the inlet; flow around the Eppler 387 foil at a Reynolds number of 3×10^5 and angles of attack of 1 deg. and 7 deg. flow around the NACA 0015 foil at a Reynolds number of 1.8×10^5 and angles of attack of 5 deg. and10 deg. The flat plate flow conditions correspond to natural and by-pass transition, whereas the other two test cases include laminar separation bubbles that lead to separation-induced transition. For each test case, the selected quantities of interest include both integral and local flow quantities. Geometrically similar grids with a wide range of grid refinement ratios were generated for each of the test cases to allow the estimation of numerical uncertainties for all quantities of interest selected for this study. Several RANS flow solvers were used, employing common grids with the same boundary conditions and mathematical models. Therefore, it is possible to analyze the consistency of the results, i.e., to check if the intervals defined by the different numerical solutions with their respective uncertainties overlap with each other. Modeling errors can also be addressed for the selected flow quantities that have experimental data available. However, the experimental information available in these cases is not sufficient to guarantee that experiments and simulations are performed with the same settings. Nonetheless, the available experimental data is sufficient to guarantee that modeling errors are significantly reduced with the use of the transition models when compared to simulations performed using only the k−ω SST model.

CFD Modeling↗

Study of geopotential error models used in orbit determination error analysis

The uncertainty in the geopotential model is currently one of the major error sources in the orbit determination of low-altitude Earth-orbiting spacecraft. The results of an investigation of different geopotential error models and modeling approaches currently used for operational orbit error analysis support at the Goddard Space Flight Center (GSFC) are presented, with emphasis placed on sequential orbit error analysis using a Kalman filtering algorithm. Several geopotential models, known as the Goddard Earth Models (GEMs), were developed and used at GSFC for orbit determination. The errors in the geopotential models arise from the truncation errors that result from the omission of higher order terms (omission errors) and the errors in the spherical harmonic coefficients themselves (commission errors). At GSFC, two error modeling approaches were operationally used to analyze the effects of geopotential uncertainties on the accuracy of spacecraft orbit determination - the lumped error modeling and uncorrelated error modeling. The lumped error modeling approach computes the orbit determination errors on the basis of either the calibrated standard deviations of a geopotential model's coefficients or the weighted difference between two independently derived geopotential models. The uncorrelated error modeling approach treats the errors in the individual spherical harmonic components as uncorrelated error sources and computes the aggregate effect using a combination of individual coefficient effects. This study assesses the reasonableness of the two error modeling approaches in terms of global error distribution characteristics and orbit error analysis results. Specifically, this study presents the global distribution of geopotential acceleration errors for several gravity error models and assesses the orbit determination errors resulting from these error models for three types of spacecraft - the Gamma Ray Observatory, the Ocean Topography Experiment, and the Cosmic Background Explorer.

Yee, C.↗

Model Error Estimation for the CPTEC Eta Model

Statistical data assimilation systems require the specification of forecast and observation error statistics. Forecast error is due to model imperfections and differences between the initial condition and the actual state of the atmosphere. Practical four-dimensional variational (4D-Var) methods try to fit the forecast state to the observations and assume that the model error is negligible. Here with a number of simplifying assumption, a framework is developed for isolating the model error given the forecast error at two lead-times. Two definitions are proposed for the Talagrand ratio tau, the fraction of the forecast error due to model error rather than initial condition error. Data from the CPTEC Eta Model running operationally over South America are used to calculate forecast error statistics and lower bounds for tau.

Tippett, Michael K.↗

Estimation and Control of Modeling Error: A General Approach to Multiscale Modeling

This chapter describes a class of computational methods designed to handle multiscale modeling of large atomistic or molecular systems. The key to this approach is the estimation of relative modeling error in which errors in quantities of interest produced by averaging or homogenization are computed using a posteriori error estimates. The error is judged to be the relative error between the fine-scale base model and any number of surrogates produced by homogenization. Features of modeling the atomistic to continuum interface are also addressed. Specific applications of the methodology are described that involve analyzing polymer etch barriers that are used in nano-manufacturing of semi-conductors. The basic adaptive modeling strategy employs goal-oriented adaptation in which fine-scale information is systematically added to hybrid molecular-continuum models until the appropriate level of accuracy in certain quantities of interest is achieved.

97 MATHEMATICS AND COMPUTING↗

A new indirect measure of diffusion model error

We define a new indirect measure of the diffusion model error called the diffusion model error source. When this model error source is added to the diffusion equation, the transport solution for the angular-integrated intensity is obtained. This source represents a means by which a transport code can be used to generate information relating to the adequacy of diffusion theory for any given problem without actually solving the diffusion equation. The generation of this source does not relate in any way to acceleration of the iterative convergence of transport solutions. Perhaps the most well-known indirect measure of the diffusion model error is the variable-Eddington tensor. This tensor provides a great deal of information about the angular dependence of the angular intensity solution, but it is not always simple to interpret. In contrast, our diffusion model error source is a scalar that is conceptually easy to understand. In addition to defining the diffusion model error source analytically, we show how to generate this source numerically relative to the S{sub n} radiative transfer equations with linear-discontinuous spatial discretization. This numerical source is computationally tested and shown to reproduce the Sn solution for a Marshak-wave problem. (authors)

97 MATHEMATICS AND COMPUTING↗

Combined selection of the dynamic model and modeling error in nonlinear aeroelastic systems using Bayesian Inference

Here, we report a Bayesian framework for concurrent selection of physics-based models and (modeling) error models. We investigate the use of colored noise to capture the mismatch between the predictions of calibrated models and observational data that cannot be explained by measurement error alone within the context of Bayesian estimation for stochastic ordinary differential equations. Proposed models are characterized by the average data-fit, a measure of how well a model fits the measurements, and the model complexity measured using the Kullback–Leibler divergence. The use of a more complex error models increases the average data-fit but also increases the complexity of the combined model, possibly over-fitting the data. Bayesian model selection is used to find the optimal physical model as well as the optimal error model. The optimal model is defined using the evidence, where the average data-fit is balanced by the complexity of the model. The effect of colored noise process is illustrated using a nonlinear aeroelastic oscillator representing a rigid NACA0012 airfoil undergoing limit cycle oscillations due to complex fluid–structure interactions. Several quasi-steady and unsteady aerodynamic models are proposed with colored noise or white noise for the model error. The use of colored noise improves the predictive capabilities of simpler models.

42 ENGINEERING↗

An analysis of the least-squares problem for the DSN systematic pointing error model

A systematic pointing error model is used to calibrate antennas in the Deep Space Network. The least squares problem is described and analyzed along with the solution methods used to determine the model's parameters. Specifically studied are the rank degeneracy problems resulting from beam pointing error measurement sets that incorporate inadequate sky coverage. A least squares parameter subset selection method is described and its applicability to the systematic error modeling process is demonstrated on Voyager 2 measurement distribution.

Alvarez, L. S.↗

Performance Metrics, Error Modeling, and Uncertainty Quantification

A common set of statistical metrics has been used to summarize the performance of models or measurements-­ the most widely used ones being bias, mean square error, and linear correlation coefficient. They assume linear, additive, Gaussian errors, and they are interdependent, incomplete, and incapable of directly quantifying un­certainty. The authors demonstrate that these metrics can be directly derived from the parameters of the simple linear error model. Since a correct error model captures the full error information, it is argued that the specification of a parametric error model should be an alternative to the metrics-based approach. The error-modeling meth­odology is applicable to both linear and nonlinear errors, while the metrics are only meaningful for linear errors. In addition, the error model expresses the error structure more naturally, and directly quantifies uncertainty. This argument is further explained by highlighting the intrinsic connections between the performance metrics, the error model, and the joint distribution between the data and the reference.

Quantification↗

Statistical analysis of modeling error in structural dynamic systems

The paper presents a generic statistical model of the (total) modeling error for conventional space structures in their launch configuration. Modeling error is defined as the difference between analytical prediction and experimental measurement. It is represented by the differences between predicted and measured real eigenvalues and eigenvectors. Comparisons are made between pre-test and post-test models. Total modeling error is then subdivided into measurement error, experimental error and 'pure' modeling error, and comparisons made between measurement error and total modeling error. The generic statistical model presented in this paper is based on the first four global (primary structure) modes of four different structures belonging to the generic category of Conventional Space Structures (specifically excluding large truss-type space structures). As such, it may be used to evaluate the uncertainty of predicted mode shapes and frequencies, sinusoidal response, or the transient response of other structures belonging to the same generic category.

Hasselman, T. K.↗

Optimal estimation of large structure model errors

In-flight estimation of large structure model errors is usually required as a means of detecting inevitable deficiencies in large structure controller/estimator models. The present paper deals with a least-squares formulation which seeks to minimize a quadratic functional of the model errors. The properties of these error estimates are analyzed. It is shown that an arbitrary model error can be decomposed as the sum of two components that are orthogonal in a suitably defined function space. Relations between true and estimated errors are defined. The estimates are found to be approximations that retain many of the significant dynamics of the true model errors. Current efforts are directed toward application of the analytical results to a reference large structure model.

Rodriguez, G.↗

Impact of transport model errors on the global and regional methane emissions estimated by inverse modelling

A modelling experiment has been conceived to assess the impact of transport model errors on methane emissions estimated in an atmospheric inversion system. Synthetic methane observations, obtained from 10 different model outputs from the international TransCom-CH4 model inter-comparison exercise, are combined with a prior scenario of methane emissions and sinks, and integrated into the three-component PYVAR-LMDZ-SACS (PYthon VARiational-Laboratoire de Météorologie Dynamique model with Zooming capability-Simplified Atmospheric Chemistry System) inversion system to produce 10 different methane emission estimates at the global scale for the year 2005. The same methane sinks, emissions and initial conditions have been applied to produce the 10 synthetic observation datasets. The same inversion set-up (statistical errors, prior emissions, inverse procedure) is then applied to derive flux estimates by inverse modelling. Consequently, only differences in the modelling of atmospheric transport may cause differences in the estimated fluxes. Here in our framework, we show that transport model errors lead to a discrepancy of 27 Tg yr -1 at the global scale, representing 5% of total methane emissions. At continental and annual scales, transport model errors are proportionally larger than at the global scale, with errors ranging from 36 Tg yr -1 in North America to 7 Tg yr -1 in Boreal Eurasia (from 23 to 48%, respectively). At the model grid-scale, the spread of inverse estimates can reach 150% of the prior flux. Therefore, transport model errors contribute significantly to overall uncertainties in emission estimates by inverse modelling, especially when small spatial scales are examined. Sensitivity tests have been carried out to estimate the impact of the measurement network and the advantage of higher horizontal resolution in transport models. The large differences found between methane flux estimates inferred in these different configurations highly question the consistency of transport model errors in current inverse systems.

54 ENVIRONMENTAL SCIENCES↗

Characterizing the impact of model error in hydrologic time series recovery inverse problems

Hydrologic models are commonly over-smoothed relative to reality, owing to computational limitations and to the difficulty of obtaining accurate high-resolution information. When used in an inversion context, such models may introduce systematic biases which cannot be encapsulated by an unbiased “observation noise” term of the type assumed by standard regularization theory and typical Bayesian formulations. Despite its importance, model error is difficult to encapsulate systematically and is often neglected. In this paper, model error is considered for an important class of inverse problems that includes interpretation of hydraulic transients and contaminant source history inference: reconstruction of a time series that has been convolved against a transfer function (i.e., impulse response) that is only approximately known. Using established harmonic theory along with two results established here regarding triangular Toeplitz matrices, upper and lower error bounds are derived for the effect of systematic model error on time series recovery for both well-determined and over-determined inverse problems. It is seen that use of additional measurement locations does not improve expected performance in the face of model error. A Monte Carlo study of a realistic hydraulic reconstruction problem is presented, and the lower error bound is seen informative about expected behavior. Finally, a possible diagnostic criterion for blind transfer function characterization is also uncovered.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Model error estimation for large flexible spacecraft

The basic model obtained for spacecraft control system design applications is used to develop the control and estimation algorithms which constitute integral elements of the control system. The performance of this system is limited on account of modeling errors. The presence of such errors is inevitable in connection with truncated dynamics, parameter uncertainties, neglected nonlinearities, and external disturbances. In many cases, an approximate knowledge of the model errors can only be established by means of an estimation procedure. The present investigation has the objective to outline an approach to estimate errors inherent in a reduced-order model and to illustrate its application to modeling problems arising in spacecraft control system design. The considered procedure makes use of the principle of least-squares. In the discussed application this principle is employed to minimize a quadratic functional of the model errors.

Rodriguez, G.↗

Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence

Abstract Computational models of the Earth System are critical tools for modern scientific inquiry. Efforts toward evaluating and improving errors in representations of physical and chemical processes in these large computational systems are commonly stymied by highly nonlinear and complex error behavior. Recent work has shown that these errors can be effectively predicted using modern Artificial Intelligence (A.I.) techniques. In this work, we go beyond these previous studies to apply an explainable A.I. technique to not only predict model errors but also move toward understanding the underlying reasons for successful error prediction. We use XGBoost classification trees and SHapley Additive exPlanations analysis to explore the errors in the prediction of lightning occurrence in the NASA Goddard Earth Observing System model, a widely used Earth System Model. This explainable error prediction system can effectively predict the model error and indicates that the errors are strongly related to convective processes and the characteristics of the land surface.

54 ENVIRONMENTAL SCIENCES↗