SEARCH · Search NASA
Results for “error models”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Model-form error correction for an agent-based model of infectious diseases
Explore the source record for details and available documents.
Addressing Model Form Error for Time-Dependent Conservation Equations
Model form error of the type considered here is error due to an approximate or incorrect representation of physics by a computational model. Typical approaches to adjust a model based on observed differences between experiment and prediction are to calibrate the model parameters utilizing the observed discrepancies and to develop parameterized additive corrections to the model output. These approaches are generally not suitable if significant physics is missing from the model and the desired quantities of interest for an application are different than those used for calibration. The approach developed here is to build a corrected surrogate solver through a multi- step process: 1) Sampled simulation results are used to develop a surrogate computational solver that maintains the overall conservative principles of the unmodified governing equations, 2) the surrogate solver is applied to candidate linear and non-linear corrector terms to develop corrections that are consistent with the original conservative principles, 3) constant multipliers on the these terms are calibrated using the experimental observations, and 4) the resulting surrogate solver is used to predict application response for the quantity of interest. This approach and several other calibration-based approaches were applied to an example problem based on the diffusive Burgers' equation. While all the approaches provided some model correction when the measure/calibration quantity was the same as that for an application, only the present approach was able to adequately correct the CompSim results when the prediction quantity was different from the calibration quantity.
Semi-Analytical Hierarchical Bayesian Inference of Nonlinear Model Structure in Stochastic Dynamics: Applied to Compartmental Models of Infectious Diseases
A Bayesian computational framework for parsimonious inference in stochastic nonlinear dynamical systems is presented. This framework enables the concurrent estimation of system states, time-varying parameters, time-invariant parameters, and the optimal sparsity structure of the model parameters. Because differential equation-based models are often simplified mechanistic or phenomenological representations, robust inference from noisy measurement data requires explicit treatment of model error and uncertainty. Model error and time-varying parameters can be represented as random processes, enabling inference while making minimal assumptions about the underlying sources of discrepancy and variability. Adopting stochastic differential equation representations affords the model significant flexibility, but can also render it susceptible to overfitting during statistical inversion, where the inferred model may track noise rather than the underlying signal. To alleviate the effects of overfitting and to enable the discovery of the optimal sparse representation of the time-invariant parameters, a Bayesian sparse learning algorithm is embedded within the framework. This sparse learning framework adopts an approximate hierarchical Bayesian setting defined by a series of semi-analytical expressions. The model structure inference framework is validated using a stochastic compartmental model for tracking and forecasting active cases of an infectious disease. Compartmental models describe population-level infectious disease dynamics through interactions among population fractions grouped by disease state. Mathematically, such models consist of a system of coupled ordinary differential equations. This example adopts an expressive compartmental model that includes multiple possible interactions between disease states, motivated by early uncertainty surrounding COVID-19 reinfection dynamics and their implications for long-term epidemic forecasting. The sparse learning exercise permits the inference of a priori unknown epidemiological dynamics from simulated public health data, discovering the nested compartmental model that optimizes the trade-off between average data-fit and model complexity. It is shown that inducing sparsity among the model parameters eliminates redundant interactions between compartments, equivalently revealing the optimal coupling structure between differential equations.
Nonlinear and adaptive estimation techniques in reentry
The development and testing of nonlinear and adaptive estimators for reentry (e.g. space shuttle) navigation and model parameter estimation or identification are reported. Of particular interest is the identifcation of vehicle lift and drag characteristics in real time. Several nonlinear filters were developed and simulated. Adaptive filters for the real time identification of vehicle lift and drag characteristics, and unmodelable acceleration, were also developed and tested by simulation. The simulations feature an uncertain system environment with rather arbitrary model errors, thus providing a definitive test of estimator performance. It was found that nonlinear effects are indeed significant in reentry trajectory estimation and a nonlinear filter is demonstrated which successfully tracks through nonlinearities without degrading the information content of the data. Under the same conditions the usual extended Kalman filter diverges and is useless. The J-adaptive filter is shown to successfully track errors in the modeled vehicle lift and drag characteristics. The same filter concept is also shown to track successfully through rather arbitrary model errors, including lift and drag errors, vehicle mass errors, atmospheric density errors, and wind gust errors.
Stratospheric Assimilation of Chemical Tracer Observations Using a Kalman Filter: Chi-Square Validated Results and Analysis of Variance and Correlation Dynamics - Pt. 2
A Kalman filter system designed for the assimilation of limb-sounding observations of stratospheric chemical tracers, which has four tunable covariance parameters, was developed in Part I (Menard et al. 1998) The assimilation results of CH4 observations from the Cryogenic Limb Array Etalon Sounder instrument (CLAES) and the Halogen Observation Experiment instrument (HALOE) on board of the Upper Atmosphere Research Satellite are described in this paper. A robust (chi)(sup 2) criterion, which provides a statistical validation of the forecast and observational error covariances, was used to estimate the tunable variance parameters of the system. In particular, an estimate of the model error variance was obtained. The effect of model error on the forecast error variance became critical after only three days of assimilation of CLAES observations, although it took 14 days of forecast to double the initial error variance. We further found that the model error due to numerical discretization as arising in the standard Kalman filter algorithm, is comparable in size to the physical model error due to wind and transport modeling errors together. Separate assimilations of CLAES and HALOE observations were compared to validate the state estimate away from the observed locations. A wave-breaking event that took place several thousands of kilometers away from the HALOE observation locations was well captured by the Kalman filter due to highly anisotropic forecast error correlations. The forecast error correlation in the assimilation of the CLAES observations was found to have a structure similar to that in pure forecast mode except for smaller length scales. Finally, we have conducted an analysis of the variance and correlation dynamics to determine their relative importance in chemical tracer assimilation problems. Results show that the optimality of a tracer assimilation system depends, for the most part, on having flow-dependent error correlation rather than on evolving the error variance.
Characterization of Partially Observed Epidemics - Application to COVID-19
This report documents a statistical method for the "real-time" characterization of partially observed epidemics. Observations consist of daily counts of symptomatic patients, diagnosed with the disease. Characterization, in this context, refers to estimation of epidemiological parameters that can be used to provide short-term forecasts of the ongoing epidemic, as well as to provide gross information for the time-dependent infection rate. The characterization problem is formulated as a Bayesian inverse problem, and is predicated on a model for the distribution of the incubation period. The model parameters are estimated as distributions using a Markov Chain Monte Carlo (MCMC) method, thus quantifying the uncertainty in the estimates. The method is applied to the COVID-19 pandemic of 2020, using data at the country, provincial (e.g., states) and regional (e.g. county) levels. The epidemiological model includes a stochastic component due to uncertainties in the incubation period. This model-form uncertainty is accommodated by a pseudo-marginal Metropolis-Hastings MCMC sampler, which produces posterior distributions that reflect this uncertainty. We approximate the discrepancy between the data and the epidemiological model using Gaussian and negative binomial error models; the latter was motivated by the over-dispersed count data. For small daily counts we find the performance of the calibrated models to be similar for the two error models. For large daily counts the negative-binomial approximation is numerically unstable unlike the Gaussian error model. Application of the model at the country level (for the United States, Germany, Italy, etc.) generally provided accurate forecasts, as the data consisted of large counts which suppressed the day-to-day variations in the observations. Further, the bulk of the data is sourced over the duration before the relaxation of the curbs on population mixing, and is not confounded by any discernible country-wide second wave of infections. At the state-level, where reporting was poor or which evinced few infections (e.g., New Mexico), the variance in the data posed some, though not insurmountable, difficulties, and forecasts were able to capture the data with large uncertainty bounds. The method was found to be sufficiently sensitive to discern the flattening of the infection and epidemic curve due to shelter-in-place orders after around 90% quantile for the incubation distribution (about 10 days for COVID-19). The proposed model was also used at a regional level to compare the forecasts for the central and north-west regions of New Mexico. Modeling the data for these regions illustrated different disease spread dynamics captured by the model. While in the central region the daily counts peaked in the late April, in the north-west region the ramp-up continued for approximately three more weeks.
The Impact of Anisotropic Error Correlation Modelling
Most data assimilation systems assume isotropic forecast error correlations, but results from two dimensional Kalman Filter experiments indicate that the correlations can be far from isotropic. In this paper we use a simple two-dimensional data assimilation system, which analyses trace chemical species such as ozone, to assess different approaches to modelling the error correlations. We compare assimilation results using isotropic correlations with results obtained using different approaches to modelling anisotropic correlations: first, using correlations based on the concentrations of the trace chemicals, and secondly using an advective correlation model. We show that these relatively cheap ways of modelling anisotropic correlations give objectively better results than using isotropic correlations. We discuss the possible extension of these approaches to a full 3-D meteorological data assimilation system.
On-line estimation of error covariance parameters for atmospheric data assimilation
A simple scheme is presented for on-line estimation of covariance parameters in statistical data assimilation systems. The scheme is based on a maximum-likelihood approach in which estimates are produced on the basis of a single batch of simultaneous observations. Simple-sample covariance estimation is reasonable as long as the number of available observations exceeds the number of tunable parameters by two or three orders of magnitude. Not much is known at present about model error associated with actual forecast systems. Our scheme can be used to estimate some important statistical model error parameters such as regionally averaged variances or characteristic correlation length scales. The advantage of the single-sample approach is that it does not rely on any assumptions about the temporal behavior of the covariance parameters: time-dependent parameter estimates can be continuously adjusted on the basis of current observations. This is of practical importance since it is likely to be the case that both model error and observation error strongly depend on the actual state of the atmosphere. The single-sample estimation scheme can be incorporated into any four-dimensional statistical data assimilation system that involves explicit calculation of forecast error covariances, including optimal interpolation (OI) and the simplified Kalman filter (SKF). The computational cost of the scheme is high but not prohibitive; on-line estimation of one or two covariance parameters in each analysis box of an operational bozed-OI system is currently feasible. A number of numerical experiments performed with an adaptive SKF and an adaptive version of OI, using a linear two-dimensional shallow-water model and artificially generated model error are described. The performance of the nonadaptive versions of these methods turns out to depend rather strongly on correct specification of model error parameters. These parameters are estimated under a variety of conditions, including uniformly distributed model error and time-dependent model error statistics.
An error covariance model for sea surface topography and velocity derived from TOPEX/POSEIDON altimetry
In order to facilitate the use of satellite-derived sea surface topography and velocity oceanographic models, methodology is presented for deriving the total error covariance and its geographic distribution from TOPEX/POSEIDON measurements. The model is formulated using a parametric model fit to the altimeter range observations. The topography and velocity modeled with spherical harmonic expansions whose coefficients are found through optimal adjustment to the altimeter range residuals using Bayesian statistics. All other parameters, including the orbit, geoid, surface models, and range corrections are provided as unadjusted parameters. The maximum likelihood estimates and errors are derived from the probability density function of the altimeter range residuals conditioned with a priori information. Estimates of model errors for the unadjusted parameters are obtained from the TOPEX/POSEIDON postlaunch verification results and the error covariances for the orbit and the geoid, except for the ocean tides. The error in the ocean tides is modeled, first, as the difference between two global tide models and, second, as the correction to the present tide model, the correction derived from the TOPEX/POSEIDON data. A formal error covariance propagation scheme is used to derive the total error. Our global total error estimate for the TOPEX/POSEIDON topography relative to the geoid for one 10-day period is found tio be 11 cm RMS. When the error in the geoid is removed, thereby providing an estimate of the time dependent error, the uncertainty in the topography is 3.5 cm root mean square (RMS). This level of accuracy is consistent with direct comparisons of TOPEX/POSEIDON altimeter heights with tide gauge measurements at 28 stations. In addition, the error correlation length scales are derived globally in both east-west and north-south directions, which should prove useful for data assimilation. The largest error correlation length scales are found in the tropics. Errors in the velocity field are smallest in midlatitude regions. For both variables the largest errors caused by uncertainty in the geoid. More accurate representations of the geoid await a dedicated geopotential satellite mission. Substantial improvements in the accuracy of ocean tide models are expected in the very near future from research with TOPEX/POSEIDON data.
Improved Understanding of Coupled Water and Carbon Cycle Processes through Machine Learning Approaches
Focal Area(s): This white paper addresses how explainable machine learning (ML) algorithms can improve insights gained from complex data (Focal Area 3). We will also address how ML approaches related to sensor compression and low energy AI hardware can be used for efficient data acquisition (Focal Area 1). Science Challenge: We will focus on the coupled water cycle and carbon cycle processes in terrestrial ecosystems and terrestrial-aquatic interfaces. Thus, our approaches will be centered around several data-model integration challenges indicated in EESD strategic plans for two science focus areas: Terrestrial Ecosystem Science and Hydrobiogeochemistry. We can use AI to correct systematic model errors due to biases in either data or model structure/processes. Error patterns in model predictions usually vary by region, model, season etc. But there are systematic patterns in them. e.g., soil moisture tends to be overestimated in the arid western continental United States underestimated in wetter eastern USA; some land surface models tend to underestimate moisture in wet seasons and overestimate in dry seasons. Moreover, the consequences of extreme events (e.g., drought, extreme flooding, storm surges associated with tropical storms, hurricanes) on carbon cycle processes are not well represented in ecosystem and Earth system models. Redox-sensitive processes (e.g., rapid oxidation/reduction of iron and other redox-sensitive elements in soil microsites subjected to fluctuated hydrology) in terrestrial-aquatic interfaces further challenge model predictions of hot-spots (and hot-moments) due to poor understanding of underlying mechanisms. Thus, AI/ML approaches can be used to learn patterns in the data and model errors and use them to build model equations and correct process-based model errors.
Modeling, Analysis, and Control of Demand Response Resources
While the traditional goal of an electric power system has been to control supply to fulfill demand, the demand-side can plan an active role in power systems via Demand Response (DR), defined by the Department of Energy (DOE) as “a tariff or program established to motivate changes in electric use by end-use customers in response to changes in the price of electricity over time, or to give incentive payments designed to induce lower electricity use at times of high market prices or when grid reliability is jeopardized” [29]. DR can provide a variety of benefits including reducing peak electric loads when the power system is stressed and fast timescale energy balancing. Therefore, DR can improve grid reliability and reduce wholesale energy prices and their volatility. This dissertation focuses on analyzing both recent and emerging DR paradigms. Recent DR programs have focused on peak load reduction in commercial buildings and industrial facilities (C&I facilities). We present methods for using 15-minute-interval electric load data, commonly available from C&I facilities, to help building managers understand building energy consumption and ‘ask the right questions’ to discover opportunities for DR. Additionally, we present a regression-based model of whole building electric load, i.e., a baseline model, which allows us to quantify DR performance. We use this baseline model to understand the performance of 38 C&I facilities participating in an automated dynamic pricing DR program in California. In this program, facilities are expected to exhibit the same response each DR event. We find that baseline model error makes it difficult to precisely quantify changes in electricity consumption and understand if C&I facilities exhibit event-to-event variability in their response to DR signals. Therefore, we present a method to compute baseline model error and a metric to determine how much observed DR variability results from baseline model error rather than real variability in response. We find that, in general, baseline model error is large. Though some facilities exhibit real DR variability, most observed variability results from baseline model error. In some cases, however, aggregations of C&I facilities exhibit real DR variability, which could create challenges for power system operation. These results have implications for DR program design and deployment. Emerging DR paradigms focus on faster timescale DR. Here, we investigate methods to coordinate aggregations of residential thermostatically controlled loads (TCLs), including air conditioners and refrigerators, to manage frequency and energy imbalances in power systems. We focus on opportunities to centrally control loads with high accuracy but low requirements for sensing and communications infrastructure. Specifically, we compare cases when measured load state information (e.g., power consumption and temperature) is 1) available in real time; 2) available, but not in real time; and 3) not available. We develop Markov Chain models to describe the temperature state evolution of heterogeneous populations of TCLs, and use Kalman filtering for both state and joint parameter/state estimation. We present a look-ahead proportional controller to broadcast control signals to all TCLs, which always remain in their temperature dead-band. Simulations indicate that it is possible to achieve power tracking RMS errors in the range of 0.26–9.3% of steady state aggregated power consumption. Results depend upon the information available for system identification, state estimation, and control. We find that, depending upon the performance required, TCLs may not need to provide state information to the central controller in real time or at all. We also estimate the size of the TCL potential resource; potential revenue from participation in markets; and break-even costs associated with deploying DR-enabling technologies. We find that current TCL energy storage capacity in California is 8–11 GWh, with refrigerators contributing the most. Annual revenues from participation in regulation vary from $10 to $220 per TCL per year depending upon the type of TCL and climate zone, while load following and arbitrage revenues are more modest at $2 to $35 per TCL per year. These results lead to a number of policy recommendations that will make it easier to engage residential loads in fast timescale DR.
Estimating Model Prediction Error: Should You Treat Predictions as Fixed or Random?
Crop models are important tools for impact assessment of climate change, as well as for exploring management options under current climate. It is essential to evaluate the uncertainty associated with predictions of these models. We compare two criteria of prediction error; MSEP fixed, which evaluates mean squared error of prediction for a model with fixed structure, parameters and inputs, and MSEP uncertain( X), which evaluates mean squared error averaged over the distributions of model structure, inputs and parameters. Comparison of model outputs with data can be used to estimate the former. The latter has a squared bias term, which can be estimated using hindcasts, and a model variance term, which can be estimated from a simulation experiment. The separate contributions to MSEP uncertain (X) can be estimated using a random effects ANOVA. It is argued that MSEP uncertain (X) is the more informative uncertainty criterion, because it is specific to each prediction situation.
Horizon sensor errors calculated by computer models compared with errors measured in orbit
Using a computer program to model the earth's horizon and to duplicate the signal processing procedure employed by the ESA (Earth Sensor Assembly), errors due to radiance variation have been computed for a particular time of the year. Errors actually occurring in flight at the same time of year are inferred from integrated rate gyro data for a satellite of the TIROS series of NASA weather satellites (NOAA-A). The predicted performance is compared with actual flight history.
Design of robust failure detection filters
An essential aspect of the design of control systems for large, flexible spacecraft is fault tolerance. Because it is anticipated that a large number of sensors and actuators will be required to realize good control over these assemblies, the detection and isolation of component failures cannot be based on direct comparisons among replicated components. Instead, the notion of 'analytic redundancy' must be employed for the FDI function. Unfortunately this makes the FDI function sensitive to modeling errors which are certain to exist in the large space structure problem due to model truncation and parameter uncertainty. This paper addresses the robustness to model error of one method of FDI residual generation - the failure detection filter. Initial designs were found to be extremely sensitive to modeling error. The sources of this sensitivity are analyzed and modifications to the design are suggested. The improved filter is shown to have much better visibility of the failure signatures relative to the background due to modeling error.
The Importance of HRA in Human Space Flight: Understanding the Risks
Human performance is critical to crew safety during space missions. Humans interact with hardware and software during ground processing, normal flight, and in response to events. Human interactions with hardware and software can cause Loss of Crew and/or Vehicle (LOCV) through improper actions, or may prevent LOCV through recovery and control actions. Humans have the ability to deal with complex situations and system interactions beyond the capability of machines. Human Reliability Analysis (HRA) is a method used to qualitatively and quantitatively assess the occurrence of human failures that affect availability and reliability of complex systems. Modeling human actions with their corresponding failure probabilities in a Probabilistic Risk Assessment (PRA) provides a more complete picture of system risks and risk contributions. A high-quality HRA can provide valuable information on potential areas for improvement, including training, procedures, human interfaces design, and the need for automation. Modeling human error has always been a challenge in part because performance data is not always readily available. For spaceflight, the challenge is amplified not only because of the small number of participants and limited amount of performance data available, but also due to the lack of definition of the unique factors influencing human performance in space. These factors, called performance shaping factors in HRA terminology, are used in HRA techniques to modify basic human error probabilities in order to capture the context of an analyzed task. Many of the human error modeling techniques were developed within the context of nuclear power plants and therefore the methodologies do not address spaceflight factors such as the effects of microgravity and longer duration missions. This presentation will describe the types of human error risks which have shown up as risk drivers in the Shuttle PRA which may be applicable to commercial space flight. As with other large PRAs of complex machines, human error in the Shuttle PRA proved to be an important contributor (~12 percent) to LOCV. An existing HRA technique was adapted for use in the Shuttle PRA, but additional guidance and improvements are needed to make the HRA task in space-related PRAs easier and more accurate. Therefore, this presentation will also outline plans for expanding current HRA methodology to more explicitly cover spaceflight performance shaping factors.
Evaluation of semiempirical atmospheric density models for orbit determination applications
This paper presents the results of an investigation of the orbit determination performance of the Jacchia-Roberts (JR), mass spectrometer incoherent scatter 1986 (MSIS-86), and drag temperature model (DTM) atmospheric density models. Evaluation of the models was performed to assess the modeling of the total atmospheric density. This study was made generic by using six spacecraft and selecting time periods of study representative of all portions of the 11-year cycle. Performance of the models was measured for multiple spacecraft, representing a selection of orbit geometries from near-equatorial to polar inclinations and altitudes from 400 kilometers to 900 kilometers. The orbit geometries represent typical low earth-orbiting spacecraft supported by the Goddard Space Flight Center (GSFC) Flight Dynamics Division (FDD). The best available modeling and orbit determination techniques using the Goddard Trajectory Determination System (GTDS) were employed to minimize the effects of modeling errors. The latest geopotential model available during the analysis, the Goddard earth model-T3 (GEM-T3), was employed to minimize geopotential model error effects on the drag estimation. Improved-accuracy techniques identified for TOPEX/Poseidon orbit determination analysis were used to improve the Tracking and Data Relay Satellite System (TDRSS)-based orbit determination used for most of the spacecraft chosen for this analysis. This paper shows that during periods of relatively quiet solar flux and geomagnetic activity near the solar minimum, the choice of atmospheric density model used for orbit determination is relatively inconsequential. During typical solar flux conditions near the solar maximum, the differences between the JR, DTM, and MSIS-86 models begin to become apparent. Time periods of extreme solar activity, those in which the daily and 81-day mean solar flux are high and change rapidly, result in significant differences between the models. During periods of high geomagnetic activity, the standard JR model was outperformed by DTM. Modification of the JR model to use a geomagnetic heating delay of 3 hours, as used in DTM, instead of the 6.7-hour delay produced results comparable to or better than the DTM performance, reducing definitive orbit solution ephermeris overlap differences by 30 to 50 percent. The reduction in the overlap differences would be useful for mitigating the impact of geomagnetic storms on orbit prediction.
Effects of model deficiencies on parameter estimation
Reliable structural dynamic models will be required as a basis for deriving the reduced-order plant models used in control systems for large space structures. Ground vibration testing and model verification will play an important role in the development of these models; however, fundamental differences between the space environment and earth environment, as well as variations in structural properties due to as-built conditions, will make on-orbit identification essential. The efficiency, and perhaps even the success, of on-orbit identification will depend on having a valid model of the structure. It is envisioned that the identification process will primarily involve parametric methods. Given a correct model, a variety of estimation algorithms may be used to estimate parameter values. This paper explores the effects of modeling errors and model deficiencies on parameter estimation by reviewing previous case histories. The effects depend at least to some extent on the estimation algorithm being used. Bayesian estimation was used in the case histories presented here. It is therefore conceivable that the behavior of an estimation algorithm might be useful in detecting and possibly even diagnosing deficiencies. In practice, the task is complicated by the presence of systematic errors in experimental procedures and data processing and in the use of the estimation procedures themselves.