Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Characterization of Partially Observed Epidemics - Application to COVID-19

This report documents a statistical method for the "real-time" characterization of partially observed epidemics. Observations consist of daily counts of symptomatic patients, diagnosed with the disease. Characterization, in this context, refers to estimation of epidemiological parameters that can be used to provide short-term forecasts of the ongoing epidemic, as well as to provide gross information for the time-dependent infection rate. The characterization problem is formulated as a Bayesian inverse problem, and is predicated on a model for the distribution of the incubation period. The model parameters are estimated as distributions using a Markov Chain Monte Carlo (MCMC) method, thus quantifying the uncertainty in the estimates. The method is applied to the COVID-19 pandemic of 2020, using data at the country, provincial (e.g., states) and regional (e.g. county) levels. The epidemiological model includes a stochastic component due to uncertainties in the incubation period. This model-form uncertainty is accommodated by a pseudo-marginal Metropolis-Hastings MCMC sampler, which produces posterior distributions that reflect this uncertainty. We approximate the discrepancy between the data and the epidemiological model using Gaussian and negative binomial error models; the latter was motivated by the over-dispersed count data. For small daily counts we find the performance of the calibrated models to be similar for the two error models. For large daily counts the negative-binomial approximation is numerically unstable unlike the Gaussian error model. Application of the model at the country level (for the United States, Germany, Italy, etc.) generally provided accurate forecasts, as the data consisted of large counts which suppressed the day-to-day variations in the observations. Further, the bulk of the data is sourced over the duration before the relaxation of the curbs on population mixing, and is not confounded by any discernible country-wide second wave of infections. At the state-level, where reporting was poor or which evinced few infections (e.g., New Mexico), the variance in the data posed some, though not insurmountable, difficulties, and forecasts were able to capture the data with large uncertainty bounds. The method was found to be sufficiently sensitive to discern the flattening of the infection and epidemic curve due to shelter-in-place orders after around 90% quantile for the incubation distribution (about 10 days for COVID-19). The proposed model was also used at a regional level to compare the forecasts for the central and north-west regions of New Mexico. Modeling the data for these regions illustrated different disease spread dynamics captured by the model. While in the central region the daily counts peaked in the late April, in the north-west region the ramp-up continued for approximately three more weeks.

59 BASIC BIOLOGICAL SCIENCES↗

Improved Understanding of Coupled Water and Carbon Cycle Processes through Machine Learning Approaches

Focal Area(s): This white paper addresses how explainable machine learning (ML) algorithms can improve insights gained from complex data (Focal Area 3). We will also address how ML approaches related to sensor compression and low energy AI hardware can be used for efficient data acquisition (Focal Area 1). Science Challenge: We will focus on the coupled water cycle and carbon cycle processes in terrestrial ecosystems and terrestrial-aquatic interfaces. Thus, our approaches will be centered around several data-model integration challenges indicated in EESD strategic plans for two science focus areas: Terrestrial Ecosystem Science and Hydrobiogeochemistry. We can use AI to correct systematic model errors due to biases in either data or model structure/processes. Error patterns in model predictions usually vary by region, model, season etc. But there are systematic patterns in them. e.g., soil moisture tends to be overestimated in the arid western continental United States underestimated in wetter eastern USA; some land surface models tend to underestimate moisture in wet seasons and overestimate in dry seasons. Moreover, the consequences of extreme events (e.g., drought, extreme flooding, storm surges associated with tropical storms, hurricanes) on carbon cycle processes are not well represented in ecosystem and Earth system models. Redox-sensitive processes (e.g., rapid oxidation/reduction of iron and other redox-sensitive elements in soil microsites subjected to fluctuated hydrology) in terrestrial-aquatic interfaces further challenge model predictions of hot-spots (and hot-moments) due to poor understanding of underlying mechanisms. Thus, AI/ML approaches can be used to learn patterns in the data and model errors and use them to build model equations and correct process-based model errors.

54 ENVIRONMENTAL SCIENCES↗

Modeling, Analysis, and Control of Demand Response Resources

While the traditional goal of an electric power system has been to control supply to fulfill demand, the demand-side can plan an active role in power systems via Demand Response (DR), defined by the Department of Energy (DOE) as “a tariff or program established to motivate changes in electric use by end-use customers in response to changes in the price of electricity over time, or to give incentive payments designed to induce lower electricity use at times of high market prices or when grid reliability is jeopardized” [29]. DR can provide a variety of benefits including reducing peak electric loads when the power system is stressed and fast timescale energy balancing. Therefore, DR can improve grid reliability and reduce wholesale energy prices and their volatility. This dissertation focuses on analyzing both recent and emerging DR paradigms. Recent DR programs have focused on peak load reduction in commercial buildings and industrial facilities (C&I facilities). We present methods for using 15-minute-interval electric load data, commonly available from C&I facilities, to help building managers understand building energy consumption and ‘ask the right questions’ to discover opportunities for DR. Additionally, we present a regression-based model of whole building electric load, i.e., a baseline model, which allows us to quantify DR performance. We use this baseline model to understand the performance of 38 C&I facilities participating in an automated dynamic pricing DR program in California. In this program, facilities are expected to exhibit the same response each DR event. We find that baseline model error makes it difficult to precisely quantify changes in electricity consumption and understand if C&I facilities exhibit event-to-event variability in their response to DR signals. Therefore, we present a method to compute baseline model error and a metric to determine how much observed DR variability results from baseline model error rather than real variability in response. We find that, in general, baseline model error is large. Though some facilities exhibit real DR variability, most observed variability results from baseline model error. In some cases, however, aggregations of C&I facilities exhibit real DR variability, which could create challenges for power system operation. These results have implications for DR program design and deployment. Emerging DR paradigms focus on faster timescale DR. Here, we investigate methods to coordinate aggregations of residential thermostatically controlled loads (TCLs), including air conditioners and refrigerators, to manage frequency and energy imbalances in power systems. We focus on opportunities to centrally control loads with high accuracy but low requirements for sensing and communications infrastructure. Specifically, we compare cases when measured load state information (e.g., power consumption and temperature) is 1) available in real time; 2) available, but not in real time; and 3) not available. We develop Markov Chain models to describe the temperature state evolution of heterogeneous populations of TCLs, and use Kalman filtering for both state and joint parameter/state estimation. We present a look-ahead proportional controller to broadcast control signals to all TCLs, which always remain in their temperature dead-band. Simulations indicate that it is possible to achieve power tracking RMS errors in the range of 0.26–9.3% of steady state aggregated power consumption. Results depend upon the information available for system identification, state estimation, and control. We find that, depending upon the performance required, TCLs may not need to provide state information to the central controller in real time or at all. We also estimate the size of the TCL potential resource; potential revenue from participation in markets; and break-even costs associated with deploying DR-enabling technologies. We find that current TCL energy storage capacity in California is 8–11 GWh, with refrigerators contributing the most. Annual revenues from participation in regulation vary from $10 to $220 per TCL per year depending upon the type of TCL and climate zone, while load following and arbitrage revenues are more modest at $2 to $35 per TCL per year. These results lead to a number of policy recommendations that will make it easier to engage residential loads in fast timescale DR.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

POWER OF OBSERVATIONAL HUBBLE PARAMETER DATA: A FIGURE OF MERIT EXPLORATION

We use simulated Hubble parameter data in the redshift range 0 {<=} z {<=} 2 to explore the role and power of observational H(z) data in constraining cosmological parameters of the {Lambda}CDM model. The error model of the simulated data is empirically constructed from available measurements and scales linearly as z increases. By comparing the median figures of merit calculated from simulated data sets with that of current Type Ia supernova (SNIa) data, we find that as many as 64 further independent measurements of H(z) are needed to match the parameter constraining power of SNIa. If the error of H(z) could be lowered to 3%, the same number of future measurements would be needed, but then the redshift coverage would only be required to reach z = 1. We also show that accurate measurements of the Hubble constant H{sub 0} can be used as priors to increase the H(z) data's figure of merit.

79 ASTRONOMY AND ASTROPHYSICS↗

Ultra-deep mutant spectrum profiling: improving sequencing accuracy using overlapping read pairs

Backgound: High throughput sequencing is beginning to make a transformative impact in the area of viral evolution. Deep sequencing has the potential to reveal the mutant spectrum within a viral sample at high resolution, thus enabling the close examination of viral mutational dynamics both within- and between-hosts. The challenge however, is to accurately model the errors in the sequencing data and differentiate real viral mutations, particularly those that exist at low frequencies, from sequencing errors. Results: We demonstrate that overlapping read pairs (ORP) -- generated by combining short fragment sequencing libraries and longer sequencing reads -- significantly reduce sequencing error rates and improve rare variant detection accuracy. Using this sequencing protocol and an error model optimized for variant detection, we are able to capture a large number of genetic mutations present within a viral population at ultra-low frequency levels (<0.05%). Conclusions: Our rare variant detection strategies have important implications beyond viral evolution and can be applied to any basic and clinical research area that requires the identification of rare mutations.

59 BASIC BIOLOGICAL SCIENCES↗

PyOED: An Extensible Suite for Data Assimilation and Model-Constrained Optimal Design of Experiments

This article describes PyOED, a highly extensible scientific package that enables developing and testing model-constrained optimal experimental design (OED) for inverse problems. Specifically, PyOED aims to be a comprehensive Python toolkit for model-constrained OED. The package targets scientists and researchers interested in understanding the details of OED formulations and approaches. It is also meant to enable researchers to experiment with standard and innovative OED technologies with a wide range of test problems (e.g., simulation models). OED, inverse problems (e.g., Bayesian inversion), and data assimilation (DA) are closely related research fields, and their formulations overlap significantly. Thus, PyOED is continuously being expanded with a plethora of Bayesian inversion, DA, and OED methods as well as new scientific simulation models, observation error models, and observation operators. These pieces are added such that they can be permuted to enable testing OED methods in various settings of varying complexities. The PyOED core is completely written in Python and utilizes the inherent object-oriented capabilities; however, the current version of PyOED is meant to be extensible rather than scalable. Specifically, PyOED is developed to “enable rapid development and benchmarking of OED methods with minimal coding effort and to maximize code reutilization.” This article provides a brief description of the PyOED layout and philosophy and provides a set of exemplary test cases and tutorials to demonstrate the potential of the package.

97 MATHEMATICS AND COMPUTING↗

MultiPEM Toolbox: User Manual [Rev. 2]

This document explains use of the Multi-Phenomenology Explosion Monitoring (Multi PEM) Toolbox, a collection of R scripts for estimating the unknown device parameters of a new event with uncertainty quantification. The methodology and application used for illustration in this user manual are fully documented in a Los Alamos National Laboratory technical report hereafter designated “WPA” for reference. Additional details on the application are found in a recent journal article. Two assessment types are available: rapid and complete. Rapid assessments are conducted in two stages, as described in Section 2. In the first stage, calibration data are used to estimate forward and error model parameters (WPA, §5.1) and (if relevant) errors-in-variables yield values for calibration sources (WPA, §3, Equation (3)). In the second stage, new event data are used to estimate the unknown new event device parameters (WPA, §5.2) with uncertainty quantification. Two options for treating the inferred first stage parameters in second stage Bayesian analysis are available: fixing them at their maximum likelihood estimate (default), or multiple imputation. Multiple imputation involves utilizing several posterior samples (imputations) of the first stage parameters as fixed values in the second stage posterior sampling of the new event device parameters. Second stage sampling is conducted across imputations in parallel to improve computational efficiency. This method produces improved uncertainty quantification of the new event device parameters compared with the default treatment of the first stage parameters, at the expense of additional computation. Complete assessments are conducted in a single stage, as described in Section 3. Calibration and (if relevant) new event data are used simultaneously to estimate all forward model, error model, and (if relevant) new event device parameters with uncertainty quantification on the latter. As the name suggests, rapid assessments generally run substantially faster than complete assessments (even with multiple imputation), because the results of first stage analysis can be stored and incorporated into estimating a relatively low-dimensional space of new event device parameters whenever relevant new event data becomes available. On the other hand, complete assessments must be run on the full set of model and device parameters with calibration and new event data every time the latter becomes available.

97 MATHEMATICS AND COMPUTING↗

Model-form Error Correction using Universal Differential Equations for an Agent-Based Model of Infectious Disease

This report demonstrates universal differential equations (UDEs) as an approach to bridge the gap between ordinary differential equations (ODE) models and agent-based models (ABMs). Using UDE models as surrogates for ABMs allows us to preserve the foundational ODE that represents global disease dynamics while coupling it with a neural network model to approximate functions for the local behaviors of the ABM.

59 BASIC BIOLOGICAL SCIENCES↗

Accounting for erroneous model structures in biokinetic process models

In engineering practice, model-based design requires not only a good process-based model, but also a good description of stochastic disturbances and measurement errors to learn credible parameter values from observations. However, typical methods use Gaussian error models, which often cannot describe the complex temporal patterns of residuals. Consequently, this results in overconfidence in the identified parameters and, in turn, optimistic reactor designs. Here, we assess the strengths and weaknesses of a method to statistically describe these patterns with autocorrelated error models. This method produces increased widths of the credible prediction intervals following the inclusion of the bias term, in turn leading to more conservative design choices. However, we also show that the augmented error model is not a universal tool, as its application cannot guarantee the desired reliability of the resulting wastewater reactor design.

42 ENGINEERING↗

Direct numerical simulations in solid mechanics for quantifying the macroscale effects of microstructure and material model-form error

Two fundamental approximations in macroscale solid-mechanics modeling are (1) the assumption of scale separation in homogenization theory and (2) the use of a macroscopic plasticity material model that represents, in a mean sense, the multitude of inelastic processes occurring at the microscale. With the goal of quantifying the errors induced by these approximations on engineering quantities of interest, we perform a set of direct numerical simulations (DNS) in which polycrystalline microstructures are embedded throughout a macroscale structure. The largest simulations model over 50,000 grains. The microstructure is idealized using a randomly close-packed Voronoi tessellation in which each polyhedral Voronoi cell represents a grain. An face centered cubic crystal-plasticity model is used to model the mechanical response of each grain. The overall grain structure is equiaxed, and each grain is randomly oriented with no overall texture. The detailed results from the DNS simulations are compared to results obtained from conventional macroscale simulations that use homogeneous isotropic plasticity models. The macroscale plasticity models are calibrated using a representative volume element of the idealized microstructure. Furthermore, we envision that DNS modeling will be used to gain new insights into the mechanics of material deformation and failure.

36 MATERIALS SCIENCE↗

Model uncertainty in accelerator application simulations

Monte-Carlo nuclear reaction and transport codes are widely used to devise accelerator-based nuclear physics experiments; at the same time, many experiments are performed to validate the Monte-Carlo codes, which can be used for the design of full-scale nuclear power applications or the design of new benchmark experiments. Dedicated model benchmark studies investigate a broad range of nuclear reactions and quantities. Examples of these include isotope formation or secondary particle fluxes that result from the interactions of GeV-range hadrons with monoisotopic targets, which can be used to assess the respective systematic uncertainty of models. Such benchmark studies, as well as many nuclear application experiments and simulations carried out by various groups over the last few decades, enable us to draw methodological lessons. In this work, model uncertainty determined based on available experimental data allow us to identify the effects of practitioner expertise as well as the design of codes (user access to micro-scale parameters) on the range of uncertainties. We found that in cases when simulations are performed by code developers or users that are very experienced in performing simulations, the model to experiment quantity ratios generally agree with the limits determined by dedicated benchmark studies. In other cases, the ratios generally tend to be either smaller (underestimation of model error) or larger (overestimation of model error). A plausible explanation of the aforementioned effects is suggested.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Interpolation Models and Error Bounds for Verifiable Scientific Machine Learning

This repository contains python scripts and numerical data accompanying the paper: "Leveraging Interpolation Models and Error Bounds for Verifiable Scientific Machine Learning," Tyler Chang, Andrew Gillette, Romit Maulik, 2024. The following subdirectories are included: - "interpolants" contains our interpolation scripts used for all studies - "experiments" contains scripts demonstrating our experiments with synthetic data - "airfoil" contains scripts demonstrating our experiments with the publicly available UIUC airfoil dataset. Further instructions are provided in READMEs within the sub-directories.

Gillette, Andrew↗

Verification and source-position error analysis of film reconstruction techniques used in the brachytherapy planning systems

A method was presented that employs standard linac QA tools to verify the accuracy of film reconstruction algorithms used in the brachytherapy planning system. Verification of reconstruction techniques is important as suggested in the ESTRO booklet 8: ''The institution should verify the full process of any reconstruction technique employed clinically.'' Error modeling was also performed to analyze seed-position errors. The ''isocentric beam checker'' device was used in this work. It has a two-dimensional array of steel balls embedded on its surface. The checker was placed on the simulator couch with its center ball coincident with the simulator isocenter, and one axis of its cross marks parallel to the axis of gantry rotation. The gantry of the simulator was rotated to make the checker behave like a three-dimensional array of balls. Three algorithms used in the ABACUS treatment planning system: orthogonal film, 2-films-with-variable-angle, and 3-films-with-variable-angle were tested. After exposing and digitizing the films, the position of each steel ball on the checker was reconstructed and compared to its true position, which can be accurately calculated. The results showed that the error is dependent on the object-isocenter distance, but not the magnification of the object. The averaged errors were less than 1 mm within the tolerance level defined by Roueet al. [''The EQUAL-ESTRO audit on geometric reconstruction techniques in brachytherapy,'' Radiother. Oncol. 78, 78-83 (2006)]. However, according to the error modeling, the theoretical error would be greater than 2 mm if the objects were located more than 20 cm away from the isocenter with a 0.5 deg. reading error of the gantry and collimator angles. Thus, in addition to carefully performing the QA of the gantry and collimator angle indicators, it is suggested that the patient, together with the applicators or seeds inside, should be placed close to the isocenter as much as possible. This method could be used to test the reconstruction techniques of any planning system, and the most suitable one can be chosen for clinical use.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Variability in Automated Responses of Commercial Buildings and Industrial Facilities to Dynamic Electricity Prices

Changes in the electricity consumption of commercial buildings and industrial facilities (C&I facilities) during Demand Response (DR) events are usually estimated using counterfactual baseline models. Model error makes it difficult to precisely quantify these changes in consumption and understand if C&I facilities exhibit event-to-event variability in their response to DR signals. This paper seeks to understand baseline model error and DR variability in C&I facilities facing dynamic electricity prices. Using a regression-based baseline model, we present a method to compute the error associated with estimates of several DR parameters. We also develop a metric to determine how much observed DR variability results from baseline model error rather than real variability in response. We analyze 38 C&I facilities participating in an automated DR program and find that DR parameter errors are large. Though some facilities exhibit real DR variability, most observed variability results from baseline model error. Therefore, facilities with variable DR parameters may actually respond consistently from event to event. Consequently, in DR programs in which repeatability is valued, individual buildings may be performing better than previously thought. In some cases, however, aggregations of C&I facilities exhibit real DR variability, which could create challenges for power system operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NuSTAR and Suzaku X-ray spectroscopy of NGC 4151: Evidence for reflection from the inner accretion disk

We present X-ray timing and spectral analyses of simultaneous 150 ks Nuclear Spectroscopic Telescope Array (NuSTAR) and Suzaku X-ray observations of the Seyfert 1.5 galaxy NGC 4151. We disentangle the continuum emission, absorption, and reflection properties of the active galactic nucleus (AGN) by applying inner accretion disk reflection and absorption-dominated models. With a time-averaged spectral analysis, we find strong evidence for relativistic reflection from the inner accretion disk. We find that relativistic emission arises from a highly ionized inner accretion disk with a steep emissivity profile, which suggests an intense, compact illuminating source. We find a preliminary, near-maximal black hole spin a>0.9 accounting for statistical and systematic modeling errors. We find a relatively moderate reflection fraction with respect to predictions for the lamp post geometry, in which the illuminating corona is modeled as a point source. Through a time-resolved spectral analysis, we find that modest coronal and inner disk reflection (IDR) flux variation drives the spectral variability during the observations. We discuss various physical scenarios for the IDR model and we find that a compact corona is consistent with the observed features.

79 ASTRONOMY AND ASTROPHYSICS↗

Bayesian calibration of terrestrial ecosystem models: a study of advanced Markov chain Monte Carlo methods

Calibration of terrestrial ecosystem models is important but challenging. Bayesian inference implemented by Markov chain Monte Carlo (MCMC) sampling provides a comprehensive framework to estimate model parameters and associated uncertainties using their posterior distributions. The effectiveness and efficiency of the method strongly depend on the MCMC algorithm used. In this work, a differential evolution adaptive Metropolis (DREAM) algorithm is used to estimate posterior distributions of 21 parameters for the data assimilation linked ecosystem carbon (DALEC) model using 14 years of daily net ecosystem exchange data collected at the Harvard Forest Environmental Measurement Site eddy-flux tower. The calibration of DREAM results in a better model fit and predictive performance compared to the popular adaptive Metropolis (AM) scheme. Moreover, DREAM indicates that two parameters controlling autumn phenology have multiple modes in their posterior distributions while AM only identifies one mode. The application suggests that DREAM is very suitable to calibrate complex terrestrial ecosystem models, where the uncertain parameter size is usually large and existence of local optima is always a concern. In addition, this effort justifies the assumptions of the error model used in Bayesian calibration according to the residual analysis. Here, the result indicates that a heteroscedastic, correlated, Gaussian error model is appropriate for the problem, and the consequent constructed likelihood function can alleviate the underestimation of parameter uncertainty that is usually caused by using uncorrelated error models.

54 ENVIRONMENTAL SCIENCES↗

Quantifying modeling uncertainty in simplified beam models for building response prediction

The use of simple models for response prediction of building structures is preferred in earthquake engineering for risk evaluations at regional scales, as they make computational studies more feasible. The primary impediment in their gainful use presently is the lack of viable methods for quantifying (and reducing upon) the modeling errors/uncertainties they bear. This study presents a Bayesian calibration method wherein the modeling error is embedded into the parameters of the model. Here, the method is specifically described for coupled shear-flexural beam models here, but it can be applied to any parametric surrogate model. The major benefit the method offers is the ability to consider the modeling uncertainty in the forward prediction of any degree-of-freedom or composite response regardless of the data used in calibration. The method is extensively verified using two synthetic examples. In the first example, the beam model is calibrated to represent a similar beam model but with enforced modeling errors. In the second example, the beam model is used to represent the detailed finite element model of a 52-story building. Both examples show the capability of the proposed solution to provide realistic uncertainty estimation around the mean prediction.

47 OTHER INSTRUMENTATION↗

Successive Procedure for Solution Verification Based on User Needs

This paper discusses a revised solution verification procedure for computational fluid dynamics simulations to estimate the uncertainties in the quantities of interest based on discretization error models. This proposed procedure builds upon current procedures described in ASME V&V 20 but provides more guidance in determining the necessary number of mesh levels to build reliable discretization error models. Such guidance is particularly useful for practicing engineers without prior experience in solution verification. The key features of this proposed solution verification procedure are the ability to determine the need for additional mesh levels iteratively and the seamless treatment for underdetermined, exact, and overdetermined solutions of the power series approximation to the discretization error models. This study applies the proposed procedure to a set of synthetic examples to demonstrate the revised procedure’s clarity in determining the number of mesh solutions required for a reliable estimate of the discretization error in computational fluid dynamics settings. Additionally, this proposed procedure prevents a potential pathway in the current procedure in ASME V&V 20 that may lead to unreasonably small discretization errors.

Weinmeister, Justin↗