Search NASA⌕ Search

SEARCH · Search NASA

Results for “error modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Interpolation Models and Error Bounds for Verifiable Scientific Machine Learning

This repository contains python scripts and numerical data accompanying the paper: "Leveraging Interpolation Models and Error Bounds for Verifiable Scientific Machine Learning," Tyler Chang, Andrew Gillette, Romit Maulik, 2024. The following subdirectories are included: - "interpolants" contains our interpolation scripts used for all studies - "experiments" contains scripts demonstrating our experiments with synthetic data - "airfoil" contains scripts demonstrating our experiments with the publicly available UIUC airfoil dataset. Further instructions are provided in READMEs within the sub-directories.

Gillette, Andrew↗
DOE OSTI2011

Variability in Automated Responses of Commercial Buildings and Industrial Facilities to Dynamic Electricity Prices

Changes in the electricity consumption of commercial buildings and industrial facilities (C&I facilities) during Demand Response (DR) events are usually estimated using counterfactual baseline models. Model error makes it difficult to precisely quantify these changes in consumption and understand if C&I facilities exhibit event-to-event variability in their response to DR signals. This paper seeks to understand baseline model error and DR variability in C&I facilities facing dynamic electricity prices. Using a regression-based baseline model, we present a method to compute the error associated with estimates of several DR parameters. We also develop a metric to determine how much observed DR variability results from baseline model error rather than real variability in response. We analyze 38 C&I facilities participating in an automated DR program and find that DR parameter errors are large. Though some facilities exhibit real DR variability, most observed variability results from baseline model error. Therefore, facilities with variable DR parameters may actually respond consistently from event to event. Consequently, in DR programs in which repeatability is valued, individual buildings may be performing better than previously thought. In some cases, however, aggregations of C&I facilities exhibit real DR variability, which could create challenges for power system operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2015

NuSTAR and Suzaku X-ray spectroscopy of NGC 4151: Evidence for reflection from the inner accretion disk

We present X-ray timing and spectral analyses of simultaneous 150 ks Nuclear Spectroscopic Telescope Array (NuSTAR) and Suzaku X-ray observations of the Seyfert 1.5 galaxy NGC 4151. We disentangle the continuum emission, absorption, and reflection properties of the active galactic nucleus (AGN) by applying inner accretion disk reflection and absorption-dominated models. With a time-averaged spectral analysis, we find strong evidence for relativistic reflection from the inner accretion disk. We find that relativistic emission arises from a highly ionized inner accretion disk with a steep emissivity profile, which suggests an intense, compact illuminating source. We find a preliminary, near-maximal black hole spin a>0.9 accounting for statistical and systematic modeling errors. We find a relatively moderate reflection fraction with respect to predictions for the lamp post geometry, in which the illuminating corona is modeled as a point source. Through a time-resolved spectral analysis, we find that modest coronal and inner disk reflection (IDR) flux variation drives the spectral variability during the observations. We discuss various physical scenarios for the IDR model and we find that a compact corona is consistent with the observed features.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Bayesian calibration of terrestrial ecosystem models: a study of advanced Markov chain Monte Carlo methods

Calibration of terrestrial ecosystem models is important but challenging. Bayesian inference implemented by Markov chain Monte Carlo (MCMC) sampling provides a comprehensive framework to estimate model parameters and associated uncertainties using their posterior distributions. The effectiveness and efficiency of the method strongly depend on the MCMC algorithm used. In this work, a differential evolution adaptive Metropolis (DREAM) algorithm is used to estimate posterior distributions of 21 parameters for the data assimilation linked ecosystem carbon (DALEC) model using 14 years of daily net ecosystem exchange data collected at the Harvard Forest Environmental Measurement Site eddy-flux tower. The calibration of DREAM results in a better model fit and predictive performance compared to the popular adaptive Metropolis (AM) scheme. Moreover, DREAM indicates that two parameters controlling autumn phenology have multiple modes in their posterior distributions while AM only identifies one mode. The application suggests that DREAM is very suitable to calibrate complex terrestrial ecosystem models, where the uncertain parameter size is usually large and existence of local optima is always a concern. In addition, this effort justifies the assumptions of the error model used in Bayesian calibration according to the residual analysis. Here, the result indicates that a heteroscedastic, correlated, Gaussian error model is appropriate for the problem, and the consequent constructed likelihood function can alleviate the underestimation of parameter uncertainty that is usually caused by using uncorrelated error models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Quantifying modeling uncertainty in simplified beam models for building response prediction

The use of simple models for response prediction of building structures is preferred in earthquake engineering for risk evaluations at regional scales, as they make computational studies more feasible. The primary impediment in their gainful use presently is the lack of viable methods for quantifying (and reducing upon) the modeling errors/uncertainties they bear. This study presents a Bayesian calibration method wherein the modeling error is embedded into the parameters of the model. Here, the method is specifically described for coupled shear-flexural beam models here, but it can be applied to any parametric surrogate model. The major benefit the method offers is the ability to consider the modeling uncertainty in the forward prediction of any degree-of-freedom or composite response regardless of the data used in calibration. The method is extensively verified using two synthetic examples. In the first example, the beam model is calibrated to represent a similar beam model but with enforced modeling errors. In the second example, the beam model is used to represent the detailed finite element model of a 52-story building. Both examples show the capability of the proposed solution to provide realistic uncertainty estimation around the mean prediction.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Successive Procedure for Solution Verification Based on User Needs

This paper discusses a revised solution verification procedure for computational fluid dynamics simulations to estimate the uncertainties in the quantities of interest based on discretization error models. This proposed procedure builds upon current procedures described in ASME V&V 20 but provides more guidance in determining the necessary number of mesh levels to build reliable discretization error models. Such guidance is particularly useful for practicing engineers without prior experience in solution verification. The key features of this proposed solution verification procedure are the ability to determine the need for additional mesh levels iteratively and the seamless treatment for underdetermined, exact, and overdetermined solutions of the power series approximation to the discretization error models. This study applies the proposed procedure to a set of synthetic examples to demonstrate the revised procedure’s clarity in determining the number of mesh solutions required for a reliable estimate of the discretization error in computational fluid dynamics settings. Additionally, this proposed procedure prevents a potential pathway in the current procedure in ASME V&V 20 that may lead to unreasonably small discretization errors.

Weinmeister, Justin↗
DOE OSTI2013

A Comparison of Forecast Error Generators for Modeling Wind and Load Uncertainty

This paper presents four algorithms to generate random forecast error time series, including a truncated-normal distribution model, a state-space based Markov model, a seasonal autoregressive moving average (ARMA) model, and a stochastic-optimization based model. The error time series are used to create real-time (RT), hour-ahead (HA), and day-ahead (DA) wind and load forecast time series that statistically match historically observed forecasting data sets, used for variable generation integration studies. A comparison is made using historical DA load forecast and actual load values to generate new sets of DA forecasts with similar stoical forecast error characteristics. This paper discusses and compares the capabilities of each algorithm to preserve the characteristics of the historical forecast data sets.

Lu, Ning↗
DOE OSTI2016

Moving beyond the total sea ice extent in gauging model biases

Here, reproducing characteristics of observed sea ice extent remains an important climate modeling challenge. This study describes several approaches to improve how model biases in total sea ice distribution are quantified, and applies them to historically forced simulations contributed to phase 5 of the Coupled Model Intercomparison Project (CMIP5). The quantity of hemispheric total sea ice area, or some measure of its equatorward extent, is often used to evaluate model performance. A new approach is introduced that investigates additional details about the structure of model errors, with an aim to reduce the potential impact of compensating errors when gauging differences between simulated and observed sea ice. Using multiple observational datasets, several new methods are applied to evaluate the climatological spatial distribution and the annual cycle of sea ice cover in 41 CMIP5 models. It is shown that in some models, error compensation can be substantial, for example resulting from too much sea ice in one region and too little in another. Error compensation tends to be larger in models that agree more closely with the observed total sea ice area, which may result from model tuning. The results herein suggest that consideration of only the total hemispheric sea ice area or extent can be misleading when quantitatively comparing how well models agree with observations. Further work is needed to fully develop robust methods to holistically evaluate the ability of models to capture the finescale structure of sea ice characteristics; however, the “sector scale” metric used here aids in reducing the impact of compensating errors in hemispheric integrals.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2015

Uncertainty quantification for nuclear density functional theory and information content of new measurements

Statistical tools of uncertainty quantification can be used to assess the information content of measured observables with respect to present-day theoretical models, to estimate model errors and thereby improve predictive capability, to extrapolate beyond the regions reached by experiment, and to provide meaningful input to applications and planned measurements. To showcase new opportunities offered by such tools, we make a rigorous analysis of theoretical statistical uncertainties in nuclear density functional theory using Bayesian inference methods. By considering the recent mass measurements from the Canadian Penning Trap at Argonne National Laboratory, we demonstrate how the Bayesian analysis and a direct least-squares optimization, combined with high-performance computing, can be used to assess the information content of the new data with respect to a model based on the Skyrme energy density functional approach. Employing the posterior probability distribution computed with a Gaussian process emulator, we apply the Bayesian framework to propagate theoretical statistical uncertainties in predictions of nuclear masses, two-neutron dripline, and fission barriers. Overall, we find that the new mass measurements do not impose a constraint that is strong enough to lead to significant changes in the model parameters. In addition, the example discussed in this study sets the stage for quantifying and maximizing the impact of new measurements with respect to current modeling and guiding future experimental efforts, thus enhancing the experiment-theory cycle in the scientific method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Quantification of modeling uncertainty in the Rayleigh damping model

Understanding and accurately characterizing energy dissipation mechanisms in civil structures during earthquakes is an important element of seismic assessment and design. The most commonly used model is attributed to Rayleigh. This paper proposes a systematic approach to quantify the uncertainty associated with Rayleigh's damping model. Bayesian calibration with embedded model error is employed to treat the coefficients of the Rayleigh model as random variables using modal damping ratios. Through a numerical example, we illustrate how this approach works and how the calibrated model can address modeling uncertainty associated with the Rayleigh damping model.

42 ENGINEERING↗
DOE OSTI2015

Bayesian Calibration of the Community Land Model using Surrogates

We present results from the Bayesian calibration of hydrological parameters of the Community Land Model (CLM), which is often used in climate simulations and Earth system models. A statistical inverse problem is formulated for three hydrological parameters, conditioned on observations of latent heat surface fluxes over 48 months. Our calibration method uses polynomial and Gaussian process surrogates of the CLM, and solves the parameter estimation problem using a Markov chain Monte Carlo sampler. Posterior probability densities for the parameters are developed for two sites with different soil and vegetation covers. Our method also allows us to examine the structural error in CLM under two error models. We find that accurate surrogate models can be created for CLM in most cases. The posterior distributions lead to better prediction than the default parameter values in CLM. Climatologically averaging the observations does not modify the parameters’ distributions significantly. The structural error model reveals a correlation time-scale which can potentially be used to identify physical processes that could be contributing to it. While the calibrated CLM has a higher predictive skill, the calibration is under-dispersive.

Bayesian calibration↗
DOE OSTI2016

AMiBA: cluster Sunyaev-Zel'dovich effect observations with the expanded 13-element array

The Yuan-Tseh Lee Array for Microwave Background Anisotropy (AMiBA) is a co-planar interferometer array operating at a wavelength of 3 mm to measure the Sunyaev–Zel’dovich effect (SZE) of galaxy clusters at arcminute scales. The first phase of operation—with a compact 7-element array with 0.6 m antennas (AMiBA-7)—observed six clusters at angular scales from 5{sup ′} to 23{sup ′}. Here, we describe the expansion of AMiBA to a 13-element array with 1.2 m antennas (AMiBA-13), its subsequent commissioning, and cluster SZE observing program. The most noticeable changes compared to AMiBA-7 are (1) array re-configuration with baselines ranging from 1.4 m to 4.8 m, allowing us to sample structures between 2{sup ′} and 10{sup ′}, (2) 13 new lightweight carbon-fiber-reinforced plastic (CFRP) 1.2 m reflectors, and (3) additional correlators and six new receivers. Since the reflectors are co-mounted on and distributed over the entire six-meter CFRP platform, a refined hexapod pointing error model and phase error correction scheme have been developed for AMiBA-13. These effects—entirely negligible for the earlier central close-packed AMiBA-7 configuration—can lead to additional geometrical delays during observations. Our correction scheme recovers at least 80 ± 5% of the point-source fluxes. We, therefore, apply an upward correcting factor of 1.25 to our visibilities to correct for phase decoherence, and a ±5% systematic uncertainty is added in quadrature with our statistical errors. We demonstrate the absence of further systematics with a noise level consistent with zero in stacked uv-visibilities. From the AMiBA-13 SZE observing program, we present here maps of a subset of 12 clusters with signal-to-noise ratios above five. We demonstrate combining AMiBA-7 with AMiBA-13 observations on Abell 1689, by jointly fitting their data to a generalized Navarro–Frenk–White model. Our cylindrically integrated Compton-y values for five radii are consistent with results from the Berkeley-Illinois-Maryland Array, the Owens Valley Radio Observatory, the Sunyaev–Zel’dovich Array, and the Planck Observatory. We also report the first targeted SZE detection toward the optically selected cluster RCS J1447+0828, and we demonstrate the ability of AMiBA SZE data to serve as a proxy for the total cluster mass. Finally, we show that our AMiBA-SZE derived cluster masses are consistent with recent lensing mass measurements in the literature.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

A novel network-based approach to determining measurement representation error for model evaluation of aerosol microphysical properties

Atmospheric aerosol size and abundance influence radiative effects and climate change. To date, efforts to constrain global climate models’ radiative forcing with in situ aerosol observations have been hamstrung by uncertainty. One source of error, the regional “representation error,” arises when accurate but sparse single-point measurements of atmospheric aerosol distributions are compared with a model value, assuming that the single-point measurement is representative of the model domain. The Portable Optical Particle Spectrometer network in the Southern Great Plains (POPSnet-SGP) campaign has demonstrated that a network of nearly autonomous aerosol instruments operating at ambient temperature and relative humidity (with low measurement error) may be used to quantify measurement representation error and investigate the factors introducing heterogeneity in aerosol distributions across a rural, continental background region. Measurements were made using Portable Optical Particle Spectrometer (POPS) instruments at several sites for five months across the Department of Energy’s Aerosol Radiation Measurement Southern Great Plains (ARM-SGP) User Facility in the central USA. Measurement representation error decreased with longer averaging periods (20-40 % between 1 sec and 1 day), varied between sites by 10 – 20 % for aerosol concentration 140 – 2500 nm in diameter (N_140), and was higher for aerosols > 400 nm in diameter (N_400). Our measurements also show the influence of local meteorology on aerosol surface area (A_140) and size distributions: A_140 is positively correlated with wind speed and relative humidity, negatively correlated with precipitation, and lower given westerly winds. Based on this study, we conclude that the POPSnet approach provides considerably more insight into the spatial variability in the aerosol population that can be used to constrain climate models than would be available from similar networks of PM 2.5 monitors.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A novel implicit hybrid machine learning model and its application for reinforcement learning

A novel methodology to develop implicit hybrid models is presented. PyTorch is used to integrate physics-based equations with machine learning models. Automatic differentiation of the hybrid model is leveraged to solve the implicit equations. Iterative solving enables gradient based updates to the machine learning model. The novel methodology is compared to an explicit hybrid approach on a continuously stirred tank reactor (CSTR). The novel method results in a lower modelling error. Both hybrid models effectively train with noisy data. To test the implicit hybrid model, it is employed as a reinforcement learning (RL) training model. The RL algorithm trained on the hybrid model outperforms real time optimization of the CSTR and performs nearly as well as RL trained directly on the CSTR and a traditional gradient based approach. Training RL directly on the CSTR requires over 60,000 system interactions compared to 6000 historical data points for hybrid model development.

42 ENGINEERING↗
DOE OSTI2021

Regional-scale, sector-specific evaluation of global CO2 inversion models using aircraft data from the ACT-America project

We use vertical profiles of airborne measurements of CO2 from frontal cases during the summer 2016 Atmospheric Carbon and Transport – America (ACT-America) campaign to evaluate the skill of a set of ten global CO2 inversion models participating in the Orbiting Carbon Observatory – 2 (OCO-2) Model Intercomparison Project (MIP). Model errors and biases (model minus observation) were categorized by region (Mid-Atlantic, Midwest, and South), frontal sector (warm or cold), and transport model (predominantly Tracer Model 5 (TM5) and Goddard Earth Observing System – Chemistry (GEOS-Chem)). Overall, the inversions reproduce the general structures of the observed vertical profiles and the enhanced / depleted low-level CO2 in warm / cold sectors, but tend to underestimate the magnitude of the sector difference in each region. In the Midwest and South warm sectors, inversion biases were about 1 ppm above 1500 m AGL, though model spread (quantified by interquartile range) was often even smaller; below 1500 m AGL, model biases were negative and about -2 ppm near the surface. For the Midwest and South cold sectors, models had +2-3 ppm biases below 1500 m AGL, but with comparable model spread. Uniquely, in the Mid-Atlantic there was a consistent difference between TM5 and GEOS-Chem mole fractions for both sectors up to 3000 m AGL (TM5 lower by 2 ppm), on the order of the observation-relative biases. In the MidAtlantic TM5 inversions had negative biases in warm sectors, while GEOS-Chem inversions had positive biases in cold sectors. Possible reasons for the regional variability are discussed.

Gaudet, Brian J.↗
DOE OSTI2026

A Hybrid Biophysical‐Machine Learning Framework for Diurnal Surface Energy Flux Estimation Using Proximal Sensing

Thermal infrared-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy ( LE ) and sensible heat ( H ) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal infrared data sets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of a ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R 2 = 0.81–0.94) and H (R 2 = 0.46–0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical-machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.

evapotranspiration↗
DOE OSTI2015

Uncertainty quantification in application of the enrichment meter principle for nondestructive assay of special nuclear material

The various methods of nondestructive assay (NDA) of special nuclear material (SNM) have applications in nuclear nonproliferation, including detection and identification of illicit SNM at border crossings, and quantifying SNM at nuclear facilities for safeguards. No assay method is complete without “error bars,” which provide one way of expressing confidence in the assay result. Consequently, NDA specialists typically quantify total uncertainty in terms of “random” and “systematic” components, and then specify error bars for the total mass estimate in multiple items. Uncertainty quantification (UQ) for NDA has always been important, but it is recognized that greater rigor is needed and achievable using modern statistical methods. To this end, we describe the extent to which the guideline for expressing uncertainty in measurements (GUM) can be used for NDA. Also, we propose improvements over GUM for NDA by illustrating UQ challenges that it does not address, including calibration with errors in predictors, model error, and item-specific biases. A case study is presented using low-resolution NaI spectra and applying the enrichment meter principle to estimate the U-235 mass in an item. The case study illustrates how to update the current American Society for Testing and Materials guide for application of the enrichment meter principle using gamma spectra from a NaI detector.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗