Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Stochastic modeling and statistical calibration with model error and scarce data

This paper introduces a procedure to assess the predictive accuracy of stochastic models subject to model error and sparse data. Model error is introduced as uncertainty on the coefficients of appropriate polynomial chaos expansions (PCE). The error associated with finite sample size allows us to conceive of these coefficients as statistics of the data that we describe as random variables whose influence on output quantities of interest is evaluated through the extended polynomial chaos expansion (EPCE). A Bayesian data assimilation scheme is introduced to update these expansions by considering the resulting nested chaos expansion as a hierarchical probabilistic model. Stochastic models of quantities of interest (QoI) are thus constructed and efficiently evaluated. Here, the Metropolis–Hastings Markov chain Monte Carlo procedure is used to sample the posterior. Two illustrative analytical and numerical problems are used to demonstrate the proposed approach.

Bayesian inference↗
DOE OSTI2023

Measurements and Model Improvement: Insight into NWP Model Error Using Doppler Lidar and Other WFIP2 Measurement Systems

Abstract Doppler-lidar wind-profile measurements at three sites were used to evaluate NWP model errors from two versions of NOAA’s 3-km-grid HRRR model, to see whether updates in the latest version 4 reduced errors when compared against the original version 1. Nested (750-m grid) versions of each were also tested to see how grid spacing affected forecast skill. The measurements were part of the field phase of the Second Wind Forecasting Improvement Project (WFIP2), an 18-month deployment into central Oregon–Washington, a major wind-energy-producing region. This study focuses on errors in simulating marine intrusions, a summertime, 600–800-m-deep, regional sea-breeze flow found to generate large errors. HRRR errors proved to be complex and site dependent. The most prominent error resulted from a premature drop in modeled marine-intrusion wind speeds after local midnight, when lidar-measured winds of greater than 8 m s −1 persisted through the next morning. These large negative errors were offset at low levels by positive errors due to excessive mixing, complicating the interpretation of model “improvement,” such that the updates to the full-scale versions produced mixed results, sometimes enhancing but sometimes degrading model skill. Nesting consistently improved model performance, with version 1’s nest producing the smallest errors overall. HRRR’s ability to represent the stages of sea-breeze forcing was evaluated using radiation budget, surface-energy balance, and near-surface temperature measurements available during WFIP2. The significant site-to-site differences in model error and the complex nature of these errors mean that field-measurement campaigns having dense arrays of profiling sensors are necessary to properly diagnose and characterize model errors, as part of a systematic approach to NWP model improvement. Significance Statement Dramatic increases in NWP model skill will be required over the coming decades. This paper describes the role of major deployments of accurate profiling sensors in achieving that goal and presents an example from the Second Wind Forecast Improvement Program (WFIP2). Wind-profile data from scanning Doppler lidars were used to evaluate two versions of HRRR, the original and an updated version, and nested versions of each. This study focuses on the ability of updated HRRR versions to improve upon predicting a regional sea-breeze flow, which was found to generate large errors by the original HRRR. Updates to the full-scale HRRR versions produced mixed results, but the finer-mesh versions consistently reduced model errors.

Meteorology & Atmospheric Sciences↗
DOE OSTI2023

Uncertainty Error Modeling for Non-Linear State Estimation With Unsynchronized SCADA and µPMU Measurements

Distribution systems of the future smart grid require enhancements to the reliability of distribution system state estimation (DSSE) in the face of low measurement redundancy, unsynchronized measurements, and dynamic load profiles. Micro phasor measurement units (µPMUs) facilitate co-synchronized measurements with high granularity, albeit at an often prohibitively expensive installation cost. Supervisory control and data acquisition (SCADA) measurements can supplement µPMU data, although they are received at a slower sampling rate. Further complicating matters is the uncertainty associated with load dynamics and unsynchronized measurements–not only are the SCADA and µPMU measurements not synchronized with each other, but the SCADA measurements themselves are received at different time intervals with respect to one another. This paper proposes a non-linear state estimation framework which models dynamic load uncertainty error by updating the variances of the unsynchronized measurements, leading to a time-varying system of weights in the weighted least squares state estimator. Case studies are performed on the 33-Bus Distribution System in MATPOWER, using Ornstein–Uhlenbeck stochastic processes to simulate dynamic load conditions.

Cooper, Austin↗
DOE OSTI2023

Effects of head modeling errors on the spatial frequency representation of MEG

Abstract Objectives. We aim to investigate the effects of head model inaccuracies on signal and source reconstruction accuracies for various sensor array distances to the head. This allows for the assessment of the importance of head modeling for next-generation magnetoencephalography (MEG) sensors, optically-pumped magnetometers (OPM). Approach. A 1-shell boundary element method (BEM) spherical head model with 642 vertices of radius 9 cm and conductivity of 0.33 S m −1 was defined. The vertices were then randomly perturbed radially up to 2%, 4%, 6%, 8% and 10% of the radius. For each head perturbation case, the forward signal was calculated for dipolar sources located at 2 cm, 4 cm, 6 cm and 8 cm from the origin (center of the sphere), and for a 324 sensor array located at 10 cm to 15 cm from the origin. Equivalent current dipole (ECD) source localization was performed for each of these forward signals. The signal for each perturbed spherical head case was then analyzed in the spatial frequency domain, and the signal and ECD errors were quantified relative to the unperturbed case. Main results. In the noiseless and high signal-to-noise ratio (SNR) case of approximately ≥6 dB, inaccuracies in our spherical BEM head conductor models lead to increased signal and ECD inaccuracies when sensor arrays are placed closer to the head. This is true especially in the case of deep and superficial sources. In the noisy case however, the higher SNR for closer sensor arrays allows for an improved ECD fit and outweighs the effects of head geometry inaccuracies. Significance. OPMs may be placed directly on the head, as opposed to the more commonly used superconducting quantum interference device sensors which must be placed a few centimeters away from the head. OPMs thus allow for signals of higher spatial resolution to be captured, resulting in potentially more accurate source localizations. Our results suggest that an increased emphasis on accurate head modeling for OPMs may be necessary to fully realize its improved source localization potential.

Yeo, Wan-Jin (ORCID:0000000241417123)↗
DOE OSTI2025

Reducing model error using optimized galaxy selection: weak lensing cluster mass estimation

Galaxy clusters are one of the most powerful probes to study extensions of General Relativity and the Standard Cosmological Model. Upcoming surveys like the Vera Rubin Observatory’s Legacy Survey of Space and Time are expected to revolutionise the field, by enabling the analysis of cluster samples of unprecedented size and quality. To reach this era of high-precision cluster cosmology, the mitigation of sources of systematic error is crucial. A particularly important challenge is bias in cluster mass measurements induced by the mismodelling of photometric redshift estimates of source galaxies. This work proposes a method to optimise the source sample selection in cluster weak lensing analyses drawn from wide-field survey lensing catalogs to reduce the bias on reconstructed cluster masses. We use a combinatorial optimisation scheme and methods from variational inference to select galaxies in latent space to produce a probabilistic galaxy source sample catalog for highly accurate cluster mass estimation. We show that our method reduces the critical surface mass density Σ crit relative modelling bias on the 60-70% level, while maintaining up to 90% of galaxies. We highlight that our methodology has applications beyond cluster mass estimation as an approach to jointly combine galaxy selection and model inference under sources of systematics.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Advanced Error Modeling for Prompt Yield Estimation [Slides]

Multi-Phenomenology Explosion Monitoring (MultiPEM) Toolbox provides a capability for post-detonation characterization focusing on yield estimation; Synthesize information from multiple sensor types; Leverage historical databases to inform statistical models; Allows for rapid or comprehensive analysis; Implemented in R code.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2024

Identifying Meteorological Drivers for Errors in Modeled Winds along the Northern California Coast

Abstract An accurate wind resource dataset is required for assessing the potential energy yield of floating offshore wind farms that are expected along the California outer continental shelf. The National Renewable Energy Laboratory has developed and disseminated an updated wind resource dataset offshore of California, using the Weather Research and Forecasting Model, referred to as the CA20 dataset. As compared to buoy lidar measurements that have become available recently, the CA20 dataset showed significant positive biases for 100-m wind speeds along Northern California wind energy lease areas. To investigate the meteorological drivers for the model errors, we first consider two 1-yr simulations run with two different planetary boundary layer (PBL) parameterizations: the Mellor–Yamada–Nakanishi–Niino (MYNN) PBL scheme (the chosen configuration in the CA20 dataset) and the Yonsei University PBL scheme (which significantly reduces the bias in modeled winds). By comparing the 1-yr simulations to the concurrent lidar buoy observations, we find that errors are larger with the MYNN PBL scheme in warm seasons. We then dive deeper into the analysis by running simulations for short-term (3-day) case studies to evaluate the sensitivity of initial/boundary condition forcings on model results. By analyzing the short-term simulations, we find that during synoptic-scale northerly flows driven by the North Pacific high and inland thermal low, a coastal warm bias in the MYNN simulation is mainly responsible for the modeled wind speed bias by altering the boundary layer thermodynamics. The results of our analysis will help guide the creation of an updated version of the CA20 dataset.

17 WIND ENERGY↗
DOE OSTI2024

Effects of machine learning errors on human decision-making: manipulations of model accuracy, error types, and error importance

Abstract This study addressed the cognitive impacts of providing correct and incorrect machine learning (ML) outputs in support of an object detection task. The study consisted of five experiments that manipulated the accuracy and importance of mock ML outputs. In each of the experiments, participants were given the T and L task with T-shaped targets and L-shaped distractors. They were tasked with categorizing each image as target present or target absent. In Experiment 1, they performed this task without the aid of ML outputs. In Experiments 2–5, they were shown images with bounding boxes, representing the output of an ML model. The outputs could be correct (hits and correct rejections), or they could be erroneous (false alarms and misses). Experiment 2 manipulated the overall accuracy of these mock ML outputs. Experiment 3 manipulated the proportion of different types of errors. Experiments 4 and 5 manipulated the importance of specific types of stimuli or model errors, as well as the framing of the task in terms of human or model performance. These experiments showed that model misses were consistently harder for participants to detect than model false alarms. In general, as the model’s performance increased, human performance increased as well, but in many cases the participants were more likely to overlook model errors when the model had high accuracy overall. Warning participants to be on the lookout for specific types of model errors had very little impact on their performance. Overall, our results emphasize the importance of considering human cognition when determining what level of model performance and types of model errors are acceptable for a given task.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Model Form Error Correction for a Black-Box Thermal Battery Heat Transfer Simulation

Thermal batteries are crucial for supplying power to high-consequence engineering applications such as rockets. Computational simulations have been developed to predict thermal battery behavior, but these simulations often suffer from modeling errors, including model form uncertainty. Addressing this uncertainty can be achieved by quantifying either the model discrepancy in the output or the model form error (MFE) in the governing equation. MFE is particularly valuable as it can be better extrapolated beyond observed outputs, which is essential for predictions involving changes in external system loading, system configuration and geometry, or output quantities. This paper employs a state estimation approach to estimate MFE using experimental data and then utilizes machine learning (ML) to model its relationship with state variables. A nonintrusive technique is used to estimate MFE in a black-box thermal battery heat transfer simulation. The trained machine learning model for MFE is then applied to correct simulation predictions under extrapolated initial conditions and battery configurations. In conclusion, the methodology's performance is evaluated using additional experimental data, demonstrating its effectiveness in improving prediction accuracy.

Batteries↗
DOE OSTI2026

Data for A Hybrid Biophysical-Machine Learning Framework for Diurnal Surface Energy Flux Estimation Using Proximal Sensing

Thermal infrared-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy (LE) and sensible heat (H) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal infrared data sets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of a ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R2 = 0.81–0.94) and H (R2 = 0.46–0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical-machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.

AI/ML↗
DOE OSTI2026

Semi-Analytical Hierarchical Bayesian Inference of Nonlinear Model Structure in Stochastic Dynamics: Applied to Compartmental Models of Infectious Diseases

A Bayesian computational framework for parsimonious inference in stochastic nonlinear dynamical systems is presented. This framework enables the concurrent estimation of system states, time-varying parameters, time-invariant parameters, and the optimal sparsity structure of the model parameters. Because differential equation-based models are often simplified mechanistic or phenomenological representations, robust inference from noisy measurement data requires explicit treatment of model error and uncertainty. Model error and time-varying parameters can be represented as random processes, enabling inference while making minimal assumptions about the underlying sources of discrepancy and variability. Adopting stochastic differential equation representations affords the model significant flexibility, but can also render it susceptible to overfitting during statistical inversion, where the inferred model may track noise rather than the underlying signal. To alleviate the effects of overfitting and to enable the discovery of the optimal sparse representation of the time-invariant parameters, a Bayesian sparse learning algorithm is embedded within the framework. This sparse learning framework adopts an approximate hierarchical Bayesian setting defined by a series of semi-analytical expressions. The model structure inference framework is validated using a stochastic compartmental model for tracking and forecasting active cases of an infectious disease. Compartmental models describe population-level infectious disease dynamics through interactions among population fractions grouped by disease state. Mathematically, such models consist of a system of coupled ordinary differential equations. This example adopts an expressive compartmental model that includes multiple possible interactions between disease states, motivated by early uncertainty surrounding COVID-19 reinfection dynamics and their implications for long-term epidemic forecasting. The sparse learning exercise permits the inference of a priori unknown epidemiological dynamics from simulated public health data, discovering the nested compartmental model that optimizes the trade-off between average data-fit and model complexity. It is shown that inducing sparsity among the model parameters eliminates redundant interactions between compartments, equivalently revealing the optimal coupling structure between differential equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

PyOED: An Extensible Suite for Data Assimilation and Model-Constrained Optimal Design of Experiments

This article describes PyOED, a highly extensible scientific package that enables developing and testing model-constrained optimal experimental design (OED) for inverse problems. Specifically, PyOED aims to be a comprehensive Python toolkit for model-constrained OED. The package targets scientists and researchers interested in understanding the details of OED formulations and approaches. It is also meant to enable researchers to experiment with standard and innovative OED technologies with a wide range of test problems (e.g., simulation models). OED, inverse problems (e.g., Bayesian inversion), and data assimilation (DA) are closely related research fields, and their formulations overlap significantly. Thus, PyOED is continuously being expanded with a plethora of Bayesian inversion, DA, and OED methods as well as new scientific simulation models, observation error models, and observation operators. These pieces are added such that they can be permuted to enable testing OED methods in various settings of varying complexities. The PyOED core is completely written in Python and utilizes the inherent object-oriented capabilities; however, the current version of PyOED is meant to be extensible rather than scalable. Specifically, PyOED is developed to “enable rapid development and benchmarking of OED methods with minimal coding effort and to maximize code reutilization.” This article provides a brief description of the PyOED layout and philosophy and provides a set of exemplary test cases and tutorials to demonstrate the potential of the package.

97 MATHEMATICS AND COMPUTING↗