Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

A quantitative assessment of the model form error of friction models across different interface representations for jointed structures

Hysteretic models are widely used to model frictional interactions in joints to recreate experimental behavior. However, it is unclear which models are best suited for fitting or predicting the responses of structures. Here, the present study evaluates 26 friction model/interface representation combinations to quantify the model form error. A Quasi-Static Modal Analysis approach (termed Rayleigh Quotient Nonlinear Modal Analysis) is adopted to calculate the nonlinear system response, and a Multi-Objective Optimization is solved to fit experimental data of the first mode of the Brake-Reuß Beam. Optimized parameters from the first mode are applied to the second and third bending modes to quantify the predictive ability of the models. Formulations for both tracing full hysteresis loops and recreating hysteresis loops from a single loading curve (Masing assumptions) are considered. Smoothly varying models applied to a five patch representation showed the highest flexibility (for fitting mode 1) and good predictive potential (for modes 2 and 3). For a second formulation, which uses 152 frictional elements to represent the interface, the physically motivated spring in series with a Coulomb slip model (elastic dry friction) has high error for fitting mode 1 and performs near the middle for predicting higher modes. For both interface representation, the best fit models are not the most physical, but rather the ones with the most parameters (as expected); however, the more physical models perform somewhat better for predicting the higher modes.

42 ENGINEERING↗

Adaptive reduction of constitutive model-form error using a posteriori error estimation techniques

In engineering practice, models are typically kept as simple as possible for ease of setup and use, computational efficiency, maintenance, and overall reduced complexity to achieve robustness. In solid mechanics, a simple and efficient constitutive model may be favored over one that is more predictive, but is difficult to parameterize, is computationally expensive, or is simply not available within a simulation tool. In order to quantify the modeling error due to the choice of a relatively simple and less predictive constitutive model, we adopt the use of a posteriori model-form error-estimation techniques. Based on local error indicators in the energy norm, an algorithm is developed for reducing the modeling error by spatially adapting the material parameters in the simpler constitutive model. The resulting material parameters are not material properties per se , but depend on the given boundary-value problem. As a first step to the more general nonlinear case, we focus here on linear elasticity in which the “complex” constitutive model is general anisotropic elasticity and the chosen simpler model is isotropic elasticity. As a result, the algorithm for adaptive error reduction is demonstrated using two examples: (1) A transversely-isotropic plate with hole subjected to tension, and (2) a transversely-isotropic tube with two side holes subjected to torsion.

42 ENGINEERING↗

PYOED: AN ETENSIBLE SUITE FOR DATA ASSIMILATION AND MODEL-CONSTRAINED OPTIMAL DESIGN OF EXPERIMENTS

SF-23-005 PyOED is a highly extensible scientific package that enables developing and testing model-constrained optimal experimental design (OED) for inverse problems. Specifically, PyOED aims to be a comprehensive Python toolkit for model-constrained OED. The package targets scientists and researchers interested in understanding the details of OED formulations and approaches. It is also meant to enable researchers to experiment with standard and innovative OED technologies with a wide range of test problems (e.g., simulation models). OED, inverse problems (e.g., Bayesian inversion), and data assimilation (DA) are closely related research fields, and their formulations overlap significantly. Thus, PyOED is continuously being expanded with a plethora of Bayesian inversion, DA, and OED methods as well as new scientific simulation models, observation error models, and observation operators. These pieces are added such that they can be permuted to enable testing OED methods in various settings of varying complexities. The PyOED core is completely written in Python and utilizes the inherent object-oriented capabilities; however, PyOED is meant to be extensible rather than scalable. Specifically, PyOED is developed to ``enable rapid development and benchmarking of OED methods with minimal coding effort and to maximize code reutilization.'' PyOED will be continuously expanded with a plethora of Bayesian inversion, DA, and OED methods as well as new scientific simulation models, observation error models, and observation operators.

ATTIA, AHMEDMOHAMED↗

Effects of machine learning errors on human decision-making: manipulations of model accuracy, error types, and error importance

Abstract This study addressed the cognitive impacts of providing correct and incorrect machine learning (ML) outputs in support of an object detection task. The study consisted of five experiments that manipulated the accuracy and importance of mock ML outputs. In each of the experiments, participants were given the T and L task with T-shaped targets and L-shaped distractors. They were tasked with categorizing each image as target present or target absent. In Experiment 1, they performed this task without the aid of ML outputs. In Experiments 2–5, they were shown images with bounding boxes, representing the output of an ML model. The outputs could be correct (hits and correct rejections), or they could be erroneous (false alarms and misses). Experiment 2 manipulated the overall accuracy of these mock ML outputs. Experiment 3 manipulated the proportion of different types of errors. Experiments 4 and 5 manipulated the importance of specific types of stimuli or model errors, as well as the framing of the task in terms of human or model performance. These experiments showed that model misses were consistently harder for participants to detect than model false alarms. In general, as the model’s performance increased, human performance increased as well, but in many cases the participants were more likely to overlook model errors when the model had high accuracy overall. Warning participants to be on the lookout for specific types of model errors had very little impact on their performance. Overall, our results emphasize the importance of considering human cognition when determining what level of model performance and types of model errors are acceptable for a given task.

97 MATHEMATICS AND COMPUTING↗

Testing contamination source identification methods for water distribution networks

In the event of contamination in a water distribution network (WDN), source identification (SI) methods that analyze sensor data can be used to identify the source location(s). Knowledge of the source location and characteristics are important to inform contamination control and cleanup operations. Various SI strategies that have been developed by researchers differ in their underlying assumptions and solution techniques. The following manuscript presents a systematic procedure for testing and evaluating SI methods. The performance of these SI methods is affected by various factors including the size of WDN model, measurement error, modeling error, time and number of contaminant injections, and time and number of measurements. This paper includes test cases that vary these factors and evaluates three SI methods on the basis of accuracy and specificity. The tests are used to review and compare these different SI methods, highlighting their strengths in handling various identification scenarios. These SI methods and a testing framework that includes the test cases and analysis tools presented in this paper have been integrated into EPA’s Water Security Toolkit (WST), a suite of software tools to help researchers and others in the water industry evaluate and plan various response strategies in case of a contamination incident. Lastly, a set of recommendations are made for users to consider when working with different categories of SI methods.

54 ENVIRONMENTAL SCIENCES↗

Information transmission with continuous variable quantum erasure channels

Quantum capacity, as the key figure of merit for a given quantum channel, upper bounds the channel's ability in transmitting quantum information. Identifying different types of channels, evaluating the corresponding quantum capacity, and finding the capacity-approaching coding scheme are the major tasks in quantum communication theory. Quantum channel in discrete variables has been discussed enormously based on various error models, while error model in the continuous variable channel has been less studied due to the infinite dimensional problem. In this paper, we investigate a general continuous variable quantum erasure channel. By defining an effective subspace of the continuous variable system, we find a continuous variable random coding model. We then derive the quantum capacity of the continuous variable erasure channel in the framework of decoupling theory. The discussion in this paper fills the gap of a quantum erasure channel in continuous variable setting and sheds light on the understanding of other types of continuous variable quantum channels.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Bayesian characterization and forward propagation of the uncertainty in thermodynamic models for redox active materials

Demonstrating the thermodynamic efficiency of hydrogen conversion processes using various materials is a critical step in developing new technologies for storing concentrated solar energy, and is largely accomplished by using a thermodynamic model derived from experimental data. A main goal of this project is to calculate the uncertainty of the thermodynamic efficiency by calculating the uncertainty of the components that feed into the efficiency. Many different models and data sets were used to test the workflow. First, the models were fit to the data using a Bayesian Inference and a method called Markov Chain Monte Carlo (MCMC), which found the maximum a priori parameters, and a posterior probability distribution of the parameters. Next, the different models were compared to each other using model evidence values. It was found that for cleaner data sets, overfitting had not yet been reached, and the most complicated model was ideal, but on the noisier data sets, the less complex models were favored because the more complicated models resulted in overfitting. Next, forward propagation was used to calculate the enthalpy change and its associated uncertainty. A few variations on the models were tried, such as fitting in a different variable, producing negligible or negative effects on the fits of the models. Thus, the original models were used. A sensitivity analysis was performed, and used to calculate the model error. On the cleaner data sets, there was very minimal experimental noise, and thus, all resulting error was from the model. With consideration of the model error, the models fit the data very well, and the simpler model had a high model error, as expected. All these components will then be used to calculate the thermodynamic efficiency of the different materials.

14 SOLAR ENERGY↗

Model Form Error Correction for a Black-Box Thermal Battery Heat Transfer Simulation

Thermal batteries are crucial for supplying power to high-consequence engineering applications such as rockets. Computational simulations have been developed to predict thermal battery behavior, but these simulations often suffer from modeling errors, including model form uncertainty. Addressing this uncertainty can be achieved by quantifying either the model discrepancy in the output or the model form error (MFE) in the governing equation. MFE is particularly valuable as it can be better extrapolated beyond observed outputs, which is essential for predictions involving changes in external system loading, system configuration and geometry, or output quantities. This paper employs a state estimation approach to estimate MFE using experimental data and then utilizes machine learning (ML) to model its relationship with state variables. A nonintrusive technique is used to estimate MFE in a black-box thermal battery heat transfer simulation. The trained machine learning model for MFE is then applied to correct simulation predictions under extrapolated initial conditions and battery configurations. In conclusion, the methodology's performance is evaluated using additional experimental data, demonstrating its effectiveness in improving prediction accuracy.

Batteries↗

Modeling mechanical behavior of an additively manufactured metal structure with local texture variations: a study on model form error

Here, the repetitive rapid solidification that occurs in metal additive manufacturing (AM) processes creates microstructures distinctly different from wrought materials. Local variability in AM microstructures (either by design or unintentional) raises questions as to how AM structures should be modeled at the part-scale to minimize modeling error. The key goal of this work is to demonstrate a posteriori error estimation applied to an AM part. It is assumed that the actual microstructure is unknown and an approximate, spatially uniform material model is used. Error bounds are calculated for many reference models based on AM microstructures with elongated grain morphology and localized or global fiber textures during a post-processing step. The current findings promote confidence that a posteriori model form error estimation could be used effectively in mechanical performance simulations of AM parts to quickly obtain quantitative error metrics between an approximate model result and many microstructure-based reference models. The a posteriori error estimation introduces significant time savings compared to computing the full reference model solutions. Tight bounds on model form error are obtained when texture variations in the reference models occur on large length scales. For materials with property variation at small length scales, multi-scale error estimation techniques are needed to properly account for the many interfaces present between areas with different properties.

36 MATERIALS SCIENCE↗

Data for A Hybrid Biophysical-Machine Learning Framework for Diurnal Surface Energy Flux Estimation Using Proximal Sensing

Thermal infrared-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy (LE) and sensible heat (H) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal infrared data sets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of a ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R2 = 0.81–0.94) and H (R2 = 0.46–0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical-machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.

AI/ML↗

Learn-as-you-go acceleration of cosmological parameter estimates

Cosmological analyses can be accelerated by approximating slow calculations using a training set, which is either precomputed or generated dynamically. However, this approach is only safe if the approximations are well understood and controlled. This paper surveys issues associated with the use of machine-learning based emulation strategies for accelerating cosmological parameter estimation. We describe a learn-as-you-go algorithm that is implemented in the Cosmo++ code and (1) trains the emulator while simultaneously estimating posterior probabilities; (2) identifies unreliable estimates, computing the exact numerical likelihoods if necessary; and (3) progressively learns and updates the error model as the calculation progresses. We explicitly describe and model the emulation error and show how this can be propagated into the posterior probabilities. We apply these techniques to the Planck likelihood and the calculation of ΛCDM posterior probabilities. The computation is significantly accelerated without a pre-defined training set and uncertainties in the posterior probabilities are subdominant to statistical fluctuations. We have obtained a speedup factor of 6.5 for Metropolis-Hastings and 3.5 for nested sampling. Finally, we discuss the general requirements for a credible error model and show how to update them on-the-fly.

79 ASTRONOMY AND ASTROPHYSICS↗

A Method for the Quantification of Model Form Error Associated with Physical Systems

In the process of model validation, models are often declared valid when the differences between model predictions and experimental data sets are satisfactorily small. However, little consideration is given to the effectiveness of a model using parameters that deviate slightly from those that were fitted to data, such as a higher load level. Furthermore, few means exist to compare and choose between two or more models that reproduce data equally well. These issues can be addressed by analyzing model form error, which is the error associated with the differences between the physical phenomena captured by models and that of the real system. This report presents a new quantitative method for model form error analysis and applies it to data taken from experiments on tape joint bending vibrations. Two models for the tape joint system are compared, and suggestions for future improvements to the method are given. As the available data set is too small to draw any statistical conclusions, the focus of this paper is the development of a methodology that can be applied to general problems.

97 MATHEMATICS AND COMPUTING↗

Addressing Model Form Error for Time-Dependent Conservation Equations

Model form error of the type considered here is error due to an approximate or incorrect representation of physics by a computational model. Typical approaches to adjust a model based on observed differences between experiment and prediction are to calibrate the model parameters utilizing the observed discrepancies and to develop parameterized additive corrections to the model output. These approaches are generally not suitable if significant physics is missing from the model and the desired quantities of interest for an application are different than those used for calibration. The approach developed here is to build a corrected surrogate solver through a multi- step process: 1) Sampled simulation results are used to develop a surrogate computational solver that maintains the overall conservative principles of the unmodified governing equations, 2) the surrogate solver is applied to candidate linear and non-linear corrector terms to develop corrections that are consistent with the original conservative principles, 3) constant multipliers on the these terms are calibrated using the experimental observations, and 4) the resulting surrogate solver is used to predict application response for the quantity of interest. This approach and several other calibration-based approaches were applied to an example problem based on the diffusive Burgers' equation. While all the approaches provided some model correction when the measure/calibration quantity was the same as that for an application, only the present approach was able to adequately correct the CompSim results when the prediction quantity was different from the calibration quantity.

97 MATHEMATICS AND COMPUTING↗

Semi-Analytical Hierarchical Bayesian Inference of Nonlinear Model Structure in Stochastic Dynamics: Applied to Compartmental Models of Infectious Diseases

A Bayesian computational framework for parsimonious inference in stochastic nonlinear dynamical systems is presented. This framework enables the concurrent estimation of system states, time-varying parameters, time-invariant parameters, and the optimal sparsity structure of the model parameters. Because differential equation-based models are often simplified mechanistic or phenomenological representations, robust inference from noisy measurement data requires explicit treatment of model error and uncertainty. Model error and time-varying parameters can be represented as random processes, enabling inference while making minimal assumptions about the underlying sources of discrepancy and variability. Adopting stochastic differential equation representations affords the model significant flexibility, but can also render it susceptible to overfitting during statistical inversion, where the inferred model may track noise rather than the underlying signal. To alleviate the effects of overfitting and to enable the discovery of the optimal sparse representation of the time-invariant parameters, a Bayesian sparse learning algorithm is embedded within the framework. This sparse learning framework adopts an approximate hierarchical Bayesian setting defined by a series of semi-analytical expressions. The model structure inference framework is validated using a stochastic compartmental model for tracking and forecasting active cases of an infectious disease. Compartmental models describe population-level infectious disease dynamics through interactions among population fractions grouped by disease state. Mathematically, such models consist of a system of coupled ordinary differential equations. This example adopts an expressive compartmental model that includes multiple possible interactions between disease states, motivated by early uncertainty surrounding COVID-19 reinfection dynamics and their implications for long-term epidemic forecasting. The sparse learning exercise permits the inference of a priori unknown epidemiological dynamics from simulated public health data, discovering the nested compartmental model that optimizes the trade-off between average data-fit and model complexity. It is shown that inducing sparsity among the model parameters eliminates redundant interactions between compartments, equivalently revealing the optimal coupling structure between differential equations.

97 MATHEMATICS AND COMPUTING↗