Accounting for Model Error in the Calibration of Physical Models.
Abstract not provided.
Najm, Habib N.↗
SEARCH · Search NASA
Results for “MODEL CALIBRATION”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Advanced Performance Metrics and their Application to the Sensitivity Analysis for Model Validation and Calibration
High-quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMUs, measurement-based approach for model validation has gained significant prominence. In this approach, the quality of a model is analyzed by visually comparing measured generator response with the model-based simulated response for large system disturbances. This paper proposes a new set of performance metrics to assess the model validation results to facilitate automation of the model validation process. In the proposed methodology, first, the slow governor response and comparatively faster oscillatory response are separated, and then a separate set of performance metrics is calculated for each of these two components. These proposed metrics quantify the mismatch between the actual and model-based response in a comprehensive manner without missing any information enabling automation of the process. Furthermore, in this paper, we are also proposing that the sensitivity analysis for model calibration be performed with respect to the proposed metrics for the systematic identification of key parameters. In this work, results obtained using both simulated and real-world case-studies validate the effectiveness of the proposed performance metrics for model validation and their application to the sensitivity analysis for model calibration.
24 POWER TRANSMISSION AND DISTRIBUTION↗
In Quest of Calibration Density and Consistency in Hydrologic Modeling: Distributed Parameter Calibration against Streamflow Characteristics
Conventional basin-by-basin approaches to calibrate hydrologic models are limited to gauged basins and typically result in spatially discontinuous parameter fields. Moreover, the consequent low calibration density in space falls seriously behind the need from present-day applications like high resolution river hydrodynamic modeling. In this study we calibrated three key parameters of the Variable Infiltration Capacity (VIC) model at every 1/8° grid-cell using machine learning-based maps of four streamflow characteristics for the conterminous United States (CONUS), with a total of 52,663 grid-cells. This new calibration approach, as an alternative to parameter regionalization, applied to ungauged regions too. A key difference made here is that we tried to regionalize physical variables (streamflow characteristics) instead of model parameters whose behavior may often be less well understood. The resulting parameter fields no longer presented any spatial discontinuities and the patterns corresponded well with climate characteristics, such as aridity and runoff ratio. The calibrated parameters were evaluated against observed streamflow from 704/648 (calibration/validation period) small-to-medium-sized catchments used to derive the streamflow characteristics, 3941/3809 (calibration/validation period) small-to-medium-sized catchments not used to derive the streamflow characteristics as well as five large basins. Comparisons indicated marked improvements in bias and Nash-Sutcliffe efficiency. Model performance was still poor in arid and semiarid regions, which is mostly due to both model structural and forcing deficiencies. Although the performance gain was limited by the relative small number of parameters to calibrate, the study and results here served as a proof-of-concept for a new promising approach for fine-scale hydrologic model calibrations.
54 ENVIRONMENTAL SCIENCES↗
Development of a simplified calibrated building simulation model of a supermarket for proposed ECMs and control strategies impact evaluation
Calibrated building energy simulation is an important pathway to more energy-efficient buildings, but the information requirements of some approaches to this problem are significant. This is particularly true for supermarkets and other so-called “big-box” retail stores. Another characteristic of supermarkets is the significant interaction between Heating Ventilating and Air Conditioning (HVAC) and refrigeration systems in these buildings. These buildings could contain a wide variety of systems and a degree of load diversity that makes calibrated modeling a challenge. This paper describes a simplified approach that uses OpenStudio and EnergyPlus to combine known building parameters with “typical” parameters, resulting in a simplified building that is amenable to calibration. This approach was applied to a big-box store located in Nashville, Tennessee, and a calibrated model was obtained that was used to study potential energy conservation measures. Further, the paper also explores the capabilities of whole-building energy modeling tools, such as EnergyPlus, for modeling the HVAC controls and sequences and their impact evaluation. Although some measures are precluded by the model simplicity, several measures were found to improve the efficiency of the model and demonstrate that the simplified modeling approach is effective. Practical Application: This paper introduces a hybrid approach of building energy model calibration using limited information available from the actual building in combination with characteristics of a “typical” building of the same type. This hybrid approach would also be applicable for other building types than discussed in this paper to calibrate the building energy model using limited information from the actual building.
42 ENGINEERING↗
Comprehensive compartmental model and calibration algorithm for the study of clinical implications of the population-level spread of COVID-19: a study protocol
The complex dynamics of the coronavirus disease 2019 (COVID-19) pandemic has made obtaining reliable long-term forecasts of the disease progression difficult. Simple mechanistic models with deterministic parameters are useful for short-term predictions but have ultimately been unsuccessful in extrapolating the trajectory of the pandemic because of unmodelled dynamics and the unrealistic level of certainty that is assumed in the predictions. We propose a 22-compartment epidemiological model that includes compartments not previously considered concurrently, to account for the effects of vaccination, asymptomatic individuals, inadequate access to hospital care, post-acute COVID-19 and recovery with long-term health complications. Additionally, new connections between compartments introduce new dynamics to the system and provide a framework to study the sensitivity of model outputs to several concurrent effects, including temporary immunity, vaccination rate and vaccine effectiveness. Subject to data availability for a given region, we discuss a means by which population demographics (age, comorbidity, socioeconomic status, sex and geographical location) and clinically relevant information (different variants, different vaccines) can be incorporated within the 22-compartment framework. Considering a probabilistic interpretation of the parameters allows the model's predictions to reflect the current state of uncertainty about the model parameters and model states. We propose the use of a sparse Bayesian learning algorithm for parameter calibration and model selection. This methodology considers a combination of prescribed parameter prior distributions for parameters that are known to be essential to the modelled dynamics and automatic relevance determination priors for parameters whose relevance is questionable. This is useful as it helps prevent overfitting the available epidemiological data when calibrating the parameters of the proposed model. Population-level administrative health data will serve as partial observations of the model states.
59 BASIC BIOLOGICAL SCIENCES↗
Discrete Regularization for Calibration of Geologic Facies Against Dynamic Flow Data
Abstract Subsurface flow model calibration involves many more unknowns than measurements, leading to ill‐posed problems with nonunique solutions. To alleviate nonuniqueness, the problem is regularized by constraining the solution space using prior knowledge. In certain sedimentary environments, such as fluvial systems, the contrast in hydraulic properties of different facies types tends to dominate the flow and transport behavior, making the effect of within facies heterogeneity less significant. Hence, flow model calibration in those formations reduces to delineating the spatial structure and connectivity of different lithofacies types and their boundaries. A major difficulty in calibrating such models is honoring the discrete, or piecewise constant, nature of facies distribution. The problem becomes more challenging when complex spatial connectivity patterns with higher‐order statistics are involved. This paper introduces a novel formulation for calibration of complex geologic facies by imposing appropriate constraints to recover plausible solutions that honor the spatial connectivity and discreteness of facies models. To incorporate prior connectivity patterns, plausible geologic features are learned from available training models. This is achieved by learning spatial patterns from training data, e.g., ‐SVD sparse learning or the traditional Principal Component Analysis. Discrete regularization is introduced as a penalty functions to impose solution discreteness while minimizing the mismatch between observed and predicted data. An efficient gradient‐based alternating directions algorithm is combined with variable splitting to minimize the resulting regularized nonlinear least squares objective function. Numerical results show that imposing learned facies connectivity and discreteness as regularization functions leads to geologically consistent solutions that improve facies calibration quality.
Khaninezhad, Mohammad‐Reza↗
Selecting Summary Statistics in Approximate Bayesian Computation for Calibrating Stochastic Models
Approximate Bayesian computation (ABC) is an approach for using measurement data to calibrate stochastic computer models, which are common in biology applications. ABC is becoming the “go-to” option when the data and/or parameter dimension is large because it relies on user-chosen summary statistics rather than the full data and is therefore computationally feasible. One technical challenge with ABC is that the quality of the approximation to the posterior distribution of model parameters depends on the user-chosen summary statistics. In this paper, the user requirement to choose effective summary statistics in order to accurately estimate the posterior distribution of model parameters is investigated and illustrated by example, using a model and corresponding real data of mitochondrial DNA population dynamics. We show that for some choices of summary statistics, the posterior distribution of model parameters is closely approximated and for other choices of summary statistics, the posterior distribution is not closely approximated. A strategy to choose effective summary statistics is suggested in cases where the stochastic computer model can be run at many trial parameter settings, as in the example.
59 BASIC BIOLOGICAL SCIENCES↗
Quantification of manganese for ChemCam Mars and laboratory spectra using a multivariate model
In this work, we report a new calibration model for manganese using the laser-induced breakdown spectroscopy instrument that is part of the ChemCam instrument suite onboard the NASA Curiosity rover. The model has been trained using an expanded set of 523 manganese-bearing rock, mineral, metal ore, and synthetic standards. The optimal calibration model uses the Partial Least Squares (PLS) and Least Absolute Shrinkage and Selection Operator (LASSO) multivariate techniques, with a novel “double blending” technique. We determined the detection limit for manganese is 82 ppm using a method blank procedure and is possibly as low as 27 ppm based on visual inspection of the spectra. Based on a representative test set consisting of measurements on 93 standards, the double blended multivariate model shows a Root Mean Squared Error of Prediction (RMSEP) accuracy of 1.39 wt% MnO for the full blended model. Employing a local RMSEP estimate where the model performance is evaluated based on nearby test samples, the accuracy is 0.03 wt% at the quantification limit (0.05 wt% MnO), 0.4 wt% accuracy at 1.0 wt% MnO, and 4.4 wt% accuracy at 100 wt% MnO. Precision is estimated using the standard deviation of the test set measurements, and is ±0.01 wt% MnO at the quantification limit, ±0.09 wt% MnO at 1.0 wt% MnO, and ± 2.1 wt% MnO at 100 wt% MnO (all 1 standard deviation). This new calibration is important for understanding the variation of manganese in the bedrock with the Curiosity rover on Mars, which provides insight into past redox conditions on Mars.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Power system model parameter conditioning tool
A power system model parameter conditioning tool including a server control processor in communication with phasor measurement unit monitored data records of multiple disturbance events, a model calibration unit providing event screening, power system model simulation, and simultaneous tuning of model parameters. The model calibration performing a simulation using default model parameters, the processor comparing the simulation results to the monitored data. If the prediction is within threshold, then terminating conditioning; else performing parameter identifiability analysis to determine differing effects of various model parameters on power system model accuracy, selecting a parameter set causing a degradation in power system model prediction, and updating the default model parameters corresponding to members of the parameter set with values selected to reduce the degradation. A method and a non-transitory computer readable medium are also disclosed.
Baone, Chaitanya Ashok↗
Modeling and Calibration of Supplier Selection Problem in Freight Agent-Based Simulations
Freight transportation modeling often struggles with data limitations, especially in accurately representing complex supplier selection processes and their impact on network flows. This research addresses this critical gap by developing a large-scale, calibrated agent-based model for supplier selection, complemented by a probabilistic heuristic for international shipments. Our approach integrates trade relationships between industry sectors, transportation costs, and a supplier-rating model adapted from existing literature. The model’s core objective is to minimize the discrepancy between modeled and observed commodity flows while ensuring a close match to regional shipping distance distributions. Implemented and tested across four major U.S. metropolitan areas—Atlanta, Chicago, Dallas–Fort Worth, and Los Angeles—the model demonstrates high fidelity in replicating observed freight patterns. Key findings reveal consistent alignment with national shipping distance trends and highlight significant spatial variations in commodity trade assignments and demand across the study regions. This behaviorally informed and transport-sensitive framework is designed to approximate real-world decision making, providing a robust tool for policymakers and planners to evaluate targeted interventions, assess infrastructure investments, and enhance supply chain resilience in the face of disruptions.
Ismael, Abdelrahman (ORCID:0000000303712110)↗
A leaf-level spectral library to support high-throughput plant phenotyping: predictive accuracy and model transfer
Abstract Leaf-level hyperspectral reflectance has become an effective tool for high-throughput phenotyping of plant leaf traits due to its rapid, low-cost, multi-sensing, and non-destructive nature. However, collecting samples for model calibration can still be expensive, and models show poor transferability among different datasets. This study had three specific objectives: first, to assemble a large library of leaf hyperspectral data (n=2460) from maize and sorghum; second, to evaluate two machine-learning approaches to estimate nine leaf properties (chlorophyll, thickness, water content, nitrogen, phosphorus, potassium, calcium, magnesium, and sulfur); and third, to investigate the usefulness of this spectral library for predicting external datasets (n=445) including soybean and camelina using extra-weighted spiking. Internal cross-validation showed satisfactory performance of the spectral library to estimate all nine traits (mean R2=0.688), with partial least-squares regression outperforming deep neural network models. Models calibrated solely using the spectral library showed degraded performance on external datasets (mean R2=0.159 for camelina, 0.337 for soybean). Models improved significantly when a small portion of external samples (n=20) was added to the library via extra-weighted spiking (mean R2=0.574 for camelina, 0.536 for soybean). The leaf-level spectral library greatly benefits plant physiological and biochemical phenotyping, whilst extra-weight spiking improves model transferability and extends its utility.
59 BASIC BIOLOGICAL SCIENCES↗
Electronic transport in VO 2 —Experimentally calibrated Boltzmann transport modeling
Materials that undergo metal-insulator transitions (MITs) are under intense study because the transition is scientifically fascinating and technologically promising for various applications. Among these materials, VO 2 has served as a prototype due to its favorable transition temperature. While the physical underpinnings of the transition have been heavily investigated experimentally and computationally, quantitative modeling of electronic transport in the two phases has yet to be undertaken. In this work, we establish a density-functional-theory (DFT)-based approach to model electronic transport properties in VO 2 in the semiconducting and metallic regimes, focusing on band transport using the Boltzmann transport equations. To do so, we synthesized high quality VO 2 films and measured the transport quantities across the transition, in order to calibrate the free parameters in the model. We find that the experimental calibration of the Hubbard correction term can efficiently and adequately model the metallic and semiconducting phases, allowing for further computational design of MIT materials for desirable transport properties.
36 MATERIALS SCIENCE↗
Calibration of the Nonlinear Viscoelastic SPECtacular Model for 828-CTBN/DEA/GMB and 828/Z/AlOx
Nonlinear viscoelastic model calibrations are developed for two filled epoxies: an adduct of Epon 828 DGEBA and CTBN cross-linked with diethanolamine (DEA) filled with glass microballoons, called 828-CTBN/DEA/GMB, and Epon 828 DGEBA cross-linked with Ancamine Z and filled with alumina, called 828/Z/AlOx. The thermo-mechanical responses of these materials are represented using the SPECtacular constitutive model, an extension of the Simplified Potential Energy Clock (SPEC) model. The model couples
Cundiff, Kenneth Noel [Sandia National Laboratorie↗
Hybrid nested sampling algorithm for Bayesian model selection applied to inverse subsurface flow problems
A Hybrid Nested Sampling (HNS) algorithm is proposed for efficient Bayesian model calibration and prior model selection. The proposed algorithm combines, Nested Sampling (NS) algorithm, Hybrid Monte Carlo (HMC) sampling and gradient estimation using Stochastic Ensemble Method (SEM). NS is an efficient sampling algorithm that can be used for Bayesian calibration and estimating the Bayesian evidence for prior model selection. Nested sampling has the advantage of computational feasibility. Within the nested sampling algorithm, a constrained sampling step is performed. For this step, we utilize HMC to reduce the correlation between successive sampled states. HMC relies on the gradient of the logarithm of the posterior distribution, which we estimate using a stochastic ensemble method based on an ensemble of directional derivatives. SEM only requires forward model runs and the simulator is then used as a black box and no adjoint code is needed. The developed HNS algorithm is successfully applied for Bayesian calibration and prior model selection of several nonlinear subsurface flow problems.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Energy Performance Assessment of Radiant Cooling System through Modeling and Calibration at Component Level
The paper describes a case study of an information technology office building with a radiant cooling system and a conventional variable air volume (VAV) system installed side by side so that performancecan be compared. First, a 3D model of the building involving architecture, occupancy, and HVAC operation was developed in EnergyPlus, a simulation tool. Second, a different calibration methodology was applied to develop the base case for assessing the energy saving potential. This paper details the calibration of the whole building energy model to the component level, including lighting, equipment, and HVAC components such as chillers, pumps, cooling towers, fans, etc. Also a new methodology for the systematic selection of influence parameter has been developed for the calibration of a simulated model which requires large time for the execution. The error at the whole building level [measured in mean bias error (MBE)] is 0.2%, and the coefficient of variation of root mean square error (CvRMSE) is 3.2%. The total errors in HVAC at the hourly are MBE = 8.7% and CvRMSE = 23.9%, which meet the criteria of ASHRAE 14 (2002) for hourly calibration. Different suggestions have been pointed out to generalize the energy saving of radiant cooling system through the existing building system. So a base case model was developed by using the calibrated model for quantifying the energy saving potential of the radiant cooling system. It was found that a base case radiant cooling system integrated with DOAS can save 28% energy compared with the conventional VAV system.
Building Energy Simulation↗