Search NASA⌕ Search

SEARCH · Search NASA

Results for “model validation and calibration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Building Model Calibration: Validation of GridLAB-D Thermal Dynamics Modeling

This report investigates how well GridLAB-D’s house model characterizes the thermal dynamics of buildings given the overpredicted diurnal electric load swing observed in the Distribution System Operation with Transactive (DSO+T) study. This study seeks to validate GridLAB-D’s house model by calibrating it to data from the well-instrumented Pacific Northwest National Laboratory Lab Homes in Richland, WA. The datasets chosen included multiple pre-cooling and pre-heating testing periods where the indoor air temperature was allowed to float over a multi-hour length of time to mimic diurnal behavior. The multi-season calibrations were evaluated by comparing the heating/cooling electric power, indoor air temperature, and the rise/decay time during temperature float periods with Lab Homes data. The default GridLAB-D assumptions for latent load fraction, air heat capacity, mass heat capacity, window-to-wall-ratio, overall envelope conductance (assumed as NORMAL thermal integrity level), and solar heat gain coefficient were compared with the calibrated model to confirm when the default assumptions were adequate and the impact of calibration on the accuracy of modeling the thermal dynamics of homes.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

End-Use Load Profiles for the U.S. Building Stock: Methodology and Results of Model Calibration, Validation, and Uncertainty Quantification

The United States is embarking on an ambitious transition to a 100% clean energy economy by 2050, which will require improving the flexibility of electric grids. One way to achieve grid flexibility is to shed or shift demand to align with changing grid needs. To facilitate this, it is critical to understand how and when energy is used. High- quality end-use load profiles (EULPs) provide this information, and can help cities, states, and utilities understand the time-sensitive value of energy efficiency, demand response, and distributed energy resources. Publicly available EULPs have traditionally had limited application because of age and incomplete geographic representation (Frick, Eckman, and Goldman 2017; Frick 2019). To help fill this gap, the U.S. Department of Energy (DOE) funded a three-year project - End-Use Load Profiles for the U.S. Building Stock - that culminated in the release of a publicly available dataset1 of simulated EULPs representing residential and commercial buildings across the contiguous United States. The motivation for this work is further detailed in a November 2019 report: Market Needs, Use Cases, and Data Gaps (Mims Frick et al. 2019). This Methodology and Results report provides detailed descriptions of how the dataset was developed, intended for an audience of dataset and model users interested in the technical details. These details include descriptions of all of the model improvements made for calibration and the final comparisons to empirical data sources. A companion report, End-Use Load Profiles for the U.S. Building Stock: Applications and Opportunities, will be published subsequently and will describe example applications and considerations for using the dataset, intended for an audience of general dataset users.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Advanced Performance Metrics and their Application to the Sensitivity Analysis for Model Validation and Calibration

High-quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMUs, measurement-based approach for model validation has gained significant prominence. In this approach, the quality of a model is analyzed by visually comparing measured generator response with the model-based simulated response for large system disturbances. This paper proposes a new set of performance metrics to assess the model validation results to facilitate automation of the model validation process. In the proposed methodology, first, the slow governor response and comparatively faster oscillatory response are separated, and then a separate set of performance metrics is calculated for each of these two components. These proposed metrics quantify the mismatch between the actual and model-based response in a comprehensive manner without missing any information enabling automation of the process. Furthermore, in this paper, we are also proposing that the sensitivity analysis for model calibration be performed with respect to the proposed metrics for the systematic identification of key parameters. In this work, results obtained using both simulated and real-world case-studies validate the effectiveness of the proposed performance metrics for model validation and their application to the sensitivity analysis for model calibration.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Inverter Model Validation and Calibration Using Phasor Measurement Unit Data

As the penetration of inverter-based renewable energy resources increases in the power grid, especially at the distribution and microgrid levels, the need to accurately represent them in planning studies increases as well. However, due to the lack of well-established standard procedures, and vendor reluctance towards the detailed sharing of proprietary models, automated dynamic model validation and parameter calibration tools for inverter based resources (IBRs) remain scarce. This work presents a model validation and parameter calibration platform for representing IBRs with generic phasor-domain models. Phasor measurements of power system events are used for continuous validation using the data playback method, and model parameters are re-calibrated if a significant mismatch between measurements and model response is observed. Unique features of the proposed platform include- (a) an iterative Bayesian optimization approach towards parameter calibration to address a possible mismatch between the structures of generic models implemented in simulation softwares and actual commercial inverters, (b) error metrics designed to account for a possible mismatch between the time resolution of simulation and measurements, and (c) analysis of the measurement-simulation mismatch to provide guidance to engineering personnel regarding model shortcomings. The performance of the platform has been illustrated using both simulated data and field measurements to validate/calibrate inverter models in GridLAB-D.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

A Framework for Model Validation and Calibration of Microgrid Components Using PMU Data

This paper presents a framework for phasor measurement-based component model validation and calibration islanded microgrids, with emphasis on grid-forming inverters. Real measurements from the Plum Island microgrid facility are used to calibrate the model of a commercial inverter in the open-source phasor-based distribution system simulator GridLAB-D.Limitations of current distribution system simulators in replicating the behavior of commercial inverters are also identified.

microgrid, Synchrophasor data, model validation↗
DOE OSTI2024

A Causal Approach to Model Validation and Calibration

This poster presents a novel method for validation and verification that focuses on identifying causal relationships between data elements, moving beyond traditional statistical and machine learning approaches. These methods employ causal discovery techniques to reveal the underlying mechanisms of data generation. The research utilizes structural causal models and directed acyclic graphs to depict causal relationships. This approach assists in achieving alignment between simulation models and reality.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Large Language Model for Validation, Optical Calibration, and Learning (VOCAL) Distributed Temperature Sensing Interface

Distributed temperature sensing (DTS) using fiber optic sensors (FOS) offers a promising method for temperature measurements in advanced reactors, such as sodium fast reactors and molten salt cooled reactors. To support the calibration and validation of DTS measurements, Argonne National Laboratory developed the Validation, Optical Calibration, and Learning (VOCAL) software package. This report describes the integration of a local large language model (LLM) with a retrieval-augmented generation (RAG) system into the VOCAL interface to serve as an interactive user assistant. The LLM framework enhances the VOCAL platform’s accessibility to users by explaining interface components, clarifying inputs and outputs, and answering user queries dynamically in real-time. The accuracy of the LLM assistant performance was evaluated with 20 queries regarding the interface and its parameters using experimental data from the Thermal Hydraulic Experimental Test Article (THETA) facility. Results demonstrate that the LLM achieved a 95% accuracy rate, with a BERTScore of 0.8816 and SBERT value of 0.7417. Furthermore, validation of the RAG system within the LLM framework showed optimal accuracy with k-values between 1 and 2 using the k-refinement convergence test. The prompt perturbation analysis demonstrated good initial consistency for the RAG system, exhibiting the highest accuracy under punctuation variations and the greatest sensitivity under query reordering. Notably, the model’s errors were limited to data retrieval failures rather than factual hallucinations, reinforcing its baseline reliability. The integration of LLM provides a highly accurate, userfriendly enhancement to the VOCAL platform without disrupting its core computational capabilities for FOS calibration and validation.

Hong, Evan↗
DOE OSTI2024

Calibration and validation of the foundation for a multiphase strength model for tin

In this work, the Common Model of Multi-phase Strength and Equation of State (CMMP) model was applied to tin. Specifically, calibrations of the strength-specific elements of the CMMP foundation were developed with a combination of experiments and theory, and then the model was validated experimentally. The first element of the foundation is a multi-phase analytic treatment of the melt temperature and the shear modulus for the solid phases. These models were parameterized for each phase based on ab initio calculations using the software VASP (Vienna Ab initio Simulations Package) based on density functional theory. The shear modulus model for the ambient phase was validated with ultrasonic sound speed measurements as a function of pressure and temperature. The second element of the foundation is a viscoplastic strength model for the phase, upon which strength for inaccessible higher-pressure phases can be scaled as necessary. The stress–strain response of tin was measured at strain rates of 10 -3 to 3 x 10 3 s -1 and temperatures ranging from 87 to 373 K. The Preston–Tonks–Wallace (PTW) strength model was fit to that data using Bayesian model calibration. For validation, six forward and two reverse Taylor impact experiments were performed at different velocities to measure large plastic deformation of tin at strain rates up to ⁠10 5 s -1 . The PTW model accurately predicted the deformed shapes of the cylinders, with modest discrepancies attributed to the inability of PTW to capture the effects of twinning and dynamic recrystallization. Some material in the simulations of higher velocity Taylor cylinders reached the melting temperature, thus testing the multiphase model because of the presence of a second phase, the liquid. In simulations using a traditional modeling approach, the abrupt reduction of strength upon melt resulted in poor predictions of the deformed shape and non-physical temperatures. With CMMP, the most deformed material points evolved gradually to a mixed solid–liquid but never a fully liquid state, never fully lost strength, stayed at the melt temperature as the latent heat of fusion was absorbed, and predicted the deformed shape well.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Calibration and validation of the foundation for a multiphase strength model for tin

In this work, the Common Model of Multi-phase Strength and Equation of State (CMMP) model was applied to tin. Specifically, calibrations of the strength-specific elements of the CMMP foundation were developed with a combination of experiments and theory, and then the model was validated experimentally. The first element of the foundation is a 10 multi-phase analytic treatment of the melt temperature and the shear modulus for the solid phases. These models were parameterized for each phase based on ab initio calculations using the software VASP (Vienna Ab initio Simulations Package) based on density functional theory (DFT). The shear modulus model for the ambient β phase was validated with ultrasonic sound speed measurements as a function of pressure and temperature. The second element of the foundation is a viscoplastic strength model for the β phase, upon which strength for inaccessible higher-pressure phases can be scaled as necessary. The stress-strain response of tin was measured at strain rates of 10 -3 to 3 x 10 3 s -1 and temperatures ranging from 87 to 373 K. The Preston-Tonks-Wallace (PTW) strength model was fit to that data using Bayesian model calibration. For validation, six forward and two reverse Taylor impact experiments were performed at different velocities to measure large plastic deformation of tin at strain rates up to 10 5 s -1 . The PTW model accurately predicted the deformed shapes of the cylinders, with modest discrepancies attributed to the inability 20 of PTW to capture the effects of twinning and dynamic recrystallization. Some material in the simulations of higher velocity Taylor cylinders reached the melting temperature, thus testing the multiphase model because of the presence of a second phase, the liquid. In simulations using a traditional modeling approach, the abrupt reduction of strength upon melt resulted in poor predictions of the deformed shape and non-physical temperatures. With CMMP, the most deformed material points evolved gradually to a mixed solid-liquid but never fully liquid state, never fully lost strength, 25 stayed at the melt temperature as the latent heat of fusion was absorbed, and predicted the deformed shape well.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Utilizing 3D Mechanical Earth Models for Calibration and Validation in a Large-Scale Carbon Capture and Storage Project in North Dakota

Conference paper presented at 17th International Conference on Greenhouse Gas Control Technologies (GHGT-17), Calgary, Alberta, Canada, October 20–24, 2024. Three-dimensional (3D) mechanical earth models (MEMs) are pivotal in assessing geological sites for carbon dioxide (CO 2 ) storage and mitigating potential risks associated with storage and injection. The Energy & Environmental Research Center is exploring innovative carbon storage monitoring techniques at a CO 2 storage site near Beulah, North Dakota. These methods aim to provide lower-impact, faster feedback for commercial carbon capture and storage (CCS) projects. The research includes monitoring CO 2 injection using various approaches: 1) an automated, integrated modular monitoring station; 2) time-lapse electromagnetic (EM) field surveys; 3) drone-based surveillance; 4) time-lapse seismic techniques; and 5) advanced wellbore monitoring.

02 PETROLEUM↗
DOE OSTI2022

Riverine Plastic Pollution: Sampling and Analysis Methods

Riverine plastic pollution has been found in all major U.S. rivers, but the exact amount of plastic being released to the oceans has not been quantified. Field studies conducted in U.S. rivers have used a range of sampling and analysis techniques and rarely measured the mass of the plastic collected. Measurements of riverine plastic pollution are needed to calibrate and validate models used to estimate the U.S. riverine plastic emissions to the oceans. This report surveys measurement methods used to quantify riverine pollution and current estimates of U.S. riverine plastic pollution from measurements and models. Measurement methods include field sampling and laboratory analysis. Field sampling methods are described for large (macro) and small (micro) plastic particles. Laboratory analysis methods are described for macro and microplastic with an emphasis on the detailed characterization processes of microplastics. Waterborne leachate analysis is also briefly described. Three models are described that estimate plastic pollution based on mismanaged plastic waste in the river catchment basins. The models were validated and calibrated with global data sources. The data sources were predominantly outside of the U.S., where the magnitude and composition of plastic pollution is different than what is found in U.S. rivers. Comprehensive measurements of riverine plastics are needed not only to characterize the riverine plastic pollution, but also parameterize and validate models of plastic fate and transport. This report also describes five key U.S. rivers that span a range of sizes and environmental conditions that could be sampled to obtain data to support characterization and model development of plastic pollution from rivers to oceans. Sampling and analysis protocol recommendations are made to ensure the highest quality of data are collected in the five rivers.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Validation of Calibrated k–ε Model Parameters for Jet-in-Crossflow

Previous efforts determined a set of calibrated, optimal model parameter values for Reynolds-averaged Navier–Stokes (RANS) simulations of a compressible jet in crossflow (JIC) using a $k–ε$ turbulence model. These parameters were derived by comparing simulation results to particle image velocimetry (PIV) data of a complementary JIC experiment under a limited set of flow conditions. Here, a $k–ε$ model using both nominal and calibrated parameters is validated against PIV data acquired from a much wider variety of JIC cases, including a realistic flight vehicle. The results from the simulations using the calibrated model parameters showed considerable improvements over those using the nominal values, even for cases that were not used in the calibration procedure that defined the optimal parameters. This improvement is demonstrated using a number of quality metrics that test the spatial alignment of the jet core, the magnitudes of multiple flow variables, and the location and strengths of vortices in the counter-rotating vortex cores on the PIV planes. These results suggest that the calibrated parameters have applicability well outside the specific flow case used in defining them and that with the right model parameters, RANS solutions for the JIC can be improved significantly over those obtained from the nominal model.

42 ENGINEERING↗
DOE OSTI2024

Calibration verification for stochastic agent-based disease spread models

Accurate disease spread modeling is crucial for identifying the severity of outbreaks and planning effective mitigation efforts. To be reliable when applied to new outbreaks, model calibration techniques must be robust. However, current methods frequently forgo calibration verification (a stand-alone process evaluating the calibration procedure) and instead use overall model validation (a process comparing calibrated model results to data) to check calibration processes, which may conceal errors in calibration. In this work, we develop a stochastic agent-based disease spread model to act as a testing environment as we test two calibration methods using simulation-based calibration, which is a synthetic data calibration verification method. The first calibration method is a Bayesian inference approach using an empirically-constructed likelihood and Markov chain Monte Carlo (MCMC) sampling, while the second method is a likelihood-free approach using approximate Bayesian computation (ABC). Simulation-based calibration suggests that there are challenges with the empirical likelihood calculation used in the first calibration method in this context. These issues are alleviated in the ABC approach. Despite these challenges, we note that the first calibration method performs well in a synthetic data model validation test similar to those common in disease spread modeling literature. We conclude that stand-alone calibration verification using synthetic data may benefit epidemiological researchers in identifying model calibration challenges that may be difficult to identify with other commonly used model validation techniques.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

A framework for calibrating and validating an air loop dynamic model in an HVAC system in Modelica

The use of Modelica for simulating the dynamic behaviors of building heating, ventilation, and air conditioning (HVAC) systems has gained popularity. Calibration of a model that represents large and complex HVAC systems in Modelica involves the determination of hundreds of parameters using real-world operation data. Considering the coupling effects among various components, such a calibration process is complex and time-consuming. In this study, we propose a systematic framework to efficiently calibrate and validate a complex HVAC system model in Modelica. The framework includes strategies to solicit real-system operation data and to decouple the model. The goal of this calibration framework is to accurately and efficiently determine a set of Modelica model parameters that provide a good match between the simulated and real system behaviors. To demonstrate the validity of the framework, it was applied to calibrate the parameters of a Modelica model of the air loop subsystem of a real AHU-VAV system. The results show that the calibrated model can generate simulated results, including VAV air flow rate, outdoor air flow, and fan power consumption, that match well with the operational data of the real system. The coefficient of the variation of the root mean square error, CV(RMSE), of the air flow rate and power consumption are 14.6 % and 11.5 %, respectively. The results prove that the framework is valid and effective, and it can be used to calibrate other complex HVAC system Modelica models in the future.

Li, Yicheng↗
DOE OSTI2024

Simulation of Particulate Transport for Delivery of Solid Amendments into the Subsurface: FY24 Status Report

For particulate-based amendments to be viable for field-scale remediation at the Hanford Site (e.g., 200 DV-1 Operable Unit), particles need to be delivered a sufficient radial distance from an injection well and retained at concentrations high enough for effective treatment. An accurate description of the particle radius of influence (ROI) is critical for developing an overall remediation strategy. However, field-scale particle simulations are currently limited due to insufficient simulation capabilities and a lack of experimental data to validate and parameterize particle transport models. To help build toward field-scale deployment, this fiscal year (FY) we have (1) developed a pre screening tool to estimate particle transport, (2) implemented particle transport models within PFLOTRAN, and (3) conducted preliminary estimations of particle ROI. While field-scale numerical simulations will ultimately be necessary before remedy design and field implementation, we have developed a pre-screening tool that offers valuable estimations of expected particle injectability and ROI in a 1-D system. The advantage of the tool is that it does not require extensive laboratory experiments and instead makes predictions based solely on routine laboratory measurements. This tool can assist in down-selection and decision-making by identifying which particle amendment systems are worth pursuing in future laboratory experiments, such as 1-D column tests and beyond. With any system, scaling up from the lab to the field presents challenges. Currently, there is no field data available for model calibration or validation. However, the theoretical particle models being developed herein are the best tools available to guide progress toward field deployment. To help bridge this gap and verify model predictions, larger-scale lab experiments are being proposed. To advance simulation capabilities, six particle transport models are being integrated into the reactive transport simulator PFLOTRAN. These include colloid filtration theory (CFT) and five additional particle transport models (M1-M5). Each model, from M1 to M5, progressively incorporates additional particle transport and retention processes. Ultimately, the simplest model capable of accurately describing 1-D column data will be selected and parameterized. During FY24, the CFT and M1 model have been fully implemented within PFLOTRAN. Using an existing 1 D column experiment, the two currently implemented particle transport models (CFT and M1), and associated parameters, were fit to this experiment. While simpler model formulations are helpful for estimations, these formulations could not fully describe particle transport and retention behavior in the previous 1-D column experiment. Thus, additional complexities will need to be considered, which will be accounted for in the M2-M5 model formulations. Additionally, because a viscous, shear thinning fluid was required to keep particles in suspension, considerations for flow will also need to also be accounted for. Therefore, a new immiscible two-phase flow mode is currently being implemented in PFLOTRAN. With some modifications, this new flow module could also support simulation of non-Newtonian liquid amendments, foams, and emulsions. We also estimated the expected ROI of solid amendments using 1-D simulations. The average predicted ROI was approximately 15 ft for micron-sized zero valent iron (mZVI) suspended in xanthan gum (XG). Using the pre screening tool and ROI estimates, additional amendment-delivery laboratory characterization and experiments are proposed. The results from additional experiments can be used to validate and parametrize particulate transport model formulations, which will ultimately provide predictive capabilities for field amendment-delivery systems.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

Validation and Calibration of Energy Models with Real Vehicle Data from Chassis Dynamometer Experiments

Accurate estimation of vehicle fuel consumption typically requires detailed modeling of complex internal powertrain dynamics, often resulting in computationally intensive simulations. However, many transportation applications-such as traffic flow modeling, optimization, and control-require simplified models that are fast, interpretable, and easy to implement, while still maintaining fidelity to physical energy behavior. This work builds upon a recently developed model reduction pipeline that derives physics-like energy models from high-fidelity Autonomie vehicle simulations. These reduced models preserve essential vehicle dynamics, enabling realistic fuel consumption estimation with minimal computational overhead. While the reduced models have demonstrated strong agreement with their Autonomie counterparts, previous validation efforts have been confined to simulation environments. This study extends the validation by comparing the reduced energy model's outputs against real-world vehicle data. Focusing on the MidSUV category, we tune the baseline Autonomie model to closely replicate the characteristics of a Toyota RAV4. We then assess the accuracy of the resulting reduced model in estimating fuel consumption under actual drive conditions. Our findings suggest that, when the reference Autonomie model is properly calibrated, the simplified model produced by the reduction pipeline can provide reliable, semi-principled fuel rate estimates suitable for large-scale transportation applications.

42 ENGINEERING↗
DOE OSTI2026

1988 Delaware Valley Regional Planning Commission Household Travel Survey

The Delaware Valley Regional Planning Commission conducted household travel surveys in 1988 for the Delaware Valley region of Pennsylvania, which consists of Bucks, Chester, Delaware, Montgomery, and Philadelphia counties. The home interview travel survey was intended to measure the travel habits of area residents, and it generated needed information on trip generation rates, trip distribution, and model choice. This information was used for model calibration and validation on the trip generation, trip distribution, and model split models. The data includes demographic and socioeconomic details, as well as travel patterns, for 1,567 households. A total of 5,733 people reported a total of 18,899 trips. The survey was conducted on a weekday, Monday through Friday, and only ten survey locations contained weekend day travel.

1Hz data↗