Search NASA⌕ Search

SEARCH · Search NASA

Results for “computer model calibration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Computer Model Calibration with Time Series Data Using Deep Learning and Quantile Regression

Computer models play a key role in many scientific and engineering problems. One major source of uncertainty in computer model experiments is input parameter uncertainty. Computer model calibration is a formal statistical procedure to infer input parameters by combining information from model runs and observational data. The existing standard calibration framework suffers from inferential issues when the model output and observational data are high-dimensional dependent data, such as large time series, due to the difficulty in building an emulator and the nonidentifiability between effects from input parameters and data-model discrepancy. To overcome these challenges, we, in this work, propose a new calibration framework based on a deep neural network (DNN) with long short-term memory layers that directly emulates the inverse relationship between the model output and input parameters. Adopting the “learning with noise” idea, we train our DNN model to filter out the effects from data-model discrepancy on input parameter inference. We also formulate a new way to construct interval predictions for DNN using quantile regression to quantify the uncertainty in input parameter estimates. Through a simulation study and real data application with the Weather Research and Forecasting Model Hydrological modeling system (WRF-Hydro), we show our approach can yield accurate point estimates and well-calibrated interval estimates for input parameters.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Bayesian optimal experimental design for constitutive model calibration

Computational simulation is increasingly relied upon for high/consequence engineering decisions, which necessitates a high confidence in the calibration of and predictions from complex material models. However, the calibration and validation of material models is often a discrete, multi-stage process that is decoupled from material characterization activities, which means the data collected does not always align with the data that is needed. To address this issue, an integrated workflow for delivering an enhanced characterization and calibration procedure—Interlaced Characterization and Calibration (ICC)—is introduced and demonstrated. Further, this framework leverages Bayesian optimal experimental design (BOED), which creates a line of communication between model calibration needs and data collection capabilities in order to optimize the information content gathered from the experiments for model calibration. Eventually, the ICC framework will be used in quasi real-time to actively control experiments of complex specimens for the calibration of a high-fidelity material model. This work presents the critical first piece of algorithm development and a demonstration in determining the optimal load path of a cruciform specimen with simulated data. Calibration results, obtained via Bayesian inference, from the integrated ICC approach are compared to calibrations performed by choosing the load path a priori based on human intuition, as is traditionally done. The calibration results are communicated through parameter uncertainties which are propagated to the model output space (i.e. stress–strain). In these exemplar problems, data generated within the ICC framework resulted in calibrated model parameters with reduced measures of uncertainty compared to the traditional approaches.

42 ENGINEERING↗
DOE OSTI2022

Systems and methods for distributed power system model calibration

A computing device for distributed power system model calibration is provided. The computing device is programmed to receive event data and model response data associated with a model to simulate, wherein the model includes a plurality of parameters, divide the event data into a plurality of sets, wherein each set includes associated parameters, and transmit the plurality of sets of event data to a plurality of client nodes. Each client node of the plurality of client nodes is programmed to analyze a corresponding set of event data to determine updated parameters for the model. The computing device is further programmed to receive a plurality of updated parameters for the model from the plurality of client nodes and analyze the received plurality of updated parameters to determine at least one adjusted parameter.

Wang, Honggang↗
DOE OSTI2022

Analyzing Stochastic Computer Models: A Review with Opportunities

In modern science, computer models are often used to understand complex phenomena and a thriving statistical community has grown around analyzing them. This review aims to bring a spotlight to the growing prevalence of stochastic computer models-providing a catalogue of statistical methods for practitioners, an introductory view for statisticians (whether familiar with deterministic computer models or not), and an emphasis on open questions of relevance to practitioners and statisticians. Gaussian process surrogate models take center stage in this review, and these, along with several extensions needed for stochastic settings, are explained. The basic issues of designing a stochastic computer experiment and calibrating a stochastic computer model are prominent in the discussion. Instructive examples, with data and code, are used to describe the implementation of, and results from, various methods.

agent based model↗
DOE OSTI2022

Bayesian optimization for inverse calibration of expensive computer models: A case study for Johnson-Cook model in machining

Inverse model calibration for identifying the constitutive model parameters can be computationally demanding for expensive-to-evaluate simulation models. Here, this paper presents a modified Bayesian optimization (BO) method, denoted as BO-bound, that incorporates theoretical bounds on the quantity of interest. A case study for the inverse calibration of the Johnson Cook (J-C) flow stress model parameters is presented using machining (cutting) force data. The results show fast calibration of the five J-C parameters within 25 simulations. In general, the BO-bound method is applicable for inverse calibration of any expensive simulation models as well as optimization problems with known bounds.

Bayesian optimization↗
DOE OSTI2022

Flood hazard model calibration using multiresolution model output

Riverine floods pose a considerable risk to many communities. Improving flood hazard projections has the potential to inform the design and implementation of flood risk management strategies. Current flood hazard projections are uncertain, especially due to uncertain model parameters. Calibration methods use observations to quantify model parameter uncertainty. With limited computational resources, researchers typically calibrate models using either relatively few expensive model runs at high spatial resolutions or many cheaper runs at lower spatial resolutions. This leads to an open question: is it possible to effectively combine information from the high and low resolution model runs? We propose a Bayesian emulation–calibration approach that assimilates model outputs and observations at multiple resolutions. As a case study for a riverine community in Pennsylvania, we demonstrate our approach using the LISFLOOD-FP flood hazard model. Here, the multiresolution approach results in improved parameter inference over the single resolution approach in multiple scenarios. Results vary based on the parameter values and the number of available models runs. Our method is general and can be used to calibrate other high dimensional computer models to improve projections.

multiresolution↗
DOE OSTI2021

Comprehensive Material Characterization and Simultaneous Model Calibration for Improved Computational Simulation Credibility

Computational simulation is increasingly relied upon for high-consequence engineering decisions, and a foundational element to solid mechanics simulations is a credible material model. Our ultimate vision is to interlace material characterization and model calibration in a real-time feedback loop, where the current model calibration results will drive the experiment to load regimes that add the most useful information to reduce parameter uncertainty. The current work investigated one key step to this Interlaced Characterization and Calibration (ICC) paradigm, using a finite load-path tree to incorporate history/path dependency of nonlinear material models into a network of surrogate models that replace computationally-expensive finite-element analyses. Our reference simulation was an elastoplastic material point subject to biaxial deformation with a Hill anisotropic yield criterion. Training data was generated using either a space-filling or adaptive sampling method, and surrogates were built using either Gaussian process or polynomial chaos expansion methods. Surrogate error was evaluated to be on the order of 10 ⁻5 and 10 ⁻3 percent for the space-filling and adaptive sampling training data, respectively. Direct Bayesian inference was performed with the surrogate network and with the reference material point simulator, and results agreed to within 3 significant figures for the mean parameter values, with a reduction in computational cost over 5 orders of magnitude. These results bought down risk regarding the surrogate network and facilitated a successful FY22-24 full LDRD proposal to research and develop the complete ICC paradigm.

36 MATERIALS SCIENCE↗
DOE OSTI2021

A comparison of Gaussian processes and neural networks for computer model emulation and calibration

The Department of Energy relies on complex physics simulations for prediction in domains like cosmology, nuclear theory, and materials science. These simulations are often extremely computationally intensive, with some requiring days or weeks for a single simulation. In order to assure their accuracy, these models are calibrated against observational data in order to estimate inputs and systematic biases. Because of their great computational complexity, this process typically requires the construction of an emulator, a fast approximation to the simulation. In this paper, two emulator approaches are compared: Gaussian process regression and neural networks. Their emulation accuracy and calibration performance on three real problems of Department of Energy interest is considered. On these problems, the Gaussian process emulator tends to be more accurate with narrower, but still well-calibrated uncertainty estimates. The neural network emulator is accurate, but tends to have large uncertainty on its predictions. Finally, as a result, calibration with the Gaussian process emulator produces more constrained posteriors that still perform well in prediction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

IBM Open Science Price - SWAP Gate Challenge

On November 30, 2020, IBM announced an open science price competition aimed at improving the fidelity of a SWAP gate on their 7 qubit Quantum system named ”Casablanca”. The details of the SWAP gate problem was outlined in a Jupyter notebook, which utilizes the IBM developed open source package Qiskit. The participants were allowed to work in teams with up to five people and restricted to only use open source software in solving the problem. Their results were due on April 16, 2021. Our approach to solving the SWAP gate challenge is based on the quantum optimal control techniques implemented in our open source packages Juqbox.jl and Quandary. The results from the optimal control approach is only as good as the accuracy in the description of the quantum system dynamics, characterized by a Hamiltonian model that was provided by IBM. Based on the calibrated control pulses IBM provide for the standard gate set, we developed a reverse engineering approach to calibrate our computational model. Techniques were developed to translate between Qiskit’s pulse representation and the B-spline formulation used in Juqbox and Quandary. The fidelity of the optimized pulse sequences were estimated using Qiskit’s randomized bench marking techniques.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Validation Metrics for Fixed Effects and Mixed-Effects Calibration

The modern scientific process often involves the development of a predictive computational model. To improve its accuracy, a computational model can be calibrated to a set of experimental data. A variety of validation metrics can be used to quantify this process. Some of these metrics have direct physical interpretations and a history of use, while others, especially those for probabilistic data, are more difficult to interpret. In this work, a variety of validation metrics are used to quantify the accuracy of different calibration methods. Frequentist and Bayesian perspectives are used with both fixed effects and mixed-effects statistical models. Through a quantitative comparison of the resulting distributions, the most accurate calibration method can be selected. Two examples are included which compare the results of various validation metrics for different calibration methods. It is quantitatively shown that, in the presence of significant laboratory biases, a fixed effects calibration is significantly less accurate than a mixed-effects calibration. This is because the mixed-effects statistical model better characterizes the underlying parameter distributions than the fixed effects model. The results suggest that validation metrics can be used to select the most accurate calibration model for a particular empirical model with corresponding experimental data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Study of wrap mode impact on Pseudomonas aeruginosa motion in the chemotactic field of fungi

An agent-based discrete computational model biologically calibrated to Pseudomonas aeruginosa migration is used to explore the impacts of bacterial reversals and wrap mode on the efficiency of motion in different environments, both with and without chemotaxis. It is first shown that wrap mode increases the exploration of continuous multimodal chemotactic profiles such as those produced by biologically relevant fungal networks. For cells undergoing a run-reverse pattern, it is shown that the bacteria are likely to remain at the first local chemoattractant maximal production site on a hypha they find. However, with wrap mode, the bacteria can more easily escape these local sites to further explore their neighboring environment along the fungi, suggesting that wrap mode may be beneficial for migration along the fungi in liquid. In a different set of simulations of bacterial motion close to an isolated chemotactic source, wrap mode is shown to increase the ability of a bacterium to reorient toward the source while reducing the overall motion required for similar chemotactic efficiency as a run-reverse strategy, suggesting a potential metabolic benefit. In contrast, model simulations show that wrap mode can increase the rate of dispersal of P. aeruginosa in a nonchemotactic environment.

Hansen, Austin [University of California, Riversid↗
DOE OSTI2025

Surrogate models for linear response

Linear response theory is a well-established method in physics and chemistry for exploring excitations of many-body systems. In particular, the quasiparticle random-phase approximation (QRPA) provides a powerful microscopic framework by building excitations on top of the mean-field vacuum; however, its high computational cost limits model calibration and uncertainty quantification studies. Here, we present two complementary QRPA surrogate models and apply them to study response functions of finite nuclei. One is a reduced-order model that exploits the underlying QRPA structure, while the other utilizes the recently developed parametric matrix model algorithm to construct a map between the system’s Hamiltonian and observables. Our benchmark applications, the calculation of the electric dipole polarizability of 180 Yb and the 𝛽-decay half-life of 80 Ni, show that both emulators can achieve 0.1%–1% accuracy while offering a 6–7 orders of magnitude speedup compared to state-of-the-art QRPA solvers. These results demonstrate that the developed QRPA emulators are well positioned to enable Bayesian calibration and large-scale studies of computationally expensive physics models describing the properties of many-body systems.

Beta decay↗
DOE OSTI2022

Goal-oriented a-posteriori estimation of model error as an aid to parameter estimation

In this work, a Bayesian model calibration framework is presented that utilizes goal-oriented a-posterior error estimates in quantities of interest (QoIs) for classes of high-fidelity models characterized by PDEs. It is shown that for a large class of computational models, it is possible to develop a computationally inexpensive procedure for calibrating parameters of high-fidelity models of physical events when the parameters of low-fidelity (surrogate) models are known with acceptable accuracy. The main ingredients in the proposed model calibration scheme are goal-oriented a-posteriori estimates of error in QoIs computed using a so-called lower fidelity model compared to those of an uncalibrated higher fidelity model. The estimates of error in QoIs are used to define likelihood functions in Bayesian inversion analysis. A standard Bayesian approach is employed to compute the posterior distribution of model parameters of high-fidelity models. As applications, parameters in a quasi-linear second-order elliptic boundary-value problem (BVP) are calibrated using a second-order linear elliptic BVP. In a second application, parameters of a tumor growth model involving nonlinear time-dependent PDEs are calibrated using a lower fidelity linear tumor growth model with known parameter values.

A-posterior estimates↗
DOE OSTI2021

Robust Importance Sampling for Bayesian Model Calibration with Spatio-Temporal Data

This paper addresses two challenges in Bayesian calibration: 1) computational speed of existing sampling algorithms, and 2) calibration with spatio-temporal responses. The commonly used Markov Chain Monte Carlo (MCMC) approaches require many sequential model evaluations making the computational expense prohibitive. This paper proposes an efficient sampling algorithm: iterative importance sampling with genetic algorithm (IISGA). While iterative importance sampling enables computational efficiency, the genetic algorithm enables robustness by preventing sample degeneration and avoids getting stuck in multimodal search spaces. An inflated likelihood further enables robustness in high-dimensional parameter spaces by enlarging the target distribution. Spatio-temporal data complicate both surrogate modeling, which is necessary for expensive computational models, and the likelihood estimation. In this work, singular value decomposition is investigated for reducing the high-dimensional field data to a lower-dimensional space prior to Bayesian calibration. Then the likelihood is formulated and Bayesian inference is performed in the lower-dimension, latent space. An illustrative example is provided to demonstrate IISGA relative to existing sampling methods, and then IISGA is employed to calibrate a thermal battery model with 26 uncertain calibration parameters and spatio-temporal response data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Neglecting Model Parametric Uncertainty Can Drastically Underestimate Flood Risks

Abstract Floods drive dynamic and deeply uncertain risks for people and infrastructures. Uncertainty characterization is a crucial step in improving the predictive understanding of multi‐sector dynamics and the design of risk‐management strategies. Current approaches to estimate flood hazards often sample only a relatively small subset of the known unknowns, for example, the uncertainties surrounding the model parameters. This approach neglects the impacts of key uncertainties on hazards and system dynamics. Here we mainstream a recently developed method for Bayesian inference to calibrate a computationally expensive distributed hydrologic model. We compare three different calibration approaches: (a) stepwise line search, (b) precalibration or screening, and (c) the Fast Model Calibrations (FaMoS) approach. FaMoS deploys a particle‐based approach that takes advantage of the massive parallelization afforded by modern high‐performance computing systems. We quantify how neglecting parametric uncertainty and data discrepancy can drastically underestimate extreme flood events and risks. Precalibration improves prediction skill score over a stepwise line search. The Bayesian calibration improves the uncertainty characterization of model parameters and flood risk projections.

54 ENVIRONMENTAL SCIENCES↗