Search NASA⌕ Search

SEARCH · Search NASA

Results for “Surrogate Model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Calibrating hypersonic turbulence flow models with the HIFiRE-1 experiment using data-driven machine-learned models.

In this paper we study the efficacy of combining machine-learning methods with projection-based model reduction techniques for creating data-driven surrogate models of computationally expensive, high-fidelity physics models. Such surrogate models are essential for many-query applications e.g., engineering design optimization and parameter estimation, where it is necessary to invoke the high-fidelity model sequentially, many times. Surrogate models are usually constructed for individual scalar quantities. However there are scenarios where a spatially varying field needs to be modeled as a function of the model’s input parameters. Here we develop a method to do so, using projections to represent spatial variability while a machine-learned model captures the dependence of the model’s response on the inputs. The method is demonstrated on modeling the heat flux and pressure on the surface of the HIFiRE-1 geometry in a Mach 7.16 turbulent flow. The surrogate model is then used to perform Bayesian estimation of freestream conditions and parameters of the SST (Shear Stress Transport) turbulence model embedded in the high-fidelity (Reynolds-Averaged Navier–Stokes) flow simulator, using shock-tunnel data. The paper provides the first-ever Bayesian calibration of a turbulence model for complex hypersonic turbulent flows. We find that the primary issues in estimating the SST model parameters are the limited information content of the heat flux and pressure measurements and the large model-form error encountered in a certain part of the flow.

42 ENGINEERING↗
DOE OSTI2023

Conditional Karhunen–Loève regression model with Basis Adaptation for high-dimensional problems: Uncertainty quantification and inverse modeling

Here, we propose a methodology for improving the accuracy of surrogate models of the observable response of physical systems as a function of the systems’ spatially heterogeneous parameter fields, with applications to uncertainty quantification and parameter estimation in high-dimensional problems. Practitioners often formulate finite-dimensional representations of spatially heterogeneous parameter fields using truncated unconditional Karhunen–Loève expansions (KLEs) for a certain choice of unconditional covariance kernel and construct surrogate models of the observable response with respect to the KLE coefficients. When direct measurements of the parameter fields are available, we propose improving the accuracy of these surrogate models by representing the parameter fields via conditional Karhunen-Loève expansions (CKLEs). CKLEs are constructed by conditioning the covariance kernel of the unconditional expansion on the direct measurements of the parameter field via Gaussian process regression, and then truncating the corresponding KLE. We apply the proposed methodology to constructing surrogate models via the Basis Adaptation (BA) method of the stationary hydraulic head response, measured at spatially discrete observation locations, of a groundwater flow model of the Hanford Site, as a function of the 1000-dimensional representation of the model’s log-transmissivity field. We find that BA surrogate models of the hydraulic head based on CKLEs are more accurate than BA surrogate models based on unconditional expansions for forward uncertainty quantification tasks. Furthermore, we find that inverse estimates of the hydraulic transmissivity field computed using CKLE-based BA surrogate models are more accurate than those computed using unconditional BA surrogate models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Machine learning for surrogate process models of bioproduction pathways

Technoeconomic analysis and life-cycle assessment are critical to guiding and prioritizing bench-scale experiments and to evaluating economic and environmental performance of biofuel or biochemical production processes at scale. Traditionally, commercial process simulation tools have been used to develop detailed models for these purposes. However, developing and running such models can be costly and computationally intensive, which limits the degree to which they can be shared and reproduced in the broader research community. This study evaluates the potential of an automated machine learning approach to develop surrogate models based on conventional process simulation models. The analysis focuses on several high-value biofuels and bioproducts for which pathways of production from biomass feedstocks have been well-established. The results demonstrate that surrogate models can be an accurate and effective tool for approximating the cost, mass and energy balance outputs of more complex process simulations at a fraction of the computational expense.

09 BIOMASS FUELS↗
DOE OSTI2022

Application of machine learning and artificial intelligence to extend EFIT equilibrium reconstruction

Recent progress in the application of machine learning (ML)/artificial intelligence (AI) algorithms to improve the Equilibrium Fitting (EFIT) code equilibrium reconstruction for fusion data analysis applications is presented. A device-independent portable core equilibrium solver capable of computing or reconstructing equilibrium for different tokamaks has been created to facilitate adaptation of ML/AI algorithms. A large EFIT database comprising of DIII-D magnetic, motional Stark effect, and kinetic reconstruction data has been generated for developments of EFIT model-order-reduction (MOR) surrogate models to reconstruct approximate equilibrium solutions. Furthermore, a neural-network MOR surrogate model has been successfully trained and tested using the magnetically reconstructed datasets with encouraging results. Other progress includes developments of a Gaussian process Bayesian framework that can adapt its many hyperparameters to improve processing of experimental input data and a 3D perturbed equilibrium database from toroidal full magnetohydrodynamic linear response modeling using the Magnetohydrodynamic Resistive Spectrum - Feedback (MARS-F) code for developments of 3D-MOR surrogate models.

Gaussian process↗
DOE OSTI2026

JetGP: A derivative enhanced Gaussian process library

Derivative enhanced Gaussian Processes (DEGPs) can significantly improve surrogate model accuracy over standard Gaussian Process (GP) formulations by incorporating derivative information. However, standard implementations scale poorly with dimension, limiting their use in high dimensional engineering problems. JetGP is a Python framework that unifies existing derivative enhanced GP methodologies into a single library and extends them to support arbitrary order derivative information. The library implements four complementary formulations: standard derivative enhanced Gaussian Processes (DEGP), directional DEGP (DDEGP), generalized directional DEGP (GDDEGP), and weighted DEGP (WDEGP). By unifying these approaches in a consistent interface with robust numerical implementations, JetGP enables practitioners to balance predictive accuracy and computational efficiency for high dimensional optimization, uncertainty quantification, and sensitivity analysis in engineering design.

Derivative enhanced Gaussian process↗
DOE OSTI2024

Non-smooth Bayesian optimization in tuning scientific applications

Tuning algorithmic parameters to optimize the performance of large, complicated computational codes is an important problem involving finding the optima and identifying regimes defined by non-smooth boundaries in black-box functions. Within the Bayesian optimization framework, the Gaussian process surrogate model produces smooth mean functions, but functions in the tuning problem are often non-smooth, which is exacerbated by the fact that we usually have limited sequential samples from the black-box function. Here, motivated by these issues encountered in tuning, we propose a novel Gaussian process model called a clustered Gaussian process (cGP), where the components are dynamically updated by clustering. In our studies, the performance of cGP can be better than stationary GPs in nearly 90% of the experiments and better than non-stationary GPs in nearly 70% of the repeated experiments while requiring less computational cost. cGP provides a novel approach for dynamic GP, computes more efficiently than recursive partitioning, and discovers non-smoothness regimes. We provide extensive experiments including high-performance computing (HPC) and industrial simulation functions to show the effectiveness of our methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Machine Learning for First Principles Calculations of Material Properties for Ferromagnetic Materials

The investigation of finite temperature properties using Monte-Carlo (MC) methods requires a large number of evaluations of the system’s Hamiltonian to sample the phase space needed to obtain physical observables as function of temperature. DFT calculations can provide accurate evaluations of the energies, but they are too computationally expensive for routine simulations. To circumvent this problem, machine-learning (ML) based surrogate models have been developed and implemented on high-performance computing (HPC) architectures. In this paper, we describe two ML methods (linear mixing model and HydraGNN) as surrogates for first principles density functional theory (DFT) calculations with classical MC simulations. These two surrogate models are used to learn the dependence of target physical properties from complex compositions and interactions of their constituents. We present the predictive performance of these two surrogate models with respect to their complexity while avoiding the danger of overfitting the model. An important aspect of our approach is the periodic retraining with newly generated first principles data based on the progressive exploration of the system’s phase space by the MC simulation. The numerical results show that HydraGNN model attains superior predictive performance compared to the linear mixing model for magnetic alloy materials.

Eisenbach, Markus↗
DOE OSTI2025

Deep Learning for Subsurface Flow: A Comparative Study of U‐Net, Fourier Neural Operators, and Transformers in Underground Hydrogen Storage

Subsurface flow research is essential for the sustainable management of natural resources and the environment. Deep learning (DL) has significantly advanced this field by developing efficient and accurate surrogate models to replace computationally expensive physics‐based simulations. These surrogate models are commonly used to predict the spatiotemporal evolution of state variables, such as gas saturation and reservoir pressure, in heterogeneous geological formations. Despite the various DL models applied to this task, there is a lack of studies systematically comparing their performance. This absence of comparative analysis leads to somewhat arbitrary DL model selection in subsurface flow research, resulting in suboptimal performance and potentially inaccurate predictions. To bridge this gap, we conduct a systematic comparison study of three popular DL architectures—U‐Net, Fourier Neural Operators (FNO), and Segmentation Transformer (SETR)—in surrogate modeling of underground hydrogen storage (UHS). We focus on UHS due to its promise of enhancing clean energy resilience and its cyclic operational conditions that represent common scenarios in various subsurface applications. We evaluate the models based on accuracy, training cost, and inference speed. The comparison shows that U‐Net achieves the highest accuracy, followed by SETR and FNO. Despite its lower accuracy, FNO has the highest inference speed. SETR offers competitive accuracy with the least training memory usage, demonstrating the potential of transformers in learning subsurface flow. Our results provide guidance for selecting DL models for surrogate modeling in a wide range of subsurface flow problems.

42 ENGINEERING↗
DOE OSTI2020

UQpy: A general purpose Python package and development environment for uncertainty quantification

In this paper, we present the UQpy software toolbox, an open-source Python package for general uncertainty quantification (UQ) in mathematical and physical systems. The software serves as both a user-ready toolbox that includes many of the latest methods for UQ in computational modeling and a convenient development environment for Python programmers advancing the field of UQ. The paper presents an introduction to the software's architecture and existing capabilities, divided in the code in a set of modules centered around different UQ tasks such as sampling methods, generation of random processes and random fields, probabilistic inverse modeling, reliability analysis, surrogate modeling, and active learning. The paper also highlights the importance of the RunModel module, which is used to drive simulations in the uncertainty analyses performed in UQpy. This module conveniently allows the user to define computational models directly in Python, or to run simulations from a third-party software in serial or in parallel. To illustrate the various capabilities, two examples are tracked throughout the paper and analyzed repeatedly for various UQ tasks. The first is a Python model solving a nonlinear structural dynamics problem, used to illustrate UQpy's capabilities in sampling and forward propagation of high dimensional random vectors (stochastic processes), and probabilistic inference. The second model is a third-party Abaqus finite element model solving the thermomechanical response of a beam structure. This example is used to illustrate UQpy's capabilities in variance reduction sampling techniques, reliability analysis, surrogate modeling and active learning techniques.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Physics-Informed Machine Learning Model for Ceramic Matrix Composite Creep

A physics-informed recurrent neural network (RNN) based surrogate model is developed to emulate the nonlinear, time-dependent constitutive behavior of ceramic matrix composites (CMCs) driven by matrix damage and constituent creep at the microscale. Physics-informed constraints are introduced into the surrogate model through regularization to ground the prediction in physics and improve its predictive capabilities. Training data is generated using the high-fidelity generalized method of cells (HFGMC) approach which calls appropriate creep and damage models for each of the constituents. This coupling permits simulating the nonlinear behavior of CMCs based on constituent response at the microscale along with microstructural features such as fiber and porosity volume fraction and fiber radius. The microscale repeating unit cell is loaded under creep fatigue conditions to replicate the material loading experienced in a turbine engine. Therefore, the RNN-based surrogate model is tasked with predicting, as a function of variable input stress sequence, temperature, and microstructural features, the resulting strain history response while satisfying physical constraints related to creep rate, isochoric inelastic deformation, and strain energy density. The trained surrogate model is shown to effectively match the strain history over quantified distributions of microstructural features and relevant loading regimes and temperatures. Neural network based surrogate models can offer efficient alternatives to running computationally intensive multiscale material models to simulate the nonlinear response of large structural models. Therefore, the presented work provides evidence towards the feasibility of developing, training, and running such models for CMCs with complex microstructures, nonlinear time-dependent material response, and under non-monotonic loading conditions.

ceramic matrix composites↗
DOE OSTI2020

ROM-Based Surrogate Systems Modeling of EBR-II

We report the System Analysis Module (SAM), developed and maintained by Argonne National Laboratory, is designed to provide whole-plant transient safety analysis capabilities for a number of advanced non-light water reactors, including sodium-cooled fast reactor (SFR), lead-cooled fast reactor (LFR), and molten salt reactor (MSR)/fluoride-salt-cooled high-temperature reactor (FHR) designs. SAM is primarily constructed as a systems-level analysis tool, with the potential to incorporate reduced order models from three-dimensional computational fluid dynamics (CFD) simulations to improve characterization of complex, multidimensional physics. It is recognized that the computational expense associated with CFD can be intractable for various engineering analyses, such as uncertainty quantification, inference, and design optimization. This paper explores the reducibility of a SAM model using recent advances in randomized linear algebra techniques, which attempt to find recurring patterns in the various realizations generated by a model after randomly perturbing all its input parameters. The reduction is described in terms of fewer degrees of freedom (DOFs), referred to as the active DOFs, for the model variables such as input model parameters and model responses. The results indicate that there is significant room for additional reduction that may be leveraged for additional computational gains when employing SAM for engineering-intensive analyses that require repeated model executions. Different from physics-based reduction approaches, the proposed approach allows one to estimate upper bounds on the reduction errors, which are rigorously developed in this work. Finally, different methods for surrogate model construction, such as regression and neural network-based training, are employed to correlate the input and output active DOFs, which are related back to the original variables using matrix-based linear transformations.

42 ENGINEERING↗
DOE OSTI2023

Towards fast, accurate predictions of RF simulations via data-driven modeling: Forward and lateral models

Three machine learning techniques (multilayer perceptron, random forest, and Gaussian process) provide fast surrogate models for lower hybrid current drive (LHCD) simulations. A single GENRAY/CQL3D simulation without radial diffusion of fast electrons requires several minutes of wall-clock time to complete, which is acceptable for many purposes, but too slow for integrated modeling and real-time control applications. More accurate simulations with fast electron diffusion are even slower, requiring multiple hours of run time with parallel processing. The machine learning models use a database of 16,000+ GEN-RAY/CQL3D simulations for training, validation, and testing. Latin hypercube sampling methods implemented in πScope ensure that the database covers the range of 9 input parameters (n e0 , T e0 , I p , B t , R 0 , n ∥︀ , Z e f f , V loop , P LHCD ) with sufficient density in all regions of parameter space. The surrogate models reduce the computation time from minutes-hours to ms with high accuracy across the input parameter space. Data-driven surrogate models also allow for solving inverse and “lateral” problems. A surrogate model for the inverse problem maps from a desired current drive or power deposition profile to a set of input parameters that would result in such a profile, while a surrogate model for the lateral problem maps from a measured experimental quantity such as hard x-ray emission to a current drive or power deposition profile. In conclusion, the πScope database creation workflow is flexible and applicable to other RF simulation codes such as TORIC.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Methods for Computing Physically Realistic Estimates of Electric Water Heater Demand Response Resource Suitable for Bulk Power System Planning Models

Demand response is commonly called on to reduce load during system peak times or to respond to contingency events. In future power systems with higher shares of wind and solar generation (which we describe together as variable generation [VG]), demand response could have more opportunities to provide energy shifting or operating reserve services. This report evaluates the ability of residential electric water heaters, both electric resistance water heaters (ERWHs) and heat pump water heaters (HPWHs), to provide such services starting from detailed whole-building energy models that realistically represent New England single family home stock. We use a parsimonious surrogate model to represent operational flexibility in a form suitable for linear and mixed integer programming. This enables relatively fast determination of aggregate contingency reserve resource, price-taking energy shifting outcomes, and in some cases the determination of aggregate models at the megawatt (MW) scale that can be directly included in large-scale grid models. After selecting modeling methods and parameters through various computational experiments, we find interquartile ranges of contingency reserve resource in ISO-NE for about 603,400 ERWHs of 45 MW - 69 MW for Claim10 (50 minute responses provided with 10 minutes of advanced notification) and 65 MW - 102 MW for Claim30 (30 minute responses provided with 30 minutes of advanced notification), and for about 619,000 HPWHs of 48 MW - 88 MW for Claim10 and 52 MW - 90 MW for Claim30. The overall reserve resource is up to 32% of total load for ERWHs providing Claim10 service, 47% for ERWHs providing Claim30 service, 93% for HPWHs providing Claim10 service, and 97% for HPWHs providing Claim30 service. More work is required to determine if HPWHs are inherently more suitable than ERWHs for providing contingency reserve or if these results reflect idiosyncrasies of the single family home stock model used in this study. The value of this contingency resource in a Near-term VG model of ISO-NE is $\$ 0.40$ to $\$1.20$ per water heater-year, and significantly larger, $\$ 3.80$ to $\$ 5.30$ per water heater-year in a Mid-term VG model of ISONE. Aggregating surrogate models to the MW-scale for energy shifting service is more challenging than for contingency service and we only present such results for ERWHs, because we were unable to determine satisfactory ways to deal with HPWHs' time-varying and path dependent operational characteristics. Individual surrogate models suitable for evaluating the energy shifting resource from both ERWHs and HPWHs are created, however, and dispatched against day-ahead prices from the Near-Term VG and Mid-Term VG models of ISO-NE. The individual surrogate models are able to access and potentially shift all 640 GWh of HPWH load and 1,547 GWh of ERWH load we modeled in two different single family home stock models. In contrast, the most effective model of aggregate ERWH shifting resource we created only captured 34.7% of the total ERWH load. Energy shifting affected by price-taking dispatch against modeled day-ahead energy prices produces per water heater year profits of $\$19.44$ - $\$22.93$ for individual HPWHs, $\$39.11$ - $\$40.54$ for individual ERWHs, and up to $\$4.00$ - $\$4.24$ for aggregated ERWHs, with the variations mainly due to grid conditions (more or less VG). When the supply-side response to these changes is accounted for, the per water heater year production cost savings for ISO-NE are $\$7.50$ to $\$17.70$ for the most effective set of endogenously dispatched aggregate ERWHs, $\$15.60$ to $\$15.70$ for individual ERWHs dispatched against the DA prices, and $\$10.70$ to $\$11.20$ for individual HPWHs dispatched against DA prices. Those ranges primarily represent the difference between Near-Term VG and Mid-Term VG grid conditions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Data-driven model for divertor plasma detachment prediction

We present a fast and accurate data-driven surrogate model for divertor plasma detachment prediction leveraging the latent feature space concept in machine learning research. Our approach involves constructing and training two neural networks: an autoencoder that finds a proper latent space representation (LSR) of plasma state by compressing the multi-modal diagnostic measurements and a forward model using multi-layer perception (MLP) that projects a set of plasma control parameters to its corresponding LSR. By combining the forward model and the decoder network from autoencoder, this new data-driven surrogate model is able to predict a consistent set of diagnostic measurements based on a few plasma control parameters. In order to ensure that the crucial detachment physics is correctly captured, highly efficient 1D UEDGE model is used to generate training and validation data in this study. The benchmark between the data-driven surrogate model and UEDGE simulations shows that our surrogate model is capable of providing accurate detachment prediction (usually within a few per cent relative error margin) but with at least four orders of magnitude speed-up, indicating that performance-wise, it has the potential to facilitate integrated tokamak design and plasma control. Comparing with the widely used two-point model and/or two-point model formatting, the new data-driven model features additional detachment front prediction and can be easily extended to incorporate richer physics. This study demonstrates that the complicated divertor and scrape-off-layer plasma state has a low-dimensional representation in latent space. Understanding plasma dynamics in latent space and utilising this knowledge could open a new path for plasma control in magnetic fusion energy research.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Toward a machine-guided approach to energetic material discovery

In this article, we trained a machine learning (ML) model to connect microstructural details of an energetic material formulation to its performance for the purpose of guiding the discovery of new explosive formulations. Our hypothesis was that the algorithm would robustly learn the training data and produce an accurate surrogate model. Specifically, the algorithm learned the relationship between details of the void size distribution (VSD), initiating shock pressure, and the energetic material performance. We used realistic constraints on the VSD and a range of cases were ingested by a physically informed reactive flow model working within a hydrodynamic solver running on high-performance computing resources. The ML algorithm produced a surrogate model that accurately predicted known test points around the parameter space. In addition to the utility of the model and the process used for its development, we noted interesting comparisons between what we, the authors—subject matter experts, would heuristically conclude from the training data and the surrogate model predictions. We detected nuanced details that were missed by the surrogate model; however, these details are not important to an energetic material formulator. We concluded that the algorithm did indeed robustly learn the training data and produce an accurate surrogate model. We further concluded that the surrogate model is a powerful tool to guide the formulator in the absence of subject matter experts and limited-access computing resources.

42 ENGINEERING↗
DOE OSTI2023

ICRF wave propagation and absorption modelling via machine learning

A surrogate model of the wave absorption in the ion cyclotron range of frequencies is presented. The model is trained to capture the physics of 1D electron and ion power absorption profiles for both the high harmonic fast wave scheme in NSTX, and the minority heating scheme in WEST. The surrogate models, based on both the random forest regressor and the multilayer perceptron algorithms, reduce inference time of 1D power absorption profiles from 1-5 minutes required by TORIC to ∼50 µs with high accuracy (i.e. R2 = 0.71−0.96).

Sánchez-Villar↗
DOE OSTI2021

A systematic analysis of phase stability in refractory high entropy alloys utilizing linear and non-linear cluster expansion models

We report that obtaining surrogate models for various types of materials has long been a goal of computational material scientists and physicists, as they allow the calculation of thermodynamic quantities and properties of interest far more efficiently than DFT. Many surrogate models, including interatomic potentials generated by machine learning algorthims and cluster expansions, rely on incorporating many-body interactions such as three-body or four-body interactions. When utilizing machine learning algorithms for interatomic potentials, it is common to start with two-body interactions and systematically include three- and four-body clusters to improve the fit, stopping when including the additional clusters does not measurably change the predictive power of the model. In cluster expansions for FCC and related lattices, it is often enough to include mostly two-body interactions and a handful of three-body interactions. However, it remains to be seen whether those will be enough in the case of lattices with a lower packing fraction, such as BCC.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Tencoder: tensor-product encoder-decoder architecture for predicting solutions of PDEs with variable boundary data

It is widely hoped that artificial intelligence will boost data-driven surrogate models in science and engineering. However, fundamental spatial aspects of AI surrogate models remain under-studied. We investigate the ability of neural-network surrogate models to predict solutions to PDEs under variable boundary values. We do not wish to retrain the model when the boundary values change but to make them inputs to the model and infer the solution of the PDE under those boundary conditions. Such a capability is essential to making AI-based surrogate models practically useful. While simple feedforward networks are used for one-dimensional (1D) Poisson equation, an encoder-decoder architecture with a tensor-product layer is developed for the two-dimensional Poisson equation posed on a rectangular domain. We show that it is indeed possible to infer solutions to PDEs from variable boundary data using neural networks in this relatively simple setting, and point to future directions.

Kashi, Aditya↗