Search NASA⌕ Search

SEARCH · Search NASA

Results for “multi-fidelity optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Multi-Fidelity Active Subspaces for Wind Farm Uncertainty Quantification

Wind plants operate in stochastic environments characterized by complex turbulent flow dynamics and high-dimensional random variables. A key step in uncertainty quantification studies is sensitivity analysis and dimension reduction that can facilitate the development of surrogate models to be used for forward and inverse propagation or optimization under uncertainty. Prior work has shown active subspaces are an effective tool for identifying important directions in the space of stochastic inputs; however, they have only been applied to single-fidelity wind plant models. In this study, we investigate the efficacy of a multi-fidelity active subspace method for analyzing the uncertainty in wind plant power output. The multi-fidelity active subspace estimator offers the promise of increased accuracy in identifying active subspaces as compared to a single-fidelity estimator for the same computational cost, or a reduction in cost for the same accuracy. This makes the study of uncertainty in larger wind plants and with higher fidelity physics tractable. The multi-fidelity active subspace method is applied to gridded and existing wind plant layouts with single and multiple inflow conditions and its performance for surrogate modeling and uncertainty propagation is compared against a single-fidelity active subspace method. This multi-fidelity approach yields substantial computational speedups of 2x - 3.4x across the test cases along with acceptable accuracy in surrogate modeling and computing statistical moments.

active subspace↗
DOE OSTI2025

Adaptive Computing for Scale-Up Problems

Adaptive Computing is an application-agnostic outer loop framework to strategically deploy simulations and experiments to guide decision making for scale-up analysis. Resources are allocated over successive batches, which makes the allocation adaptive to some objective such as optimization or model training. The framework enables the characterization and management of uncertainties associated with predictive models of complex systems when scale-up questions lead to significant model extrapolation. A key advancement of this framework is its integration of multi-fidelity surrogate modeling, uncertainty management, and automated orchestration of various computing and experimentation resources into a single integrated software package. This enables efficient multi-fidelity modeling across multiple computing resources by incorporating real-world constraints such as relative queue times and throughput on individual machines into the multi-fidelity sampling decision. We discuss applications of this framework to problems in the renewable energy space, including biofuels production, material synthesis, perovskite crystal growth, and building electrical loads.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2020

Multi Model Monte Carlo with Python (MXMCPy)

Multi Model Monte Carlo with Python (\mxmc {}) is a software package developed as a general capability for computing the statistics of outputs from an expensive, high-fidelity model by leveraging faster, low-fidelity models for speedup. Motivated by uncertainty propagation problems where classical Monte Carlo (MC) simulation is computationally intractable, various multi-model MC approaches have recently emerged that yield unbiased estimators with significantly reduced variance relative to MC for the same cost. These existing methods include multi-level Monte Carlo (MLMC), multi-fidelity Monte Carlo (MFMC), and approximate control variates (ACV). Given a fixed computational budget and a collection of models with varying cost/accuracy, each method seeks a sample allocation strategy across the models that results in an estimator with optimal variance reduction. \mxmc {} is a versatile tool that enables convenient access to many existing multi-model MC approaches within one modular and extensible package. With \mxmc {}, users can easily compare existing methods to determine the best choice for their particular problem, while developers have a basis for implementing and sharing new variance reduction approaches. This report introduces the \mxmc {} software, providing a summary of the problem-solving workflow for users as well as a brief overview of the code layout for developers.

Geoffrey F Bomarito↗
DOE OSTI2021

Surrogate multi-fidelity data and model fusion for scientific discovery and uncertainty quantification in Earth System Models

This whitepaper addresses the Earth and Environmental Systems Sciences Division (EESSD)’s predictability challenges in modeling the integrated water cycle and data-model integration. Specifically, it focuses on reducing and characterizing the uncertainty in the representation of process models for unresolved physics, either due to model resolution or limited by the physical under standing or computational efficiency, and the use of observational data for in-situ process parameter optimization within ESM. The described methods may also be used to determine the nature of responses (e.g. strength and direction), and hence to identify critical processes that drive the overall ESM responses to perturbation in the forcing

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements

Physics-informed neural networks (PINNs) were recently proposed in [18] as an alternative way to solve partial differential equations (PDEs). A neural network (NN) represents the solution, while a PDE-induced NN is coupled to the solution NN, and all differential operators are treated using automatic differentiation. Here, we first employ the standard PINN and a stochastic version, sPINN, to solve forward and inverse problems governed by a non-linear advection–diffusion–reaction (ADR) equation, assuming we have some sparse measurements of the concentration field at random or pre-selected locations. Subsequently, we attempt to optimise the hyper-parameters of sPINN by using the Bayesian optimisation method (meta-learning) and compare the results with the empirically selected hyper-parameters of sPINN. In particular, for the first part in solving the inverse deterministic ADR, we assume that we only have a few high-fidelity measurements, whereas the rest of the data is of lower fidelity. Hence, the PINN is trained using a composite multi-fidelity network, first introduced in [12], that learns the correlations between the multi-fidelity data and predicts the unknown values of diffusivity, transport velocity and two reaction constants as well as the concentration field. For the stochastic ADR, we employ a Karhunen–Loève (KL) expansion to represent the stochastic diffusivity, and arbitrary polynomial chaos (aPC) to represent the stochastic solution. Correspondingly, we design multiple NNs to represent the mean of the solution and learn each aPC mode separately, whereas we employ a separate NN to represent the mean of diffusivity and another NN to learn all modes of the KL expansion. For the inverse problem, in addition to stochastic diffusivity and concentration fields, we also aim to obtain the (unknown) deterministic values of transport velocity and reaction constants. The available data correspond to 7spatial points for the diffusivity and 20 space–time points for the solution, both sampled 2000 times. We obtain good accuracy for the deterministic parameters of the order of 1–2% and excellent accuracy for the mean and variance of the stochastic fields, better than three digits of accuracy. In the second part, we consider the previous stochastic inverse problem, and we use Bayesian optimisation to find five hyper-parameters of sPINN, namely the width, depth and learning rate of two NNs for learning the modes. Here, we obtain much deeper and wider optimal NNs compared to the manual tuning, leading to even better accuracy, i.e., errors less than 1% for the deterministic values, and about an order of magnitude less for the stochastic fields.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Simulations of edge and SOL turbulence in diverted negative and positive triangularity plasmas

Optimizing the performance of magnetic confinement fusion devices is critical to achieving an attractive fusion reactor design. Negative triangularity (NT) scenarios have been shown to achieve excellent levels of energy confinement, while avoiding edge localized modes. Modeling turbulent transport in the edge and SOL is key in understanding the impact of NT on turbulence and extrapolating the results to future devices and regimes. Previous gyrokinetic turbulence studies have reported beneficial effects of NT across a broad range of parameters. However, most simulations have focused on the inner plasma region, neglecting the impact of NT on the outermost edge. In this work, we investigate the effect of NT in edge and scrape-off layer simulations, including the magnetic X-point and separatrix. For the first time, we employ a multi-fidelity approach, combining global, non-linear gyrokinetic simulations with drift-reduced fluid simulations, to gain a deeper understanding of the underlying physics at play. First-principles simulations using the GENE-X code demonstrate that in comparable NT and PT geometries, similar profiles are achieved, while the turbulent heat flux is reduced by more than 50% in NT. Comparisons with results from the drift-reduced fluid turbulence code GRILLIX suggest that the turbulence is driven by trapped electron modes. The parallel heat flux width on the divertor targets is reduced in NT, primarily due to a lower spreading factor S.

GENE-X↗
DOE OSTI2021

Multi-fidelity modeling to predict the rheological properties of fiber suspensions

Unveiling the rheological properties of fiber suspensions is of paramount interest to many industrial applications like biofuel production. The 3D numerical simulations of the suspension of fibers are often computationally expensive and time-consuming. Machine learning methods such as neural networks can simplify the prediction of rheological behavior; however, they require a relatively large training data set. Multi -fidelity models, which combine high-fidelity data from numerical simulations and less expensive lower fidelity data from resources such as simplified physical equations, can lead to optimized predictions. Here, we focus on a neural network with two levels of fidelity, i.e., high and low fidelity networks. To produce high-fidelity data, we perform direct numerical simulations to model the fibers as one-dimensional inextensible slender bodies that obey the Euler- Bernoulli beam equation. The Navier-Stokes equations govern the suspended fluid, and an immersed boundary method is used to couple the fluid and solid motion. The low-fidelity data is produced by using constitutive equations. Noticeable improvements have been observed in the accuracy of predicting the rheological behavior when a multi-fidelity network is used compared to the single-fidelity network.

Boodaghidizaji, Miad↗
DOE OSTI2020

NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations

In the last 50 years there has been a tremendous progress in solving numerically the Navier-Stokes equations using finite differences, finite elements, spectral, and even meshless methods. Yet, in many real cases, we still cannot incorporate seamlessly (multi-fidelity) data into existing algorithms, and for industrial-complexity applications the mesh generation is time consuming and still an art. Moreover, solving ill-posed problems (e.g., lacking boundary conditions) or inverse problems is often prohibitively expensive and requires different formulations and new computer codes. Here, we employ physics-informed neural networks (PINNs), encoding the governing equations directly into the deep neural network via automatic differentiation, to overcome some of the aforementioned limitations for simulating incompressible laminar and turbulent flows. We develop the Navier-Stokes flow nets (NSFnets) by considering two different mathematical formulations of the Navier-Stokes equations: the velocity-pressure (VP) formulation and the vorticity-velocity (VV) formulation. Since this is a new approach, we first select some standard benchmark problems to assess the accuracy, convergence rate, computational cost and flexibility of NSFnets; analytical solutions and direct numerical simulation (DNS) databases provide proper initial and boundary conditions for the NSFnet simulations. The spatial and temporal coordinates are the inputs of the NSFnets, while the instantaneous velocity and pressure fields are the outputs for the VP-NSFnet, and the instantaneous velocity and vorticity fields are the outputs for the VV-NSFnet. This is unsupervised learning and, hence, no labeled data are required beyond boundary and initial conditions and the fluid properties. The residuals of the VP or VV governing equations, together with the initial and boundary conditions, are embedded into the loss function of the NSFnets. No data is provided for the pressure to the VP-NSFnet, which is a hidden state and is obtained via the incompressibility constraint without extra computational cost. Unlike the traditional numerical methods, NSFnets inherit the properties of neural networks (NNs), hence the total error is composed of the approximation, the optimization, and the generalization errors. Here, we empirically attempt to quantify these errors by varying the sampling (“residual”) points, the iterative solvers, and the size of the NN architecture. For the laminar flow solutions, we show that both the VP and the VV formulations are comparable in accuracy but their best performance corresponds to different NN architectures. The initial convergence rate is fast but the error eventually saturates to a plateau due to the dominance of the optimization error. For the turbulent channel flow, we show that NSFnets can sustain turbulence at , but due to expensive training we only consider part of the channel domain and enforce velocity boundary conditions on the subdomain boundaries provided by the DNS data base. We also perform a systematic study on the weights used in the loss function for balancing the data and physics components, and investigate a new way of computing the weights dynamically to accelerate training and enhance accuracy. In the last part, we demonstrate how NSFnets should be used in practice, namely for ill-posed problems with incomplete or noisy boundary conditions as well as for inverse problems. We obtain reasonably accurate solutions for such cases as well without the need to change the NSFnets and at the same computational cost as in the forward well-posed problems. As a result, we also present a simple example of transfer learning that will aid in accelerating the training of NSFnets for different parameter settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Inverse design of photonic surfaces via multi fidelity ensemble framework and femtosecond laser processing

We demonstrate a multi-fidelity (MF) machine learning ensemble framework for the inverse design of photonic surfaces, trained on a dataset of 11,759 samples that we fabricate using high throughput femtosecond laser processing. The MF ensemble combines an initial low fidelity model for generating design solutions, with a high fidelity model that refines these solutions through local optimization. The combined MF ensemble can generate multiple disparate sets of laser-processing parameters that can each produce the same target input spectral emissivity with high accuracy (root mean squared errors < 2%). SHapley Additive exPlanations analysis shows transparent model interpretability of the complex relationship between laser parameters and spectral emissivity. Finally, the MF ensemble is experimentally validated by fabricating and evaluating photonic surface designs that it generates for improved efficiency energy harvesting devices. Our approach provides a powerful tool for advancing the inverse design of photonic surfaces in energy harvesting applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

HydraGNN_Predictive_GFM_2026 - Ensemble of predictive graph foundation models for atomistic materials modeling

This release contains data and parameters of HydraGNN-based graph foundation models trained as a result of the work published in the pre-print "Exascale Multi-Task Graph Foundation Models for Imbalanced, Multi-Fidelity Atomistic Data" by M. Lupo Pasini et al. (https://arxiv.org/abs/2604.15380). We jointly train on 16 open first-principles datasets (544+ million structures covering 85+ elements) using a multi-task architecture with per-dataset heads and a scalable ADIOS2/DDStore data pipeline. On Frontier, we execute six large-scale DeepHyper hyperparameter optimization campaigns in FP64 and promote the top-performing message-passing models to sustained 2,048-node training, yielding a PaiNN-based lead model. The version of HydraGNN used to generate the outputs provided in this release is HydraGNN v5.0 (https://github.com/ORNL/HydraGNN/releases/tag/v5.0) The list of datasets used for the training of the graph foundation model is the following: 1) Alexandria [1] 2) ANI1x [2] 3) MPTrj [3] 4) Open Catalyst 2020 (OC20) [4] 5) Open Catalyst 2022 (OC22) [5] 6) Open Catalyst 2025 (OC25) [6] 7) Open Direct ir Capture 2023 (ODAC23) [7] 8) Open Materials 2024 (OMat24) [8] 9) Open Molecules 2025 (OMol25) [9] 10) OMol25-neutral (subset of OMol25 that contains only molecules with zero total charge) 11) OMol25-non-neutral (subset of OMol25 that contains only molecules with non-zero total charge) 12) Open Polymers 2026 (OPoly2026) [10] 13) Nabla2DFT [11] 14) QCML [12] 15) QM7X [reference 13] 16) transition1x [14] Dataset references: [1] J. Schmidt et al., “A dataset of 175k stable and metastable materials calculated with the PBEsol and SCAN functionals,” Scientific Data, vol. 9, p. 64, 2022. [2] J. S. Smith et al., “The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules,” Scientific Data, vol. 7, p. 134, 2020. [Online]. Available: https: //www.nature.com/articles/s41597-020-0473-z [3] A. Jain et al., “Commentary: The Materials Project: A materials genome approach to accelerating materials innovation,” APL Materials, vol. 1, no. 1, p. 011002, 07 2013. [Online]. Available: https://doi.org/10.1063/1.4812323 [4] L. Chanussot et al., “Open catalyst 2020 (oc20) dataset and community challenges,” ACS Catalysis, vol. 11, no. 10, pp. 6059–6072, 2021. [Online]. Available: https://doi.org/10.1021/acscatal.0c04525 [5] K. Tran et al., “Open catalyst 2022 (oc22) dataset and challenges for oxidation electrocatalysts,” ACS Catalysis, vol. 13, no. 5, pp. 3066–3084, 2023. [Online]. Available: https://doi.org/10.1021/acscatal.2c05426 [6] S. J. Sahoo et al., “The open catalyst 2025 (oc25) dataset and models for solid-liquid interfaces,” arXiv preprint arXiv:2509.17862, 2025. [Online]. Available: https://arxiv.org/abs/2509.17862 [7] A. Sriram et al., “The open DAC 2023 dataset and challenges for sorbent discovery in direct air capture,” ACS Central Science, vol. 10, no. 5, pp. 923–941, 2024. [8] L. Barroso-Luque et al., “Open materials 2024 (omat24) inorganic materials dataset and models,” 2024. [Online]. Available: https://arxiv.org/abs/2410.12771 [9] D. S. Levine et al., “The open molecules 2025 (OMol25) dataset, evaluations, and models,” 2025. [Online]. Available: https://arxiv.org/abs/2505.08762 [10] D. S. Levine et al., The open polymers 2026 (OPoly26) dataset and evaluations,” arXiv preprint arXiv:2512.23117, 2025. [Online]. Available: https://arxiv.org/abs/2512.23117 [11] K. Khrabrov et al., “Nabla2dft: A universal quantum chemistry dataset of drug-like molecules and a benchmark for neural network potentials,” in NeurIPS 2024 Datasets and Benchmarks Track, 2024. [Online]. Available: https://openreview.net/forum?id=ElUrNM9U8c [12] S. Ganscha et al., “The QCML dataset, quantum chemistry reference data from 33.5M DFT and 14.7B semi-empirical calculations,” Scientific Data, vol. 12, p. 406, 2025. [13] J. Hoja et al., “QM7-X, a comprehensive dataset of quantum-mechanical properties spanning the chemical space of small organic molecules,” Scientific Data, vol. 8, p. 43, 2021. [Online]. Available: https://www.nature.com/articles/s41597-021-00812-2 [14] M. Schreiner et al., “Transition1x - a dataset for building generalizable reactive machine learning potentials,” Scientific Data, vol. 9, p. 779, 2022. The folder "datasets_ADIOS2_format" contains the set of pre-processed datasets in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used for the development and training of GFMs in this work. The "datasets_ADIOS2_format" directory contains 2 sub-directories, one for the version "v1" of the datasets and one for the version "v2" of the datasets. The version "v1" of the datasets provides values of the total energy as they are extracted from the original data as it was released by the respective institutions. The version "v2" of the datasets provides values of the energy that have been realigned. The realignment was performed by training a linear regression model that predicts the total energy as a function of the chemical composition of the atomistic structure, and then subtract such prediction from the original value of the total energy. Both folders "v1" and "v2" contain 16 sub-directories, each corresponding to an ADIOS2-formatted dataset The folder "DeepHyper-results" contains the configurational files and model's parameters for all the 186 HPO trials that were successfully completed by the scalable hyperparameter optimization (HPO) runs on Frontier. The content of the folder "DeepHyper-results" I structured as follows: 1) task-list.txt: list of mpnn name, jobid, and deephyper task id 2) gfm_${MPNN}_${JOBID}_0.${TASKID}: run directory with checkpoint files 3) gfm_${MPNN}: deephyper summary directory (*.csv) for each specific MPNN type 4) deephyper-experiment-${JOBID}: output and error logs for each job The file "deephyper-sorted.csv" contains the details of each HydraGNN model built and tested by HPO, obtained by merging the (*.csv) filed from each HPO run executed. Out of all the HPO trials, we selected 10 to continue the training of the respective HydraGNN models. Due to limited computational budget available in the LRN070 allocation we could not complete the training till convergence for all these 10 selected models. The folder "models" contains multiple sub-folders, one per each HydraGNN model trained. Each model sub-folder contains the parameters of each HydraGNN model, with multiple checkpoint-restarts. The list of sub-folders are as follows: 1) multidataset_hpo-BEST1-fp64 2) multidataset_hpo-BEST2-fp64 3) multidataset_hpo-BEST3-fp64 4) multidataset_hpo-BEST4-fp64 5) multidataset_hpo-BEST5-fp64 6) multidataset_hpo-BEST6-fp64 7) multidataset_hpo-BEST7-fp64 8) multidataset_hpo-BEST8-fp64 9) multidataset_hpo-BEST9-fp64 10) multidataset_hpo-BEST10-fp64 Within each one of these folders, additional auxiliary log files are provided with descriptions about how the training proceeded. The lead PaiNN-model is contained inside "multidataset_hpo-BEST6-fp64". The file "mlp_branch_weights" contains the parameters of the multi-layer perceptron (MLP) used to reconcile the predictions of the 16 output decoding heads of the HydragNN architectures. The MLP takes in input the chemical composition of the atomistic structure and predicts averaging weights to linearly mix the predictions of each output decoding head toward consolidating them into a single one. The folder "1.1billion-structure-inference" contains 1.1 billion atomistic structures randomly generated. Each structures is associated with energy and forces predicted with the lead-PaiNN model combined with the MLP model for reconciliation of the multi-branch predictions generated by the 16 output decoding heads. The folder "1.1billion-structure-inference" contains 9,300 (*.tar.gz) subdirectories, one per Frontier compute node used to execute the inference at exascale. Once uncompressed, each (*.tar.gz) subdirectory contains an ADIOS2 (*.bp) file container, where each atomistic structure is stored as a PyTorch-Geometric Data object. The file "export_dataset_environment_variables.sh" contains the environment variables that need to be set before running the HydraGNN code to reproduce the results provided in this dataset release. The code that can be used to load the ADIOS2 files, load HydraGNN models, and run inference is available at: https://github.com/ORNL/HydraGNN/releases/tag/v5.0

36 MATERIALS SCIENCE↗
DOE OSTI2019

Linear Regression Based Multi-fidelity Surrogate for Disturbance Amplification in Multi-phase Explosion

When simulations are very expensive and many are required, as for optimization or uncertainty quantification, a way to reduce cost is using surrogates. With multiple simulations to predict the quantity of interest, some being very expensive and accurate (high-fidelity simulations) and others cheaper but less accurate (low-fidelity simulations), it may be worthwhile to use multifidelity surrogates (MFSs). Moreover, if we can afford just a few high-fidelity simulations or experiments, MFS becomes necessary. Co-Kriging, which is probably the most popular MFS, replaces both low-fidelity and high-fidelity simulations by a single MFS. A recently proposed linear regression–based MFS (LR-MFS) offers the option to correct the LF simulations instead of correcting the LF surrogate in the MFS. When the low-fidelity simulation is cheap enough for use in an application, such as optimization, this may be an attractive option. Here, we explore the performance of LR-MFS using exact and surrogate-replaced low-fidelity simulations. The concern studied is a cylindrical dispersal of 100-μ m-diameter solid particles after detonation and the quantity of interest is a measure of the amplification of the departure from axisymmetry. We find very substantial accuracy improvements for this problem using the LR-MFS with exact low-fidelity simulations. Inspired by these results, we also compare the performance of co-Kriging to the use of Kriging to correct exact low-fidelity simulations and find a similar accuracy improvement when simulations are directly used. For this problem, further improvements in accuracy are achievable by taking advantage of inherent parametric symmetries. These findings may alert users of MFSs to the possible advantages of using exact low-fidelity simulations when this is affordable.

42 ENGINEERING↗
DOE OSTI2021

Model fusion with physics-guided machine learning: Projection-based reduced-order modeling

The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatiotemporal scales has enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their generalizability. This is particularly essential when data-driven models are employed within outer-loop applications like optimization. In this work, we put forth a physics-guided machine learning (PGML) framework that leverages the interpretable physics-based model with a deep learning model. Leveraging a concatenated neural network design from multi-modal data sources, the PGML framework is capable of enhancing the generalizability of data-driven models and effectively protects against or inform about the inaccurate predictions resulting from extrapolation. We apply the PGML framework as a novel model fusion approach combining the physics-based Galerkin projection model and long- to short-term memory (LSTM) network for parametric model order reduction of fluid flows. We demonstrate the improved generalizability of the PGML framework against a purely data-driven approach through the injection of physics features into intermediate LSTM layers. Our quantitative analysis shows that the overall model uncertainty can be reduced through the PGML approach, especially for test data coming from a distribution different than the training data. Moreover, we demonstrate that our approach can be used as an inverse diagnostic tool providing a confidence score associated with models and observations. The proposed framework also allows for multi-fidelity computing by making use of low-fidelity models in the online deployment of quantified data-driven models.

42 ENGINEERING↗
DOE OSTI2020

Bi-fidelity Gradient-Based Approach for Nonlinear Well Logging Inverse Problems

Solving a non-linear inverse problem is challenging in computational science and engineering. Sampling based methods require a large number of model valuations; gradientbased methods require fewer model evaluations but only find the local minima. Multifidelity optimization combines the low fidelity model and the high-fidelity model to achieve both high accuracy and high efficiency. In this paper, we present a bi-fidelity approach to solve non-linear inverse problems. In the bi-fidelity inversion method, the low-fidelity model is used to acquire a good initial guess, and the high-fidelity model is used to locate the global minimum. Combined with a multi-start optimization scheme, the proposed approach significantly increases the possibility of finding the global minimum for nonlinear inverse problems with many local minima. The method is tested with two toy problems and then applied to an electromagnetic well logging inverse problem, which is difficult to solve using traditional gradient-based methods. The bi-fidelity method provides promising inversion results and can be easily applied to traditional gradient-based methods.

42 ENGINEERING↗
DOE OSTI2021

srMO-BO-3GP: A sequential regularized multi-objective Bayesian optimization for constrained design applications using an uncertain Pareto classifier

Bayesian optimization (BO) is an efficient and flexible global optimization framework that is applicable to a very wide range of engineering applications. To leverage the capability of the classical BO, many extensions, including multi-objective, multi-fidelity, parallelization, and latent-variable modeling, have been proposed to address the limitations of the classical BO framework. In this work, we propose a novel multi-objective BO formalism, called srMO-BO-3GP, to solve multi-objective optimization problems in a sequential setting. Three different Gaussian processes (GPs) are stacked together, where each of the GPs is assigned with a different task. The first GP is used to approximate a single-objective computed from the multi-objective definition, the second GP is used to learn the unknown constraints, and the third one is used to learn the uncertain Pareto frontier. At each iteration, a multi-objective augmented Tchebycheff function is adopted to convert multi-objective to single-objective, where the regularization with a regularized ridge term is also introduced to smooth the single-objective function. Finally, we couple the third GP along with the classical BO framework to explore the convergence and diversity of the Pareto frontier by the acquisition function for exploitation and exploration. The proposed framework is demonstrated using several numerical benchmark functions, as well as a thermomechanical finite element model for flip-chip package design optimization.

42 ENGINEERING↗
DOE OSTI2022

Ensemble approximate control variate estimators: Applications to multi-fidelity importance sampling.

The recent growth in multifidelity uncertainty quantification has given rise to a large set of variance reduction techniques that leverage information from model ensembles to provide variance reduction for estimates of the statistics of a high-fidelity model. In this paper we provide two contributions: (1) we utilize an ensemble estimator to account for uncertainties in the optimal weights of approximate control variate (ACV) approaches and derive lower bounds on the number of samples required to guarantee variance reduction; and (2) we extend an existing multifidelity importance sampling (MFIS) scheme to leverage control variates. Our approach directly addresses a limitation of many multifidelity sampling strategies that require the usage of pilot samples to estimate covariances. As such we make significant progress towards both increasing the practicality of approximate control variates—for instance, by accounting for the effect of pilot samples—and using multifidelity approaches more effectively for estimating low-probability events. The numerical results indicate our hybrid MFIS-ACV estimator achieves up to 50% improvement in variance reduction over the existing state-of-the-art MFIS estimator, which had already shown an outstanding convergence rate compared to the Monte Carlo method, on several problems of computational mechanics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

HydraGNN_Predictive_GFM_2024 - Ensemble of predictive graph foundation models for ground state atomistic materials modeling

We provide the ensemble of fifteen pre-trained graph foundation models (GFMs) for atomistic materials modeling applications. Each one of the fifteen GFMs has been trained on five open-source datasets that (once aggregated) amount to over 154 million atomistic structures, which cover over two-thirds of the natural elements of the periodic table and that comprises a broad set of organic and inorganic compounds. This vast set of atomistic structures comprises ground state configurations that are dynamically stable (i.e., equilibrated structures with atomic forces approximately close to zero values) as well as dynamically unstable structures (i.e., non-equilibrium structures with non-negligible non-zero values of atomic forces). The ensemble of datasets aggregated does NOT include excited states. The datasets have been curated to remove atomistic structures with spectral norm of the force tensor above 100 eV/angstrom. Moreover, a linear term of the energy was computed for each dataset using a linear regression model that uses the chemical concentration of each natural element as regressor. The linear term predicted by the linear regression model has been subtracted from each original energy value to perform a re-alignment of the energy values across different electronic structures approximation theories performed to generate the diverse multi-source, multi-fidelity datasets. The folder "ADIOS_files" contains the set of pre-processed datasets in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used for the development and training of GFMs in this work. The "ADIOS_files" directory contains 6 sub-directories named as follows: - ANI1x-v3.bp - MPTrj-v3.bp - OC2020-20M-v3.bp - OC2020-v3.bp - OC2022-v3.bp - qm7x-v3.bp Each sub-directory contains the pre-processed datasets converted in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used to the development, training, and performance testing of the ensemble go predictive graph foundation models. Each GFM was developed using HydraGNN (https://github.com/ORNL/HydraGNN) as underlying graph neural network (GNN) architecture. The multi-task learning (MTL) capability of HydraGNN was used to simultaneously train the GFMs on labeled values for direct predictions of energy (a total system property of an atomistic structure that measures the chemical stability) and atomic forces (an atomic level property of an atomistic structure that measures the dynamical stability). The hyper parameters of the GFM have been tuned using scalable hyperparameter optimization (HPO) algorithms implemented in the software DeepHyper (https://github.com/deephyper/deephyper). The pre-training of each HPO trial was performed using distributed data parallelism (DDP) to scale the training across 128 compute nodes of the exascale OLCF supercomputer Frontier. Each HPO trial was trained only for 10 epochs and an early stopping was performed to avoid wasting significant computational resources on GNN architectures that were clearly underperforming. For each HPO trial, the 'omnistat' tool developed by (AMD Research - Advanced Micro Device) was used to measure the total energy consumption in kWh. The ensemble of GFMs was obtained by selecting the fifteen best performing HPO trials. Four models have been selected for their clear advantage in accuracy, and these are the GFMs with IDs 229, 156, 147, 260. Additional eleven models have been selected based on judicious balance between accuracy and energy consumption needed for training, and these are the GFMs with IDs 165, 78, 137, 1, 175, 171, 181, 67, 179, 167, 351. Each selected GFM of the ensemble was continued to cumulate a total of at most 30 epochs. In some cases, the total number of epochs actually performed was les than 30 due to two combined factors: (1) the size of the GFM (i.e., the number of model parameters to train) and (2) the total wall-clock time for which the computational resources could be allocated on OLCF-Frontier. The "Ensemble_of_models" directory contains 15 sub-directories named as follows: - gfm_0.229 - gfm_0.156 - gfm_0.147 - gfm_0.260 - gfm_0.165 - gfm_0.78 - gfm_0.137 - gfm_0.1 - gfm_0.175 - gfm_0.171 - gfm_0.181 - gfm_0.67 - gfm_0.179 - gfm_0.167 - gfm_0.351 Each one of these sub-directories refers to one of the fifteen HPO trials that have been selected to continue the pre-training with at most 30 epochs. With each sub-directory associated with a specific HPO trial, the following files can be found: - config.json: file for argument parsing to develop and train an HydraGNN architecture - gfm_0.ID_epoch_N.pk: file with model parameters for HPO ID trial after N epochs of training The ensemble of fifteen GFM architectures was used for (1) ensemble averaging to stabilize the predictions of energy and atomic forces after pre-training for post-processing analysis and (2) ensemble uncertainty quantification (UQ). The code used to develop, pre-train, and load the pre-trained models for post-processing analysis is available on the ORNL-GitHub at the following link: https://github.com/ORNL/HydraGNN/tree/Predictive_GFM_2024

36 MATERIALS SCIENCE↗
DOE OSTI2024

Analysis of the Challenges in Developing Sample-Based Multi-fidelity Estimators for Non-deterministic Models

Multifidelity (MF) uncertainty quantification (UQ) seeks to leverage and fuse information from a collection of models to achieve greater statistical accuracy with respect to a single-fidelity counterpart, while maintaining an efficient use of computational resources. Despite many recent advancements in MF UQ, several challenges remain and these often limit its practical impact in certain application areas. In this manuscript, we focus on the challenges introduced by nondeterministic models to sampling MF UQ estimators. Nondeterministic models produce different responses for the same inputs, which means their outputs are effectively noisy. MF UQ is complicated by this noise since many state-of-the-art approaches rely on statistics, e.g., the correlation among models, to optimally fuse information and allocate computational resources. Here, we demonstrate how the statistics of the quantities of interest, which impact the design, effectiveness, and use of existing MF UQ techniques, change as functions of the noise. With this in hand, we extend the unifying approximate control variate framework to account for nondeterminism, providing for the first time a rigorous means of comparing the effect of nondeterminism on different multifidelity estimators and analyzing their performance with respect to one another. Numerical examples are presented throughout the manuscript to illustrate and discuss the consequences of the presented theoretical results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

REBOUND: Reverse Engineering Bidirectional Outflow Under Non-Equilibrium Diffusion

Rare-earth elements (REEs) are essential for electronics, renewable energy, and defense technologies. However, the current supply of REEs relies on mining concentrated in a few countries and energy-intensive separations. DOE’s Basic Energy Sciences (BES) program has launched a grand challenge which aims to ensure a sustainable supply of critical REEs by developing innovative and environmentally friendly separation methods. As an alternative to costly and harmful traditional methods, the Non-Equilibrium Transport Driven Separations (NETS) initiative has created a microfluidic Y-channel co-flow method that applies external fields to exploit magneto- and electrohydrodynamic effects for separating dilute REE ions from complex feedstocks. Computational fluid dynamics (CFD) studies have identified a few operating conditions with promising ion selectivity and separation efficiency. However, challenges remain regarding Y-channel versatility across feedstocks and accurate incorporation of physical phenomena into CFD models. In this work, we develop a multi-fidelity modelling approach which integrates experimental results with CFD simulation to build a surrogate model for the dependence of separation efficiency to variation of design parameters. The surrogate model enables a reinforcement learning (RL) method to adaptively launch CFD and experimental runs, improving model fidelity around optimal Y-channel parameters.

36 MATERIALS SCIENCE↗