Search NASA⌕ Search

SEARCH · Search NASA

Results for “multi-fidelity optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Robust Multi-fidelity Bayesian Optimization with Deep Kernel and Partition

Multi-fidelity Bayesian optimization (MFBO) is a powerful approach that utilizes lowfidelity, cost-effective sources to expedite the exploration and exploitation of a high-fidelity objective function. Existing MFBO methods with theoretical foundations either lack justification for performance improvements over single-fidelity optimization or rely on strong assumptions about the relationships between fidelity sources to construct surrogate models and direct queries to low-fidelity sources. To mitigate the dependency on cross-fidelity assumptions while maintaining the advantages of low-fidelity queries, we introduce a random sampling and partition-based MFBO framework with deep kernel learning. This framework is robust to cross-fidelity model misspecification and explicitly illustrates the benefits of low-fidelity queries. Our results demonstrate that the proposed algorithm effectively manages complex cross-fidelity relationships and efficiently optimizes the target fidelity function.

Zhang, Fengxue [University of Chicago, Illinois, U↗
DOE OSTI2026

Consequential improvement acquisition function for efficient multi-fidelity Bayesian optimization

Abstract Surrogate-based Bayesian optimization has been widely applied in design optimization to increase sampling efficiency. However, the cost for each evaluation of the objective function can still be very high when physical experiments or large-scale simulations are involved. Multi-fidelity Bayesian optimization is the new approach to further improve the sampling efficiency by reducing the number of expensive samples at the highest fidelity level and supplementing them with less expensive ones at low-fidelity levels. In this paper, a new consequential improvement (CI) acquisition function is proposed to allow for the simultaneous selection of the solution and the fidelity level in problems with a known hierarchy of fidelity levels. The new CI acquisition function incorporates the consequential effectiveness of objective improvement with the considerations of cost, accuracy, and validity differences between high- and low-fidelity samples in engineering practice. The new method of multi-fidelity Bayesian optimization based on the CI is demonstrated with several analytical and simulation-based design examples. In the simulation-based design optimization example, the results show that the CI acquisition function has a decisive advantage in the sampling efficiency over the other methods of multi-fidelity Bayesian optimization with simultaneous selection. The results indicate that the proposed method is particularly advantageous in solving high-dimensional problems and when large cost ratios between high- and low-fidelity evaluations exist and high-fidelity validation is mandatory. Furthermore, the method robustly avoids the prevalent issue of over sampling at low-fidelity levels.

Aydogdu, Ibrahim [Georgia Institute of Technology,↗
DOE OSTI2025

Optimization Through Multi-Fidelity Modeling

We present a novel method for optimizing parameter selection for simulations with an evaluation budget. We start with an existing method for building a multi-fidelity model out of many low-fidelity simulations and few high-fidelity simulations. We propose a novel method to simplify parameter selection without sacrificing performance. We verify these results and compare with existing literature. Next, we propose a novel algorithm which uses this difference model to suggest new points in the parameter design space to simulate. We add each point we simulate to the model to improve its quality for the next iteration. The algorithm trades off reducing the uncertainty of the existing model with optimization of the objective. The first is more useful when a large fraction of the computation budget remains. The second is more useful when a small fraction of the computation budget remains. Our method converges to the optimum by using a high-fidelity evaluation for just 16 of the 427 points. Our method is general enough to work if there is no low-fidelity model. Furthermore, it is agnostic to the underlying physics of the problem. Therefore, both the low-fidelity and high-fidelity models can be generated by any arbitrary function, including simulations and physical experiments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Multi-Fidelity Bayesian Optimization with Gaussian Processes for Double Shell Inertial Confinement Fusion Target Design

Reliable, secure access to energy is a major focus for national security efforts. One potential route to such energy is through fusion reactions in inertial confinement fusion (ICF) experiments. Such experiments are carried out at facilities such as the National Ignition Facility (NIF) in Livermore, California, where high powered lasers are used to compress a DT fuel-containing target to the necessary high temperature, high pressure conditions. These experiments are limited in number, which creates a heavy dependence on high fidelity predictive physics simulations and analysis performed “pre shot,” or before the experiment occurs. Many of these simulations in higher dimensions (2D and 3D) are computationally expensive, so finding optimal simulation-based designs presents its own challenges. In this work, we present our multi-fidelity Bayesian optimization with Gaussian processes (GPs) for ICF double shell targets, where a 1D surrogate model is used to help find a 2D surrogate model, enabling us to find optimal targets in the higher fidelity (2D), while saving computational cost.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Ard [SWR-25-18]

A wind farm optimization suite for wind energy that is built for modular, gradient-enabled multi-disciplinary and multi-fidelity optimizations. Dig into wind farm design. An ard is a type of simple and lightweight plow, used through the single-digit centuries to prepare a farm for planting. The intent of Ard is to be a modular, full-stack multi-disciplinary optimization tool for wind farms. The problem with wind farms is that they are complicated, multi-disciplinary objects. They are aerodynamic machines, with complicated control systems, power electronic devices, social and political objects, and the core value (and cost) of complicated financial instruments. Moreover, the design of one of these aspects affects all the rest! Ard seeks to make plant-level design choices that can incorporate these different aspects and their interactions to make wind energy projects more successful.

Frontin, Cory [National Renewable Energy Laborator↗
DOE OSTI2026

Adaptive Computing and Multi-Fidelity Learning

We describe our ongoing research in adaptive computing. Our goal is to use a combination of low- and high-fidelity simulation models to enable computationally efficient optimization and uncertainty quantification. We develop optimization formulations that take into account the compute resources currently available, which act as a constraint with regards to the fidelity level simulation we can run while maximizing information gain. We will discuss a few application examples that can benefit from this approach, especially when considering challenges arising in scaling up experiments and simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Comparison of tungsten versus molybdenum for double shell capsules using machine learning design optimization

Double shell targets are an alternative ignition platform for inertial confinement fusion. One design consideration for double shell targets is the choice of inner shell material to help trap radiation emitted by the hot fuel to aid ignition. Materials such as molybdenum and tungsten are of interest for the inner shell layer of the targets. While molybdenum has a lower density that could inhibit instability growth and allow for radiography and code benchmarking, tungsten has a higher density that could provide more compression and confinement. These tradeoffs have been explored using optimized designs for each material. Our previous work [Vazirani et al., “Coupling 1D xRAGE simulations with machine learning for graded inner shell design optimization in double shell capsules,” Phys. Plasmas 28, 122709 (2021); Vazirani et al., “Coupling multi-fidelity xRAGE with machine learning for graded inner shell design optimization in double shell capsules,” Phys. Plasmas 30, 062704 (2023); and Vazirani et al., “Bayesian batch optimization for molybdenum versus tungsten inertial confinement fusion double shell target design,” Stat. Anal. Data Min. 17, e11698 (2024)] resulted in a multi-fidelity Bayesian optimization framework to find yield-optimized double shell target geometries. By leveraging simulations of varying fidelities (one-dimensional and two-dimensional) to inform one another, the multi-fidelity optimization was able to optimize a design in the highest fidelity with significantly fewer simulations than would be used in a systematic parameter scan. In this work, we apply the multi-fidelity Bayesian optimization to explore the optimized designs of double shell targets with molybdenum and tungsten inner shells as well as the physics producing the high performing implosions. A physics exploration of all the simulations used in this study shows trends in designs that contribute to high yields, ion temperatures, and fuel areal densities. Comparison of molybdenum and tungsten simulations shows that they can produce similar implosion conditions with different geometries, which would be important to study in experiments. Graded density layers produce varying performances with the two materials but continue to be of interest for future studies along with studies of doped inner shell materials and applied surface roughness.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

ETHOS: An automated framework to generate multi-fidelity constitutive data tables and propagate uncertainties to hydrodynamic simulations

Accurate constitutive data, such as equations of state and plasma transport coefficients, are necessary for reliable hydrodynamic simulations of plasma systems such as fusion targets, planets, and stars. Here, we develop a framework for automatically generating transport-coefficient tables using a parameterized model that incorporates data from both high-fidelity sources (e.g., density functional theory calculations and reference experiments) and lower-fidelity sources (e.g., average-atom and analytic models). The framework incorporates uncertainties from these multi-fidelity sources, generating ensembles of optimally diverse tables that are suitable for uncertainty quantification of hydrodynamic simulations. We illustrate the utility of the framework with magnetohydrodynamic simulations of magnetically launched flyer plates, which are used to measure material properties in pulsed-power experiments. We explore how changes in the uncertainties assigned to the multi-fidelity data sources propagate to changes in simulation outputs and find that our simulations are most sensitive to uncertainties near the melting transition. The presented framework enables computationally efficient uncertainty quantification that readily incorporates new high-fidelity measurements or calculations and identifies plasma regimes where additional data will have high impact.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

AutoFocus: AI/ML-driven real-time wavefront diagnostics to autonomously align and optimize X-ray optics

We present an integrated system that combines advanced wavefront diagnostics with artificial intelligence (AI) to automate and optimize X-ray optics at synchrotron beamlines. This system couples real-time wavefront sensing with AI-driven control algorithms to achieve precise beam alignment, stabilization, and performance optimization. A key feature is the use of multi-fidelity transfer learning, which enables knowledge gained from both real-world beamline optimizations and ultra-realistic digital twin simulations to be effectively applied to in situ optimization. By leveraging multi-objective bayesian optimization, the system continuously refines its performance, reducing optimization time and minimizing the need for manual adjustments. Designed for seamless deployment, it operates with existing beamline hardware and provides an intuitive graphical interface. Initial deployments at the advanced photon source beamlines have demonstrated its ability to enhance beam stability, improve reproducibility, and significantly streamline alignment procedures. This AI-enhanced control framework represents a significant step toward fully autonomous beamline operation in next-generation synchrotron facilities.

Rebuffi, Luca [Argonne National Laboratory (ANL), ↗
DOE OSTI2025

DeepHyper: A Python Package for Massively Parallel Hyperparameter Optimization in Machine Learning

Machine learning models are increasingly applied across scientific disciplines, yet their effectiveness often hinges on heuristic decisions—such as data transformations, training strategies, and model architectures—that are not learned by the models themselves. Automating the selection of these heuristics and analyzing their sensitivity is crucial for building robust and efficient learning workflows. DeepHyper addresses this challenge by democratizing hyperparameter optimization, providing accessible tools to streamline and enhance machine learning workflows from a laptop to the largest supercomputer in the world. Building on top of hyperparameter optimization, it unlocks new capabilities around ensembles of models for improved accuracy and uncertainty quantification. All of these organized around efficient parallel computing.

ensemble↗
DOE OSTI2025

Hybrid renewable energy systems

In the pursuit of ecologically sustainable and resilient energy systems, increasingly more attention is being devoted to a diversity of energy generation and storage methods. As the landscape of generation technology gains nuance and complexity, a wide-ranging set of technical questions has emerged, touching on topics that range from control and optimization of hybrid systems to finance and economic viability to multi-fidelity modeling and scientific machine learning. In the context of this special issue, hybrid renewable energy systems are any systems that consider the combined dynamics of more than one form of generation, storage, or grid subsystem. Research endeavors have delved into improving the flexibility of energy systems by utilizing existing resources, introducing novel operational strategies, deploying enhanced renewable forecasts, and exploring emerging technologies. In conclusion, the interconnection among various sectors has garnered heightened attention, not only due to the provision of additional tradable energy products but also for furnishing flexible headroom to system operators.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

Surrogate-driven design optimization with uncertainty constraints in Monte Carlo simulations

In multi-objective design tasks, the computational cost increases rapidly when high-fidelity simulations are used to evaluate objective functions. Surrogate models help mitigate this cost by approximating the simulation output, simplifying the design process. However, under high uncertainty, surrogate models trained on noisy data can produce inaccurate predictions, as their performance depends heavily on the quality of training data. This study investigates the impact of data uncertainty on two multi-objective design problems modelled using Monte Carlo transport simulations: a neutron moderator and an ion-to-neutron converter. For each, a grid search was performed using five different tally uncertainty levels to generate training data for neural network surrogate models. These models were then optimized using NSGA-III. The recovered Pareto-fronts were analyzed across uncertainty levels: in the moderator problem, normalized hypervolume dropped from 0.886 at 1.0% uncertainty to 0.748 at 10% uncertainty, while in the converter problem it remained near 0.50 for all cases. Average simulation times were also compared to evaluate the trade-off between accuracy and computational cost. Results show that the influence of simulation uncertainty is strongly problem-dependent. In the neutron moderator case, higher uncertainties led to exaggerated objective sensitivities and distorted Pareto-fronts, reducing normalized hypervolume. In contrast, the ion-to-neutron converter task was less affected—low-fidelity simulations produced results similar to those from high-fidelity data. These findings suggest that a fixed-fidelity approach is not optimal. Surrogate models can recover the Pareto-front under noisy conditions, and multi-fidelity studies help identify suitable uncertainty levels for each problem to balance efficiency and accuracy.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2025

Advancements on Multi-Fidelity Random Fourier Neural Networks: Application to Hurricane Modeling for Wind Energy

Multi-fidelity approaches are emerging as effective strategies in computational science to handle otherwise intractable tasks like Uncertainty Quantification (UQ), training of Machine Learning (ML) models, and optimization, for expensive high-fidelity applications in which the amount of available simulations or data is limited. The main idea is simple: large datasets generated for low-fidelity approximations of the problem at hand are fused with a much sparser dataset for the target (high-fidelity) system. In this paper, we build on our recent success in designing random Fourier Neural Networks (rFNNs) [1] to target problems arising in wind energy applications and in particular problems of interest for hurricane modeling. In this context, data for the high-fidelity models are limited and lower fidelity alternatives are needed. In this work, we introduce a novel multi-fidelity training approach for our rFNNs and demonstrate its use on a simple verification problem and on a hurricane modeling problem in which high-fidelity data are generated via Large-Eddy Simulations (LES), while low-fidelity data are given by a mesoscale model. Initial results demonstrate how the multi-fidelity training approach can improve the quality of the resulting surrogate.

Fourier Neural Networks↗
DOE OSTI2025

Adaptive Computing for Scale-Up Problems

Adaptive Computing is an application-agnostic outer loop framework to strategically deploy simulations and experiments to guide decision making for scale-up analysis. Resources are allocated over successive batches, which makes the allocation adaptive to some objective such as optimization or model training. The framework enables the characterization and management of uncertainties associated with predictive models of complex systems when scale-up questions lead to significant model extrapolation. A key advancement of this framework is its integration of multi-fidelity surrogate modeling, uncertainty management, and automated orchestration of various computing and experimentation resources into a single integrated software package. This enables efficient multi-fidelity modeling across multiple computing resources by incorporating real-world constraints such as relative queue times and throughput on individual machines into the multi-fidelity sampling decision. We discuss applications of this framework to problems in the renewable energy space, including biofuels production, material synthesis, perovskite crystal growth, and building electrical loads.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Simulations of edge and SOL turbulence in diverted negative and positive triangularity plasmas

Optimizing the performance of magnetic confinement fusion devices is critical to achieving an attractive fusion reactor design. Negative triangularity (NT) scenarios have been shown to achieve excellent levels of energy confinement, while avoiding edge localized modes. Modeling turbulent transport in the edge and SOL is key in understanding the impact of NT on turbulence and extrapolating the results to future devices and regimes. Previous gyrokinetic turbulence studies have reported beneficial effects of NT across a broad range of parameters. However, most simulations have focused on the inner plasma region, neglecting the impact of NT on the outermost edge. In this work, we investigate the effect of NT in edge and scrape-off layer simulations, including the magnetic X-point and separatrix. For the first time, we employ a multi-fidelity approach, combining global, non-linear gyrokinetic simulations with drift-reduced fluid simulations, to gain a deeper understanding of the underlying physics at play. First-principles simulations using the GENE-X code demonstrate that in comparable NT and PT geometries, similar profiles are achieved, while the turbulent heat flux is reduced by more than 50% in NT. Comparisons with results from the drift-reduced fluid turbulence code GRILLIX suggest that the turbulence is driven by trapped electron modes. The parallel heat flux width on the divertor targets is reduced in NT, primarily due to a lower spreading factor S.

GENE-X↗
DOE OSTI2025

Inverse design of photonic surfaces via multi fidelity ensemble framework and femtosecond laser processing

We demonstrate a multi-fidelity (MF) machine learning ensemble framework for the inverse design of photonic surfaces, trained on a dataset of 11,759 samples that we fabricate using high throughput femtosecond laser processing. The MF ensemble combines an initial low fidelity model for generating design solutions, with a high fidelity model that refines these solutions through local optimization. The combined MF ensemble can generate multiple disparate sets of laser-processing parameters that can each produce the same target input spectral emissivity with high accuracy (root mean squared errors < 2%). SHapley Additive exPlanations analysis shows transparent model interpretability of the complex relationship between laser parameters and spectral emissivity. Finally, the MF ensemble is experimentally validated by fabricating and evaluating photonic surface designs that it generates for improved efficiency energy harvesting devices. Our approach provides a powerful tool for advancing the inverse design of photonic surfaces in energy harvesting applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

HydraGNN_Predictive_GFM_2026 - Ensemble of predictive graph foundation models for atomistic materials modeling

This release contains data and parameters of HydraGNN-based graph foundation models trained as a result of the work published in the pre-print "Exascale Multi-Task Graph Foundation Models for Imbalanced, Multi-Fidelity Atomistic Data" by M. Lupo Pasini et al. (https://arxiv.org/abs/2604.15380). We jointly train on 16 open first-principles datasets (544+ million structures covering 85+ elements) using a multi-task architecture with per-dataset heads and a scalable ADIOS2/DDStore data pipeline. On Frontier, we execute six large-scale DeepHyper hyperparameter optimization campaigns in FP64 and promote the top-performing message-passing models to sustained 2,048-node training, yielding a PaiNN-based lead model. The version of HydraGNN used to generate the outputs provided in this release is HydraGNN v5.0 (https://github.com/ORNL/HydraGNN/releases/tag/v5.0) The list of datasets used for the training of the graph foundation model is the following: 1) Alexandria [1] 2) ANI1x [2] 3) MPTrj [3] 4) Open Catalyst 2020 (OC20) [4] 5) Open Catalyst 2022 (OC22) [5] 6) Open Catalyst 2025 (OC25) [6] 7) Open Direct ir Capture 2023 (ODAC23) [7] 8) Open Materials 2024 (OMat24) [8] 9) Open Molecules 2025 (OMol25) [9] 10) OMol25-neutral (subset of OMol25 that contains only molecules with zero total charge) 11) OMol25-non-neutral (subset of OMol25 that contains only molecules with non-zero total charge) 12) Open Polymers 2026 (OPoly2026) [10] 13) Nabla2DFT [11] 14) QCML [12] 15) QM7X [reference 13] 16) transition1x [14] Dataset references: [1] J. Schmidt et al., “A dataset of 175k stable and metastable materials calculated with the PBEsol and SCAN functionals,” Scientific Data, vol. 9, p. 64, 2022. [2] J. S. Smith et al., “The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules,” Scientific Data, vol. 7, p. 134, 2020. [Online]. Available: https: //www.nature.com/articles/s41597-020-0473-z [3] A. Jain et al., “Commentary: The Materials Project: A materials genome approach to accelerating materials innovation,” APL Materials, vol. 1, no. 1, p. 011002, 07 2013. [Online]. Available: https://doi.org/10.1063/1.4812323 [4] L. Chanussot et al., “Open catalyst 2020 (oc20) dataset and community challenges,” ACS Catalysis, vol. 11, no. 10, pp. 6059–6072, 2021. [Online]. Available: https://doi.org/10.1021/acscatal.0c04525 [5] K. Tran et al., “Open catalyst 2022 (oc22) dataset and challenges for oxidation electrocatalysts,” ACS Catalysis, vol. 13, no. 5, pp. 3066–3084, 2023. [Online]. Available: https://doi.org/10.1021/acscatal.2c05426 [6] S. J. Sahoo et al., “The open catalyst 2025 (oc25) dataset and models for solid-liquid interfaces,” arXiv preprint arXiv:2509.17862, 2025. [Online]. Available: https://arxiv.org/abs/2509.17862 [7] A. Sriram et al., “The open DAC 2023 dataset and challenges for sorbent discovery in direct air capture,” ACS Central Science, vol. 10, no. 5, pp. 923–941, 2024. [8] L. Barroso-Luque et al., “Open materials 2024 (omat24) inorganic materials dataset and models,” 2024. [Online]. Available: https://arxiv.org/abs/2410.12771 [9] D. S. Levine et al., “The open molecules 2025 (OMol25) dataset, evaluations, and models,” 2025. [Online]. Available: https://arxiv.org/abs/2505.08762 [10] D. S. Levine et al., The open polymers 2026 (OPoly26) dataset and evaluations,” arXiv preprint arXiv:2512.23117, 2025. [Online]. Available: https://arxiv.org/abs/2512.23117 [11] K. Khrabrov et al., “Nabla2dft: A universal quantum chemistry dataset of drug-like molecules and a benchmark for neural network potentials,” in NeurIPS 2024 Datasets and Benchmarks Track, 2024. [Online]. Available: https://openreview.net/forum?id=ElUrNM9U8c [12] S. Ganscha et al., “The QCML dataset, quantum chemistry reference data from 33.5M DFT and 14.7B semi-empirical calculations,” Scientific Data, vol. 12, p. 406, 2025. [13] J. Hoja et al., “QM7-X, a comprehensive dataset of quantum-mechanical properties spanning the chemical space of small organic molecules,” Scientific Data, vol. 8, p. 43, 2021. [Online]. Available: https://www.nature.com/articles/s41597-021-00812-2 [14] M. Schreiner et al., “Transition1x - a dataset for building generalizable reactive machine learning potentials,” Scientific Data, vol. 9, p. 779, 2022. The folder "datasets_ADIOS2_format" contains the set of pre-processed datasets in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used for the development and training of GFMs in this work. The "datasets_ADIOS2_format" directory contains 2 sub-directories, one for the version "v1" of the datasets and one for the version "v2" of the datasets. The version "v1" of the datasets provides values of the total energy as they are extracted from the original data as it was released by the respective institutions. The version "v2" of the datasets provides values of the energy that have been realigned. The realignment was performed by training a linear regression model that predicts the total energy as a function of the chemical composition of the atomistic structure, and then subtract such prediction from the original value of the total energy. Both folders "v1" and "v2" contain 16 sub-directories, each corresponding to an ADIOS2-formatted dataset The folder "DeepHyper-results" contains the configurational files and model's parameters for all the 186 HPO trials that were successfully completed by the scalable hyperparameter optimization (HPO) runs on Frontier. The content of the folder "DeepHyper-results" I structured as follows: 1) task-list.txt: list of mpnn name, jobid, and deephyper task id 2) gfm_${MPNN}_${JOBID}_0.${TASKID}: run directory with checkpoint files 3) gfm_${MPNN}: deephyper summary directory (*.csv) for each specific MPNN type 4) deephyper-experiment-${JOBID}: output and error logs for each job The file "deephyper-sorted.csv" contains the details of each HydraGNN model built and tested by HPO, obtained by merging the (*.csv) filed from each HPO run executed. Out of all the HPO trials, we selected 10 to continue the training of the respective HydraGNN models. Due to limited computational budget available in the LRN070 allocation we could not complete the training till convergence for all these 10 selected models. The folder "models" contains multiple sub-folders, one per each HydraGNN model trained. Each model sub-folder contains the parameters of each HydraGNN model, with multiple checkpoint-restarts. The list of sub-folders are as follows: 1) multidataset_hpo-BEST1-fp64 2) multidataset_hpo-BEST2-fp64 3) multidataset_hpo-BEST3-fp64 4) multidataset_hpo-BEST4-fp64 5) multidataset_hpo-BEST5-fp64 6) multidataset_hpo-BEST6-fp64 7) multidataset_hpo-BEST7-fp64 8) multidataset_hpo-BEST8-fp64 9) multidataset_hpo-BEST9-fp64 10) multidataset_hpo-BEST10-fp64 Within each one of these folders, additional auxiliary log files are provided with descriptions about how the training proceeded. The lead PaiNN-model is contained inside "multidataset_hpo-BEST6-fp64". The file "mlp_branch_weights" contains the parameters of the multi-layer perceptron (MLP) used to reconcile the predictions of the 16 output decoding heads of the HydragNN architectures. The MLP takes in input the chemical composition of the atomistic structure and predicts averaging weights to linearly mix the predictions of each output decoding head toward consolidating them into a single one. The folder "1.1billion-structure-inference" contains 1.1 billion atomistic structures randomly generated. Each structures is associated with energy and forces predicted with the lead-PaiNN model combined with the MLP model for reconciliation of the multi-branch predictions generated by the 16 output decoding heads. The folder "1.1billion-structure-inference" contains 9,300 (*.tar.gz) subdirectories, one per Frontier compute node used to execute the inference at exascale. Once uncompressed, each (*.tar.gz) subdirectory contains an ADIOS2 (*.bp) file container, where each atomistic structure is stored as a PyTorch-Geometric Data object. The file "export_dataset_environment_variables.sh" contains the environment variables that need to be set before running the HydraGNN code to reproduce the results provided in this dataset release. The code that can be used to load the ADIOS2 files, load HydraGNN models, and run inference is available at: https://github.com/ORNL/HydraGNN/releases/tag/v5.0

36 MATERIALS SCIENCE↗
DOE OSTI2024

HydraGNN_Predictive_GFM_2024 - Ensemble of predictive graph foundation models for ground state atomistic materials modeling

We provide the ensemble of fifteen pre-trained graph foundation models (GFMs) for atomistic materials modeling applications. Each one of the fifteen GFMs has been trained on five open-source datasets that (once aggregated) amount to over 154 million atomistic structures, which cover over two-thirds of the natural elements of the periodic table and that comprises a broad set of organic and inorganic compounds. This vast set of atomistic structures comprises ground state configurations that are dynamically stable (i.e., equilibrated structures with atomic forces approximately close to zero values) as well as dynamically unstable structures (i.e., non-equilibrium structures with non-negligible non-zero values of atomic forces). The ensemble of datasets aggregated does NOT include excited states. The datasets have been curated to remove atomistic structures with spectral norm of the force tensor above 100 eV/angstrom. Moreover, a linear term of the energy was computed for each dataset using a linear regression model that uses the chemical concentration of each natural element as regressor. The linear term predicted by the linear regression model has been subtracted from each original energy value to perform a re-alignment of the energy values across different electronic structures approximation theories performed to generate the diverse multi-source, multi-fidelity datasets. The folder "ADIOS_files" contains the set of pre-processed datasets in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used for the development and training of GFMs in this work. The "ADIOS_files" directory contains 6 sub-directories named as follows: - ANI1x-v3.bp - MPTrj-v3.bp - OC2020-20M-v3.bp - OC2020-v3.bp - OC2022-v3.bp - qm7x-v3.bp Each sub-directory contains the pre-processed datasets converted in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used to the development, training, and performance testing of the ensemble go predictive graph foundation models. Each GFM was developed using HydraGNN (https://github.com/ORNL/HydraGNN) as underlying graph neural network (GNN) architecture. The multi-task learning (MTL) capability of HydraGNN was used to simultaneously train the GFMs on labeled values for direct predictions of energy (a total system property of an atomistic structure that measures the chemical stability) and atomic forces (an atomic level property of an atomistic structure that measures the dynamical stability). The hyper parameters of the GFM have been tuned using scalable hyperparameter optimization (HPO) algorithms implemented in the software DeepHyper (https://github.com/deephyper/deephyper). The pre-training of each HPO trial was performed using distributed data parallelism (DDP) to scale the training across 128 compute nodes of the exascale OLCF supercomputer Frontier. Each HPO trial was trained only for 10 epochs and an early stopping was performed to avoid wasting significant computational resources on GNN architectures that were clearly underperforming. For each HPO trial, the 'omnistat' tool developed by (AMD Research - Advanced Micro Device) was used to measure the total energy consumption in kWh. The ensemble of GFMs was obtained by selecting the fifteen best performing HPO trials. Four models have been selected for their clear advantage in accuracy, and these are the GFMs with IDs 229, 156, 147, 260. Additional eleven models have been selected based on judicious balance between accuracy and energy consumption needed for training, and these are the GFMs with IDs 165, 78, 137, 1, 175, 171, 181, 67, 179, 167, 351. Each selected GFM of the ensemble was continued to cumulate a total of at most 30 epochs. In some cases, the total number of epochs actually performed was les than 30 due to two combined factors: (1) the size of the GFM (i.e., the number of model parameters to train) and (2) the total wall-clock time for which the computational resources could be allocated on OLCF-Frontier. The "Ensemble_of_models" directory contains 15 sub-directories named as follows: - gfm_0.229 - gfm_0.156 - gfm_0.147 - gfm_0.260 - gfm_0.165 - gfm_0.78 - gfm_0.137 - gfm_0.1 - gfm_0.175 - gfm_0.171 - gfm_0.181 - gfm_0.67 - gfm_0.179 - gfm_0.167 - gfm_0.351 Each one of these sub-directories refers to one of the fifteen HPO trials that have been selected to continue the pre-training with at most 30 epochs. With each sub-directory associated with a specific HPO trial, the following files can be found: - config.json: file for argument parsing to develop and train an HydraGNN architecture - gfm_0.ID_epoch_N.pk: file with model parameters for HPO ID trial after N epochs of training The ensemble of fifteen GFM architectures was used for (1) ensemble averaging to stabilize the predictions of energy and atomic forces after pre-training for post-processing analysis and (2) ensemble uncertainty quantification (UQ). The code used to develop, pre-train, and load the pre-trained models for post-processing analysis is available on the ORNL-GitHub at the following link: https://github.com/ORNL/HydraGNN/tree/Predictive_GFM_2024

36 MATERIALS SCIENCE↗