Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural operators”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Data-driven emulation of modal aerosol microphysics via neural operator-based modeling

The complexity and the small characteristic scales of aerosol microphysical processes pose a big challenge for accurate and efficient Earth system simulations at regional and global scales. In this work, we construct and evaluate a surrogate model: the aerosol deep operator network (ADON), a physics-inspired dual-net architecture for emulating the aerosol microphysics parameterization suite in the version 2 of the Energy Earth System Model (E3SMv2). The current version of the surrogate model is trained on a dataset comprising 9.8 million samples obtained from a global E3SMv2 simulation with the horizontal resolution of about one degree under cloud-free conditions. Incorporating domain spatial and temporal coordinates, as well as principle components extracted from training data, the dual-net surrogate model effectively captures the intricate representations of aerosol and the relationship with atmospheric state variables, achieving an R-squared score over $$95.7\%$$ for all the lognormal aerosol modes in the extrapolated regime. The validated model provides feature importance of input variables and their impact on the predictive capacity of the surrogate model in relation to the E3SM. The computational cost of online inference time deployed on CPUs and GPUs with lower precisions highlights ADON’s efficiency and potential in robust predictive modeling for large-scale Earth system computations.

Bai, Zhe↗
DOE OSTI2025

Resolving turbulent magnetohydrodynamics: a hybrid operator-diffusion framework

We present a hybrid machine learning framework that combines physics-informed neural operators (PINOs) with score-based generative diffusion models to simulate the full spatio-temporal evolution of two-dimensional, incompressible, resistive magnetohydrodynamic turbulence across a broad range of Reynolds numbers (Re). The framework leverages the equation-constrained generalization capabilities of PINOs to predict coherent, low-frequency dynamics, while a conditional diffusion model stochastically corrects high-frequency residuals, enabling accurate modeling of fully developed turbulence. Trained on a comprehensive ensemble of high-fidelity simulations with Re ϵ {100, 250, 500, 750, 1000, 3000, 10000}, the approach achieves state-of-the-art accuracy in regimes previously inaccessible to deterministic surrogates. At Re = 1000 and 3000, the model faithfully reconstructs the full spectral energy distributions of both velocity and magnetic fields late into the simulation, capturing non-Gaussian statistics, intermittent structures, and cross-field correlations with high fidelity. At extreme turbulence levels (Re = 10 000), it remains the first surrogate capable of recovering the high-wavenumber evolution of the magnetic field, preserving large-scale morphology and enabling statistically meaningful predictions.

Diffusion-Integrated Neural Operators↗
NASA NTRS2004

NASA Experimental Program to Stimulate Competitive Research: South Carolina

The use of an appropriate relationship model is critical for reliable prediction of future urban growth. Identification of proper variables and mathematic functions and determination of the weights or coefficients are the key tasks for building such a model. Although the conventional logistic regression model is appropriate for handing land use problems, it appears insufficient to address the issue of interdependency of the predictor variables. This study used an alternative approach to simulation and modeling urban growth using artificial neural networks. It developed an operational neural network model trained using a robust backpropagation method. The model was applied in the Myrtle Beach region of South Carolina, and tested with both global datasets and areal datasets to examine the strength of both regional models and areal models. The results indicate that the neural network model not only has many theoretic advantages over other conventional mathematic models in representing the complex urban systems, but also is practically superior to the logistic model in its capability to predict urban growth with better - accuracy and less variation. The neural network model is particularly effective in terms of successfully identifying urban patterns in the rural areas where the logistic model often falls short. It was also found from the area-based tests that there are significant intra-regional differentiations in urban growth with different rules and rates. This suggests that the global modeling approach, or one model for the entire region, may not be adequate for simulation of a urban growth at the regional scale. Future research should develop methods for identification and subdivision of these areas and use a set of area-based models to address the issues of multi-centered, intra- regionally differentiated urban growth.

Sutton, Michael A.↗
NASA NTRS1990

Neural networks for aircraft control

Current research in Artificial Neural Networks indicates that networks offer some potential advantages in adaptation and fault tolerance. This research is directed at determining the possible applicability of neural networks to aircraft control. The first application will be to aircraft trim. Neural network node characteristics, network topology and operation, neural network learning and example histories using neighboring optimal control with a neural net are discussed.

Linse, Dennis↗
DOE OSTI2025

Real-time inference and extrapolation with Time-Conditioned UNet: Applications in hypersonic flows, incompressible flows, and global temperature forecasting

Neural Operators are fast and accurate surrogates for nonlinear mappings between functional spaces within training domains. Extrapolation beyond the training domain remains a grand challenge across all application areas. We present Time-Conditioned UNet (TC-UNet) as an operator learning method to solve time-dependent PDEs continuously in time without any temporal discretization, including in extrapolation scenarios. TC-UNet incorporates the temporal evolution of the PDE into its architecture by combining a parameter conditioning approach with the attention mechanism from the Transformer architecture. After training, TC-UNet makes real-time inferences on an arbitrary temporal grid. We demonstrate its extrapolation capability on a climate problem by estimating the global temperature for several years and also for inviscid hypersonic flow around a double cone. We propose different training strategies involving temporal bundling and sub-sampling. We demonstrate performance improvements for several benchmarks, performing extrapolation for long time intervals and zero-shot super-resolution time.

Deep learning↗
DOE OSTI2025

Stacked networks improve physics-informed training: Applications to neural networks and deep operator networks

Physics-informed neural networks and operator networks have shown promise for effectively solving equations modeling physical systems. However, these networks can happen to be difficult or impossible to train accurately. Here, we present a novel multifidelity framework for stacking physics-informed neural networks and operator networks that facilitates training. We successively build a chain of networks, where the output at one step can act as a low-fidelity input for training a longer chain, gradually increasing the expressivity of the learnt model. The equations imposed at each step of the iterative process can be the same or different (akin to simulated annealing). The iterative (stacking) nature of the proposed method allows us to learn progressively features of a solution which could have been hard to learn directly. Through benchmark problems including a nonlinear pendulum, the wave equation, and the viscous Burgers equation, we show how stacking can be used to improve the accuracy and reduce the required size of physics-informed neural networks and operator networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗
DOE OSTI2025

Benchmarking Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this paper, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, use of local memory, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2024

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this work, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, shared local memory accesses, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Deep Learning–Assisted Multiobjective Optimization of Geological CO 2 Storage Performance under Geomechanical Risks

In geological CO 2 storage, designing the optimal well control strategy for CO 2 injection to maximize CO 2 storage while minimizing the associated geomechanical risks is not trivial. This challenge arises due to pressure buildup, CO 2 plume migration, the highly nonlinear nature of geomechanical responses to rock-fluid interaction, and the high computational cost associated with coupled flow and geomechanics simulations. In this paper, we introduce a novel optimization framework to address these challenges. The optimization problem is formulated as follows: maximize total CO 2 storage while minimizing geomechanical risks by adjusting the injection schedules within bounded constraints. The geomechanical risks are primarily driven by injection-induced pressure build-up, which is characterized by ground displacement and the induced microseismicity. We used the Fourier neural operator (FNO)-based deep learning model to construct surrogate models, replacing the time-consuming coupled flow and geomechanics simulations for evaluating the aforementioned objective functions. The developed surrogate models have been incorporated into a multiobjective optimization framework through a genetic algorithm to reduce the computational burden. The proposed optimization framework reduces the computational cost from approximately 2,400 hours, when using objective function evaluations based on physics-based simulations, to around 20 minutes. A set of Pareto-optimal solutions of the proposed workflow yields nontrivial optimal decisions, reducing the microseismicity potential and the vertical displacement. This Pareto front highlights the optimal trade-offs between CO 2 storage amount, safety, and ground displacement, emphasizing the need for careful optimization and management of injection strategies to achieve a balanced outcome. The novelty of this work is twofold. First, we demonstrate the importance of incorporating the minimization of the geomechanical risks as objective functions into the CO 2 storage optimization workflow to mitigate the potential risk of induced microseismicity and ground displacement. Second, we leverage the FNO-based surrogate models to optimize a real-field CO 2 storage operation.

42 ENGINEERING↗
DOE OSTI2024

Physics-constrained machine learning for electrodynamics without gauge ambiguity based on Fourier transformed Maxwell’s equations

We utilize a Fourier transformation-based representation of Maxwell’s equations to develop physics-constrained neural networks for electrodynamics without gauge ambiguity, which we label the Fourier–Helmholtz–Maxwell neural operator method. In this approach, both of Gauss’s laws and Faraday’s law are built in as hard constraints, as well as the longitudinal component of Ampère–Maxwell in Fourier space, assuming the continuity equation. An encoder–decoder network acts as a solution operator for the transverse components of the Fourier transformed vector potential, $\hat{A}_⟂(k,t)$, whose two degrees of freedom are used to predict the electromagnetic fields. This method was tested on two electron beam simulations. Among the models investigated, it was found that a U-Net architecture exhibited the best performance as it trained quicker, was more accurate and generalized better than the other architectures examined. We demonstrate that our approach is useful for solving Maxwell’s equations for the electromagnetic fields generated by intense relativistic charged particle beams and that it generalizes well to unseen test data, while being orders of magnitude quicker than conventional simulations. We show that the model can be re-trained to make highly accurate predictions in as few as 20 epochs on a previously unseen data set.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Spatiotemporal forecasting of the edge localized modes in tokamak plasmas using neural networks

Artificial intelligence techniques have been increasingly adopted by the plasma and fusion science to address problems like plasma reconstruction, surrogate modeling, and tokamak/stellarator optimization. A key focus in sustained fusion research is the prediction and mitigation of edge-localized-modes (ELMs), instabilities that occur in short, periodic bursts and can cause erosion to the tokamak vessel wall. Recent research has demonstrated the power of neural networks in approximating continuous functions. In this work, we build spatiotemporal forecasting models that can predict the onset of ELMs and their evolution at early stages. We leverage recent advances in generative modeling, sequence-to-sequence modeling, and Fourier neural operators to propose architectures and training strategies that can learn to forecast short to long term dynamics of the noisy signals due to ELMs. We benchmark the developed model against a state-of-the-art foundation model using the beam emission spectroscopy (BES) data that captures the plasma fluctuations due to ELMs over a 8 x 8 spatial grid. Our models demonstrate high accuracy, outperforming the baselines, in predicting the evolution of BES signals during ELM events. Furthermore, the developed models exhibit high accuracy in predicting the rapid rise and relaxation of the signals due to ELMs within 30–80 µs.

edge localized modes↗
DOE OSTI2025

Non-destructive structural characterization of graphite components using mechanical resonance and deep learning

As compared to conventional nuclear reactors, microreactors have the potential to significantly reduce construction timelines and capital costs, decreasing the barriers for advanced nuclear reactor technologies. However, the lower power output of these microreactors (typically < 20 MWe) creates challenging economics if operation and maintenance costs cannot be sufficiently reduced. The compact size of these designs presents an opportunity for comprehensive in-situ structural health monitoring to provide real-time feedback in order to reduce operational costs associated with maintenance and downtime. Many microreactor concepts use graphite for both in-core neutron moderation and as a structural material, which has typically required some form of periodic and laborious inspection. This report provides a description and assessment of recent work with graphite to couple acoustic-based experimental measurements and characterization with machine learning models to mature structural health monitoring capabilities and generate benefits for the nuclear microreactor industry. With resilient embedded sensors in development in other programs funded by the US Department of Energy’s Office of Nuclear Energy and elsewhere, the work described herein builds upon previously funded efforts to mature non-destructive testing technology that relates measured vibrational signatures to structural changes, using a combination of new experimental measurements and machine learning processing. Building on past successful demonstrations of predictive workflows to identify structural changes in a hexagonal stainless steel test article with excellent acoustic propagation, we first performed baseline characterization on graphite samples with canonical geometries to ensure compatibility and confidence in the applied techniques for a material with distinctly different mechanical properties. In contrast to efforts in previous years, we worked exclusively with unidirectional vibration data that is more comparable to those expected from the existing embedded sensor technologies which are suitable for deployment in a reactor setting. Established acoustic and modern machine-learning-based characterization approaches were applied to the resulting datasets from these simple geometries. Both approaches were found to be highly capable of detecting even small geometric irregularities amongst nominally identical samples. As such, we then moved to testing these approaches for detection of artificial local stress perturbations introduced into a more complex geometry: a hexagonal block with drilled holes. A main outcome of this work is that a generalizable ML workflow can be used to detect and predict the characteristics of small artificial anomalies in a graphite component with a relevant geometry. While this work was performed using surficial vibration data, we expect the approach to be flexible and viable for other monitoring scenarios, such as those with different arrangements or types of sensor arrays. As compared to previously funded efforts, an existing ML workflow based on neural networks was enhanced through the addition of recently developed Fourier neural operators. As applied to previously collected and new vibration datasets, prediction accuracies of anomaly characterizations were greatly improved with minimal added computational cost. As trained on small durations of vibration data (tens of seconds) collected over a realistic number of locations, the model was able to reliably determine the presence of a subtle stress anomaly and begin to provide location estimates. Such an approach is likely to be viable for more relevant reactor damage scenarios for graphite components, such as progressive crack growth or creep.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.

Approximate Bayesian inference↗
DOE OSTI2024

Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations

We propose a reduced-order deep-learning surrogate model for dynamic systems described by time-dependent partial differential equations. This method employs space–time Karhunen–Loève expansions (KLEs) of the state variables and space-dependent KLEs of space-varying parameters to identify the reduced (latent) dimensions. Subsequently, a deep neural network (DNN) is used to map the parameter latent space to the state variable latent space. An approximate Bayesian method is developed for uncertainty quantification (UQ) in the proposed KL-DNN surrogate model. The KL-DNN method is tested for the linear advection–diffusion and nonlinear diffusion equations, and the Bayesian approach for UQ is compared with the deep ensembling (DE) approach, commonly used for quantifying uncertainty in DNN models. It was found that the approximate Bayesian method provides a more informative distribution of the PDE solutions in terms of the coverage of the reference PDE solutions (the percentage of nodes where the reference solution is within the confidence interval predicted by the UQ methods) and log predictive probability. The DE method is found to underestimate uncertainty and introduce bias. For the nonlinear diffusion equation, we compare the KL-DNN method with the Fourier Neural Operator (FNO) method and find that KL-DNN is 10% more accurate and needs less training time than the FNO method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Towards robust surrogate models: Benchmarking machine learning approaches to expediting phase field simulations of brittle fracture

Data-driven approaches have the potential to make modeling complex, nonlinear physical phenomena significantly more computationally tractable. For example, computational modeling of fracture is a core challenge where machine learning techniques have the potential to provide a much needed speedup that would enable progress in areas such as multi-scale modeling and uncertainty quantification. Currently, phase field modeling (PFM) of fracture is one such approach that offers a convenient variational formulation to model crack nucleation, branching and propagation. To date, machine learning techniques have shown promise in approximating PFM simulations. While standard fracture benchmarks represent realistic scenarios frequently observed in practice, they typically do not provide sufficiently challenging tests for data-driven methods. Here, to address this gap, we introduce a challenging dataset based on PFM simulations designed to benchmark and advance ML methods for fracture modeling. This dataset includes three energy decomposition methods, two boundary conditions, and 1000 random initial crack configurations for a total of 6000 simulations. Each sample contains 100 time steps capturing the temporal evolution of the crack field. Alongside this dataset, we also implement and evaluate Physics Informed Neural Networks (PINN), Fourier Neural Operators (FNO), and UNet models as baselines, and explore the impact of ensembling strategies on prediction accuracy. With this combination of our dataset and baseline models drawn from the literature we aim to provide a standardized and challenging benchmark for evaluating machine learning approaches to solid mechanics. Our results highlight both the promise and limitations of popular current models, and demonstrate the utility of this dataset as a testbed for advancing machine learning in fracture mechanics research.

Benchmark dataset↗
DOE OSTI2024

Toward a Machine Learning Approach to Interpreting X-ray Spectra of Trace Impurities by Converting XANES to EXAFS

The fact that the photoabsorption spectrum of a material contains information about the atomic structure, commonly understood in terms of multiple scattering theory, is the basis of the popular extended X-ray absorption spectroscopy (EXAFS) technique. How much of the same structural information is present in other complementary spectroscopic signals is not obvious. Here we use a machine learning approach to demonstrate that within theoretical models that accurately predict the EXAFS signal, the extended near-edge region does indeed contain the EXAFS-accessible structural information. We do this by exhibiting deep operator neural networks (DeepONets) that have learned the relationship between the extended and near edge portions of the X-ray absorption spectrum to predict the former from the latter. We find that we can accurately predict the EXAFS spectrum between 6 and 14 Å –1 from the first 6 Å –1 (≈100 eV) of the absorption spectrum of Cu 2 + substitutional defects in the Fe 3+ mineral hematite (α-Fe 2 O 3 ). This surprising finding implies that theoretical analyses of X-ray absorption spectra could be implemented that extract the same conclusions as high-quality EXAFS studies from spectra collected over a much smaller range of photon energies. This relaxes a host of experimental limitations related to the X-ray source and measurement sample, including collection time, minimum dopant concentration, source brilliance, and energy range. We describe the theoretical data sets and DeepONet construction and show that the resulting DeepONets produce EXAFS that recovers linear combination fits to experimental data with accuracy approaching the original ab initio calculations. We discuss the implications of our findings for minor constituent characterization and for understanding the information content of spectroscopic data more broadly, including how this approach might be applied to measured experimental spectra. In conclusion, to encourage similar efforts, the simulated X-ray spectra, machine learning, and fitting code are publicly available.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Efficient and generalizable nested Fourier-DeepONet for three-dimensional geological carbon sequestration

Geological carbon sequestration (GCS) involves injecting CO2 into subsurface geological formationsfor permanent storage. Numerical simulations could guide decisions in GCS projects by predictingCO 2 migration pathways and the pressure distribution in storage formation. However, these simula-tions are often computationally expensive due to highly coupled physics and large spatial-temporalsimulation domains. Surrogate modelling with data-driven machine learning has become a promis-ing alternative to accelerate physics-based simulations. Among these, the Fourier neural operator(FNO) has been applied to three-dimensional synthetic subsurface models. Despite its good accuracyin simulating CO 2 plume migration, it requires large computational resources in training and alsolacks generalizability. Here, to further improve performance, we have developed a nested Fourier-DeepONet by combining the expressiveness of the FNO with the modularity of a deep operatornetwork (DeepONet). This new framework is twice as efficient as a nested FNO for training and has atleast 80% lower GPU memory requirement due to its flexibility to treat temporal coordinates sepa-rately. These performance improvements are achieved without compromising prediction accuracy.In addition, the generalization and extrapolation ability of nested Fourier-DeepONet beyond thetraining range has been thoroughly evaluated. Nested Fourier-DeepONet outperformed the nestedFNO for extrapolation in time with more than 50% reduced error. It also exhibited good extrapolationaccuracy beyond the training range in terms of reservoir properties, number of wells, and injectionrate.

Lee, Jonathan E. [Department of Chemical and Envir↗