Bridging scales in multiscale bubble growth dynamics with correlated fluctuations using neural operator learning
Explore the source record for details and available documents.
Rayleigh-Plesset equation↗
SEARCH · Search NASA
Results for “Neural Operator Learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
TANTE: Time-adaptive operator learning via neural Taylor expansion
Operator learning for time-dependent partial differential equations (PDEs) has seen rapid progress in recent years, enabling efficient approximation of complex spatiotemporal dynamics. However, most existing methods rely on fixed time step sizes during rollout, which limits their ability to adapt to varying temporal complexity and often leads to error accumulation. In this work, we propose the Time-Adaptive Transformer with Neural Taylor Expansion (TANTE), a novel operator-learning framework that produces continuous-time predictions with adaptive step sizes. TANTE predicts future states by performing a Taylor expansion at the current state, where neural networks learn both the higher-order temporal derivatives and the local radius of convergence. This allows the model to dynamically adjust its rollout based on the local behavior of the solution, thereby reducing cumulative error and improving computational efficiency. We demonstrate the effectiveness of TANTE across a wide range of PDE benchmarks, achieving superior accuracy and adaptability compared to fixed-step baselines, delivering accuracy gains of 60-80 % and speed-ups of 30-40 % at inference time.
97 MATHEMATICS AND COMPUTING↗
MIONet: Learning Multiple-Input Operators via Tensor Product
As an emerging paradigm in scientific machine learning, neural operators aim to learn operators, via neural networks, that map between infinite-dimensional function spaces. Several neural operators have been recently developed. However, all the existing neural operators are only designed to learn operators defined on a single Banach space; i.e., the input of the operator is a single function. Here, for the first time, we study the operator regression via neural networks for multiple-input operators defined on the product of Banach spaces. We first prove a universal approximation theorem of continuous multiple-input operators. We also provide a detailed theoretical analysis including the approximation error, which provides guidance for the design of the network architecture. Based on our theory and a low-rank approximation, we propose a novel neural operator, MIONet, to learn multiple-input operators. MIONet consists of several branch nets for encoding the input functions and a trunk net for encoding the domain of the output function. Here, we demonstrate that MIONet can learn solution operators involving systems governed by ordinary and partial differential equations. In our computational examples, we also show that we can endow MIONet with prior knowledge of the underlying system, such as linearity and periodicity, to further improve accuracy.
97 MATHEMATICS AND COMPUTING↗
Scaling Field-Theoretic Simulation for Multicomponent Mixtures with Neural Operators
Multicomponent polymer mixtures are ubiquitous in biological self-organization but are notoriously difficult to study computationally. Plagued by both slow single molecule relaxation times and slow equilibration within dense mixtures, molecular dynamics simulations are typically infeasible at the spatial scales required to study the stability of mesophase structure. Polymer field theories offer an attractive alternative, but analytical calculations are only tractable for mean-field theories and nearby perturbations, constraints that become especially problematic for fluctuation-induced effects such as coacervation. Here, we show that a recently developed technique for obtaining numerical solutions to partial differential equations based on operator learning, neural operators, lends itself to a highly scalable training strategy by parallelizing per-species operator maps. We illustrate the efficacy of our approach on six-component mixtures with randomly selected compositions and that it significantly outperforms the state-of-the-art pseudospectral integrators for field-theoretic simulations, especially as polymer lengths become long.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Model-parallel Fourier neural operators as learned surrogates for large-scale parametric PDEs
Fourier neural operators (FNOs) are a recently introduced neural network architecture for learning solution operators of partial differential equations (PDEs), which have been shown to perform significantly better than comparable deep learning approaches. Once trained, FNOs can achieve speed-ups of multiple orders of magnitude over conventional numerical PDE solvers. However, due to the high dimensionality of their input data and network weights, FNOs have so far only been applied to two-dimensional or small three-dimensional problems. To remove this limited problem-size barrier, we propose a model-parallel version of FNOs based on domain-decomposition of both the input data and network weights. Here, we demonstrate that our model-parallel FNO is able to predict time-varying PDE solutions of over 2.6 billion variables on Perlmutter using up to 512 A100 GPUs and show an example of training a distributed FNO on the Azure cloud for simulating multiphase CO 2 dynamics in the Earth’s subsurface.
58 GEOSCIENCES↗
Bridging Hydrological Ensemble Simulation and Learning Using Deep Neural Operators
Ensemble-based simulation and learning (ESnL) has long been used in hydrology for parameter inference, but computational demands of process-based ESnL can be quite high. To address this issue, we propose a deep neural operator learning approach. Neural operators are generic machine learning algorithms that can learn functional mappings between infinite-dimensional spaces, providing a highly flexible tool for scientific machine learning. Our approach is built upon DeepONet, a specific deep neural operator, and is designed to address several common problems in hydrology, namely, model parameter estimation, prediction at ungaged locations, and uncertainty quantification. Here we demonstrate the effectiveness of our DeepONet-based workflow using an existing large model ensemble created for an eastern U.S. watershed that is instrumented with 10 streamflow gages. Results suggest DeepONet achieves high efficiency in learning an ML surrogate model from the model ensemble, with the modified Kling-Gupta Efficiency exceeding 0.9 on holdout test sets. Parameter inference, carried out using the trained DeepONet surrogate model and genetic algorithm, also yields robust results. Additionally, we formulate and train a separate DeepONet model for physics-informed, seq-to-seq streamflow forecasting, which further reduces biases in the pre-trained DeepONet surrogate model. While this study focuses primarily on a single watershed, our approach is general and may be extended to enable learning from model ensembles across multiple basins or models. Thus, this research represents a significant contribution to the application of hybrid machine learning in hydrology.
54 ENVIRONMENTAL SCIENCES↗
Application of adjoint operators to neural learning
A technique for the efficient analytical computation of such parameters of the neural architecture as synaptic weights and neural gain is presented as a single solution of a set of adjoint equations. The learning model discussed concentrates on the adiabatic approximation only. A problem of interest is represented by a system of N coupled equations, and then adjoint operators are introduced. A neural network is formalized as an adaptive dynamical system whose temporal evolution is governed by a set of coupled nonlinear differential equations. An approach based on the minimization of a constrained neuromorphic energylike function is applied, and the complete learning dynamics are obtained as a result of the calculations.
Barhen, J.↗
Improved Adjoint-Operator Learning For A Neural Network
Improved method of adjoint-operator learning reduces amount of computation and associated computational memory needed to make electronic neural network learn temporally varying pattern (e.g., to recognize moving object in image) in real time. Method extension of method described in "Adjoint-Operator Learning for a Neural Network" (NPO-18352).
Toomarian, Nikzad↗
Operator learning for energy-efficient building ventilation control with computational fluid dynamics simulation of a real-world classroom
Energy-efficient ventilation control plays an important role in reducing building energy consumption while ensuring occupant health and comfort. While Computational Fluid Dynamics (CFD) simulations provide detailed and physically accurate representations of indoor airflow, their high computational cost limits their use in real-time building control. In this work, we present a neural operator learning framework that combines the physical accuracy of CFD with the computational efficiency of machine learning to enable building ventilation control with the high-fidelity fluid dynamics models. Our method jointly optimizes the airflow supply rates and vent angles to reduce energy use and adhere to air quality constraints. We train an ensemble of neural operator transformer models to learn the mapping from building control actions to airflow fields using high-resolution CFD data. This learned neural operator is then embedded in an optimization-based control framework for building ventilation control. Experimental results show that our approach achieves significant energy savings compared to maximum airflow rate control, rule-based control, as well as data-driven control methods using spatially averaged CO 2 prediction and deep learning–based reduced-order models, while consistently maintaining safe indoor air quality. These results highlight the practicality and scalability of our method in maintaining energy efficiency and indoor air quality in real-world buildings.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
On the influence of over-parameterization in manifold based surrogates and deep neural operators
Constructing accurate and generalizable approximators (surrogate models) for complex physico-chemical processes exhibiting highly non-smooth dynamics is challenging. The main question is what type of surrogate models we should construct and should these models be under-parameterized or over-parameterized. In this work, we propose new developments and perform comparisons for two promising approaches: manifold-based polynomial chaos expansion (m-PCE) and the deep neural operator (DeepONet), and we examine the effect of over-parameterization on generalization. While m-PCE enables the construction of a mapping by first identifying low-dimensional embeddings of the input functions, parameters, and quantities of interest (QoIs), a neural operator learns the nonlinear mapping via the use of deep neural networks. Here, we demonstrate the performance of these methods in terms of generalization accuracy by solving the 2D time-dependent Brusselator reaction-diffusion system with uncertainty sources, modeling an autocatalytic chemical reaction between two species. We first propose an extension of the m-PCE by constructing a mapping between latent spaces formed by two separate embeddings of the input functions and the output QoIs. To further enhance the accuracy of the DeepONet, we introduce weight self-adaptivity in the loss function. We demonstrate that the performance of m-PCE and DeepONet is comparable for cases of relatively smooth input-output mappings. However, when highly non-smooth dynamics is considered, DeepONet shows higher approximation accuracy. We also find that for m-PCE, modest over-parameterization leads to better generalization, both within and outside of distribution, whereas aggressive over-parameterization leads to over-fitting. In contrast, an even highly over-parameterized DeepONet leads to better generalization for both smooth and non-smooth dynamics. Furthermore, we compare the performance of the above models with another recently proposed operator learning model, the Fourier Neural Operator, and show that its over-parameterization also leads to better generalization. Taken together, our studies show that m-PCE can provide very good accuracy at very low training cost, whereas a highly over-parameterized DeepONet can provide better accuracy and robustness to noise but at higher training cost. In both methods, the inference cost is negligible.
97 MATHEMATICS AND COMPUTING↗
Learning nonlinear operators in latent spaces for real-time predictions of complex dynamics in physical systems
Abstract Predicting complex dynamics in physical applications governed by partial differential equations in real-time is nearly impossible with traditional numerical simulations due to high computational cost. Neural operators offer a solution by approximating mappings between infinite-dimensional Banach spaces, yet their performance degrades with system size and complexity. We propose an approach for learning neural operators in latent spaces, facilitating real-time predictions for highly nonlinear and multiscale systems on high-dimensional domains. Our method utilizes the deep operator network architecture on a low-dimensional latent space to efficiently approximate underlying operators. Demonstrations on material fracture, fluid flow prediction, and climate modeling highlight superior prediction accuracy and computational efficiency compared to existing methods. Notably, our approach enables approximating large-scale atmospheric flows with millions of degrees, enhancing weather and climate forecasts. Here we show that the proposed approach enables real-time predictions that can facilitate decision-making for a wide range of applications in science and engineering.
97 MATHEMATICS AND COMPUTING↗
Reliable extrapolation of deep neural operators informed by physics or sparse observations
Deep neural operators can learn nonlinear mappings between infinite-dimensional function spaces via deep neural networks. As promising surrogate solvers of partial differential equations (PDEs) for real-time prediction, deep neural operators such as deep operator networks (DeepONets) provide a new simulation paradigm in science and engineering. Pure data-driven neural operators and deep learning models, in general, are usually limited to interpolation scenarios, where new predictions utilize inputs within the support of the training set. However, in the inference stage of real-world applications, the input may lie outside the support, i.e., extrapolation is required, which may result to large errors and unavoidable failure of deep learning models. Here, we address this challenge of extrapolation for deep neural operators. First, we systematically investigate the extrapolation behavior of DeepONets by quantifying the extrapolation complexity, via the 2-Wasserstein distance between two function spaces and propose a new strategy of bias–variance trade-off for extrapolation with respect to model capacity. Subsequently, we develop a complete workflow, including extrapolation determination, and we propose five reliable learning methods that guarantee a safe prediction under extrapolation by requiring additional information—the governing PDEs of the system or sparse new observations. The proposed methods are based on either fine-tuning a pre-trained DeepONet or multifidelity learning. We demonstrate the effectiveness of the proposed framework for various types of parametric PDEs. Furthermore, our systematic comparisons provide practical guidelines for selecting a proper extrapolation method depending on the available information, desired accuracy, and required inference speed.
42 ENGINEERING↗
Adjoint-Operator Learning For A Neural Network
Electronic neural networks made to synthesize initially unknown mathematical models of time-dependent phenomena or to learn temporally evolving patterns by use of algorithms based on adjoint operators. Algorithms less complicated, involve less computation and solve learning equations forward in time possibly simultaneously with equations of evolution of neural network, thereby both increasing computational efficiency and making real-time applications possible.
Barhen, Jacob↗
Mixture of neural operator experts for learning boundary conditions and model selection
Explore the source record for details and available documents.
Patel, Ravi Ghanshyam [Sandia National Laboratorie↗
Peridynamic neural operators: A data-driven nonlocal constitutive model for complex material responses
Neural operators, which can act as implicit solution operators of hidden governing equations, have recently become popular tools for learning the responses of complex real-world physical systems. Nevertheless, most neural operator applications have thus far been data-driven and neglect the intrinsic preservation of fundamental physical laws in data. Here, in this work, we introduce a novel integral neural operator architecture called the Peridynamic Neural Operator (PNO) that learns a nonlocal constitutive law from data. This neural operator provides a forward model in the form of state-based peridynamics, with objectivity and momentum balance laws automatically guaranteed. As applications, we demonstrate the expressivity and efficacy of our model in learning complex material behaviors from both synthetic and experimental data sets. We also compare the performances with baseline models that use predefined constitutive laws. We show that, owing to its ability to capture complex responses, our learned neural operator achieves improved accuracy and efficiency. Moreover, by preserving the essential physical laws within the neural network architecture, the PNO is robust in treating noisy data. The method shows generalizability to different domain configurations, external loadings, and discretizations.
42 ENGINEERING↗
Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study
Neural scaling laws play a pivotal role in the performance of deep neural networks and have been observed in a wide range of tasks. However, a complete theoretical framework for understanding these scaling laws remains underdeveloped. In this paper, we explore the neural scaling laws for deep operator networks, which involve learning mappings between function spaces, with a focus on the Chen and Chen style architecture. These approaches, which include the popular Deep Operator Network (DeepONet), approximate the output functions using a linear combination of learnable basis functions and coefficients that depend on the input functions. We establish a theoretical framework to quantify the neural scaling laws by analyzing its approximation and generalization errors. We articulate the relationship between the approximation and generalization errors of deep operator networks and key factors such as network model size and training data size. Moreover, we address cases where input functions exhibit low-dimensional structures, allowing us to derive tighter error bounds. These results also hold for deep ReLU networks and other similar structures. Our results offer a partial explanation of the neural scaling laws in operator learning and provide a theoretical foundation for their applications.
97 MATHEMATICS AND COMPUTING↗
Learning two-phase microstructure evolution using neural operators and autoencoder architectures
Abstract Phase-field modeling is an effective but computationally expensive method for capturing the mesoscale morphological and microstructure evolution in materials. Hence, fast and generalizable surrogate models are needed to alleviate the cost of computationally taxing processes such as in optimization and design of materials. The intrinsic discontinuous nature of the physical phenomena incurred by the presence of sharp phase boundaries makes the training of the surrogate model cumbersome. We develop a framework that integrates a convolutional autoencoder architecture with a deep neural operator (DeepONet) to learn the dynamic evolution of a two-phase mixture and accelerate time-to-solution in predicting the microstructure evolution. We utilize the convolutional autoencoder to provide a compact representation of the microstructure data in a low-dimensional latent space. After DeepONet is trained in the latent space, it can be used to replace the high-fidelity phase-field numerical solver in interpolation tasks or to accelerate the numerical solver in extrapolation tasks.
36 MATERIALS SCIENCE↗