Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural operator”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.

Approximate Bayesian inference↗
DOE OSTI2024

Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations

We propose a reduced-order deep-learning surrogate model for dynamic systems described by time-dependent partial differential equations. This method employs space–time Karhunen–Loève expansions (KLEs) of the state variables and space-dependent KLEs of space-varying parameters to identify the reduced (latent) dimensions. Subsequently, a deep neural network (DNN) is used to map the parameter latent space to the state variable latent space. An approximate Bayesian method is developed for uncertainty quantification (UQ) in the proposed KL-DNN surrogate model. The KL-DNN method is tested for the linear advection–diffusion and nonlinear diffusion equations, and the Bayesian approach for UQ is compared with the deep ensembling (DE) approach, commonly used for quantifying uncertainty in DNN models. It was found that the approximate Bayesian method provides a more informative distribution of the PDE solutions in terms of the coverage of the reference PDE solutions (the percentage of nodes where the reference solution is within the confidence interval predicted by the UQ methods) and log predictive probability. The DE method is found to underestimate uncertainty and introduce bias. For the nonlinear diffusion equation, we compare the KL-DNN method with the Fourier Neural Operator (FNO) method and find that KL-DNN is 10% more accurate and needs less training time than the FNO method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Towards robust surrogate models: Benchmarking machine learning approaches to expediting phase field simulations of brittle fracture

Data-driven approaches have the potential to make modeling complex, nonlinear physical phenomena significantly more computationally tractable. For example, computational modeling of fracture is a core challenge where machine learning techniques have the potential to provide a much needed speedup that would enable progress in areas such as multi-scale modeling and uncertainty quantification. Currently, phase field modeling (PFM) of fracture is one such approach that offers a convenient variational formulation to model crack nucleation, branching and propagation. To date, machine learning techniques have shown promise in approximating PFM simulations. While standard fracture benchmarks represent realistic scenarios frequently observed in practice, they typically do not provide sufficiently challenging tests for data-driven methods. Here, to address this gap, we introduce a challenging dataset based on PFM simulations designed to benchmark and advance ML methods for fracture modeling. This dataset includes three energy decomposition methods, two boundary conditions, and 1000 random initial crack configurations for a total of 6000 simulations. Each sample contains 100 time steps capturing the temporal evolution of the crack field. Alongside this dataset, we also implement and evaluate Physics Informed Neural Networks (PINN), Fourier Neural Operators (FNO), and UNet models as baselines, and explore the impact of ensembling strategies on prediction accuracy. With this combination of our dataset and baseline models drawn from the literature we aim to provide a standardized and challenging benchmark for evaluating machine learning approaches to solid mechanics. Our results highlight both the promise and limitations of popular current models, and demonstrate the utility of this dataset as a testbed for advancing machine learning in fracture mechanics research.

Benchmark dataset↗
DOE OSTI2024

Toward a Machine Learning Approach to Interpreting X-ray Spectra of Trace Impurities by Converting XANES to EXAFS

The fact that the photoabsorption spectrum of a material contains information about the atomic structure, commonly understood in terms of multiple scattering theory, is the basis of the popular extended X-ray absorption spectroscopy (EXAFS) technique. How much of the same structural information is present in other complementary spectroscopic signals is not obvious. Here we use a machine learning approach to demonstrate that within theoretical models that accurately predict the EXAFS signal, the extended near-edge region does indeed contain the EXAFS-accessible structural information. We do this by exhibiting deep operator neural networks (DeepONets) that have learned the relationship between the extended and near edge portions of the X-ray absorption spectrum to predict the former from the latter. We find that we can accurately predict the EXAFS spectrum between 6 and 14 Å –1 from the first 6 Å –1 (≈100 eV) of the absorption spectrum of Cu 2 + substitutional defects in the Fe 3+ mineral hematite (α-Fe 2 O 3 ). This surprising finding implies that theoretical analyses of X-ray absorption spectra could be implemented that extract the same conclusions as high-quality EXAFS studies from spectra collected over a much smaller range of photon energies. This relaxes a host of experimental limitations related to the X-ray source and measurement sample, including collection time, minimum dopant concentration, source brilliance, and energy range. We describe the theoretical data sets and DeepONet construction and show that the resulting DeepONets produce EXAFS that recovers linear combination fits to experimental data with accuracy approaching the original ab initio calculations. We discuss the implications of our findings for minor constituent characterization and for understanding the information content of spectroscopic data more broadly, including how this approach might be applied to measured experimental spectra. In conclusion, to encourage similar efforts, the simulated X-ray spectra, machine learning, and fitting code are publicly available.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Efficient and generalizable nested Fourier-DeepONet for three-dimensional geological carbon sequestration

Geological carbon sequestration (GCS) involves injecting CO2 into subsurface geological formationsfor permanent storage. Numerical simulations could guide decisions in GCS projects by predictingCO 2 migration pathways and the pressure distribution in storage formation. However, these simula-tions are often computationally expensive due to highly coupled physics and large spatial-temporalsimulation domains. Surrogate modelling with data-driven machine learning has become a promis-ing alternative to accelerate physics-based simulations. Among these, the Fourier neural operator(FNO) has been applied to three-dimensional synthetic subsurface models. Despite its good accuracyin simulating CO 2 plume migration, it requires large computational resources in training and alsolacks generalizability. Here, to further improve performance, we have developed a nested Fourier-DeepONet by combining the expressiveness of the FNO with the modularity of a deep operatornetwork (DeepONet). This new framework is twice as efficient as a nested FNO for training and has atleast 80% lower GPU memory requirement due to its flexibility to treat temporal coordinates sepa-rately. These performance improvements are achieved without compromising prediction accuracy.In addition, the generalization and extrapolation ability of nested Fourier-DeepONet beyond thetraining range has been thoroughly evaluated. Nested Fourier-DeepONet outperformed the nestedFNO for extrapolation in time with more than 50% reduced error. It also exhibited good extrapolationaccuracy beyond the training range in terms of reservoir properties, number of wells, and injectionrate.

Lee, Jonathan E. [Department of Chemical and Envir↗
DOE OSTI2025

Multi-Stage FNO

This is the code developed for paper - Reducing Frequency Bias of Fourier Neural Operators in 3D Seismic Wavefield Simulations Through Multi-Stage Training (https://arxiv.org/abs/2503.02023), it contains training the multi-stage FNO model to generate the seismic forward simulations in 3D medium.

Kong, Qingkai [Lawrence Livermore National Laborat↗
DOE OSTI2025

Sparsified Time-dependent PDEs FNO (STFNO) v1.0.0

STFNO (Sparsified Time-dependent PDEs FNO code) is an extension of the popular Fourier Neural Operator (FNO) architecture to the solution of coupled systems of time-dependent partial differential equations. STFNO leverages the sparsified dependencies on the field quantities based on the semi-discretiezed form of the PDEs, enabling significant reduction in the number of model parameters. STFNO has been extensively tested on two fusion simulation codes, NIMROD and GTC, and can be easily tailored to other systems of PDEs.

Rahman, Mustafa [Lawrence Berkeley National Labora↗
DOE OSTI2026

SPUS-Small-PDE-U-net-Solver

Small PDE U-Net Solver (SPUS) is a compact and efficient foundation model (FM) designed as a unified neural operator for solving a wide range of partial differentialequations (PDEs). SPUS leverages a lightweight residual U-Net-based architecture as a foundation model architecture. To enable effective learning in this minimalist framework, SPUS utilizes a simple yet powerful auto-regressive pretraining strategy which closely replicates the behavior of numerical solvers to learn the underlying physics. SPUS is designed to be pretrained on a diverse set of fluid dynamics PDEs from public benchmark datasets.

Siddik, Abu↗
DOE OSTI2024

Enhancing Fluid Flow Pressure and Saturation Prediction Accuracy and Reducing Uncertainty with Committee Machine – Illinois Basin Decatur Project (IBDP) as a Case Study

Presentation at the 17th International Conference on Greenhouse Gas Control Technologies GHGT-17 held in Calgary, Canada, October 20-24, 2024. Carbon capture and storage (CCS) is a way to play a critical role in the global transition to a low-emission economy. Current progress is hampered by a number of factors, among which the lack of risk-informed design tools and decision support frameworks is seen as a major roadblock. Significant interest exists in using artificial intelligence to accelerate CCS site feasibility studies, as well as to facilitate the permit application process. Existing works commonly train a single deep learning model. This work investigates the feasibility of using a conventional ensemble learning (committee machine) technique to further improve prediction accuracy. Ensemble-based algorithms generally improve over individual base learners in terms of robustness and accuracy. Deep ensembles, however, are time-consuming to create and train. A pragmatic question is whether small-sized ensembles may lead to prediction improvement. Here we evaluated the efficacy of an ensemble learning technique using the latent spectral model (LSM), an efficient deep neural operator algorithm, as base learners. Preliminary results, obtained using the Illinois Basin-Decatur Project (IBDP) carbon sequestration data/model, show that small-sized ensembles can improve prediction over the base learners, achieving prediction accuracy of ~1.6 psi root mean square error (RMSE) on pressure (relative the average reservoir pressure of 3150 psi), and less than 1.3% for saturation.

Sun, Alexander↗
DOE OSTI2024

Enhancing Fluid Flow Pressure and Saturation Prediction Accuracy and Reducing Uncertainty with Committee Machine – Illinois Basin Decatur Project (IBDP) as a Case Study

This is the conference paper accompanying an oral presentation at the 17th International Conference on Greenhouse Gas Control Technologies GHGT-17 held in Calgary, Canada, October 20-24, 2024. Carbon capture and storage (CCS) is a way to play a critical role in the global transition to a low-emission economy. Current progress is hampered by a number of factors, among which the lack of risk-informed design tools and decision support frameworks is seen as a major roadblock. Significant interest exists in using artificial intelligence to accelerate CCS site feasibility studies, as well as to facilitate the permit application process. Existing works commonly train a single deep learning model. This work investigates the feasibility of using a conventional ensemble learning (committee machine) technique to further improve prediction accuracy. Ensemble-based algorithms generally improve over individual base learners in terms of robustness and accuracy. Deep ensembles, however, are time-consuming to create and train. A pragmatic question is whether small-sized ensembles may lead to prediction improvement. Here we evaluated the efficacy of an ensemble learning technique using the latent spectral model (LSM), an efficient deep neural operator algorithm, as base learners. Preliminary results, obtained using the Illinois Basin-Decatur Project (IBDP) carbon sequestration data/model, show that small-sized ensembles can improve prediction over the base learners, achieving prediction accuracy of ~1.6 psi root mean square error (RMSE) on pressure (relative the average reservoir pressure of 3150 psi), and less than 1.3% for saturation.

Sun, Alexander↗
NASA NTRS2009

Evolvable synthetic neural system

An evolvable synthetic neural system includes an evolvable neural interface operably coupled to at least one neural basis function. Each neural basis function includes an evolvable neural interface operably coupled to a heuristic neural system to perform high-level functions and an autonomic neural system to perform low-level functions. In some embodiments, the evolvable synthetic neural system is operably coupled to one or more evolvable synthetic neural systems in a hierarchy.

Curtis, Steven A.↗
NASA NTRS1995

Improved Adjoint-Operator Learning For A Neural Network

Improved method of adjoint-operator learning reduces amount of computation and associated computational memory needed to make electronic neural network learn temporally varying pattern (e.g., to recognize moving object in image) in real time. Method extension of method described in "Adjoint-Operator Learning for a Neural Network" (NPO-18352).

Toomarian, Nikzad↗
DOE OSTI2025

Physics-Informed Recurrent Neural Networks to Predict Reactor Operations of the AGN-201 Nuclear Reactor

4 page paper submitted to ANS Student conference. Summary of paper similar to the following abstract: The ability to predict how a reactor will operate, understand when anomalous conditions arise, and ensure a reactor is being operated as expected is crucial for deploying new nuclear facilities. Digital twins serve as a unique solution to recognizing reactor behavior; however, they require data to be useful. For next-generation reactors, this data may not currently be available. To explore how synthetic physics-informed reactor data can be used to predict reactor operations, a recurrent neural network was implemented for the Idaho State University AGN-201 digital twin. The goal of this work is to determine how synthetic data can be used to train a recurrent neural network model for predicting the reactor power of the AGN-201. The recurrent neural network was validated using both synthetic and real operational data. We envision this approach will help bridge the gap between the virtual and physical sides of a digital twin, where reactor physics models based on as-built data can be corrected for actual operating parameters to ensure the virtual model mirrors reality.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2024

RandONets: Shallow networks with random projections for learning linear and nonlinear operators

Deep neural networks have been extensively used for the solution of both the forward and the inverse problem for dynamical systems. However, their implementation necessitates optimizing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, pose a barrier to achieving high numerical accuracy, but also interpretability. Here, to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections and tailor-made numerical analysis methods that learn accurately and fast linear and nonlinear operators. Building on previous works, we prove that RandOnets are universal approximators of linear and nonlinear operators. Due to their simplicity, RandONets provide a one-step transformation of the input space, facilitating interpretability. For the evaluation of their performance, we focus on operators of PDEs. We show, that RandONets outperform by several orders of magnitude, both in terms of numerical approximation accuracy and computational cost, the “vanilla” DeepONets. Hence, we believe that our method will trigger further developments in the field of scientific machine learning, for the development of new ‘’light”schemes that will provide high accuracy while reducing dramatically the computational cost. A MATLAB toolbox for RandONets, including demos, is available on GitHub at https://github.com/GianlucaFabiani/RandONets.

Interpretable machine learning↗
NASA NTRSDate not supplied

Neo – Mars Adaptive Training Integrative Knowledge System (MATRIKS) to Improve Operational Performance and its Neural Basis for Spaceflight

With prolonged mission durations, spaceflight crews will become increasingly dependent on onboard technologies for knowledge acquisition and maintenance. It is expected that not all skills and knowledge required for these missions can be retained and retrieved based on pre-mission training alone. Limited and delayed communication will significantly constrain support from Mission Control and crews will increasingly rely on autonomous onboard technologies to successfully perform post-landing operations. With the present project we will target NASA’s particular interest in developing and assessing an adaptive, just-in-time countermeasure that will consolidate and improve skills that are most relevant to space flight operations. To achieve this aim, NASA established a Virtual NASA Specialized Center of Research (VNSCOR)referred to as “Mars Adaptive Training Integrative Knowledge System (MATRIKS)”, comprising the following three projects: (1) “Trinity–Multi-Environment Virtual Training for Long Duration Exploration Missions”, PI: A. Anderson (UC Boulder); (2) “Morpheus–A Haptic Sensory Supplement to Optimize In-Flight Adaptive Training for Human Control of Spacecraft Robotic Arms”, PI: S. Robinson, UC Davis); and the present project “Neo–Adaptive Training integrative knowledge System to Improve Operational Performance and its Neural Basis for Spaceflight” (UPenn, PI: A.C. Stahn). Neo leverages a validated workstation called 6DF that simulates a rendezvous and docking maneuver using real spacecraft flight dynamics. It is designed to (1) train and improve sensorimotor skills relevant for inflight and post-landing operational tasks; (2) feature an autonomous and adaptive training approach that does not rely on feedback from flight operations on the ground; (3) maximize the transfer of mission-relevant motor skills; (4) allow the assessment of the neural circuitry underlying the task; and (5) deliver the training in a motivating and meaningful way to astronauts. Neocomprises two overarching aims: First, we will identify the neural circuitry underlying spaceflight relevant tasks by performing a subset of the 6DFtaskduring functional magnetic resonance imaging (MRI)in a total of up to N=30 subjects with varying levels of 6DF training experience. Second, as part of the above-mentioned VNSCOR MATRIKS the proposed 6DF autonomous intelligent tutor system will be integrated in an additive manner with a haptic feedback intervention (Morpheus), and a multi-environment virtual trainer(Trinity).It is expected that Neo, Morpheus and Trinity mutually complement each other to facilitate an effective countermeasure tool to acquire and retain operational skills that are critical for exploration class missions. To assess the efficacy of this combined effort, the VNSCOR MATRIKS will collect data inN=16 crew members in one HERA campaign of 45 days duration with N=16 crew members(four missions with N=4 crew member seach).The primary goal is to identify changes in operational performance as assessed by NASA’s simulator of Canadarm2 operations, i.e., Robotic On-board Trainer (ROBoT-r) in response to MATRIKS. As part of Neo we will also identify if, and to what extent MATRIKS will promote transfer to general cognitive performance (Cognition battery), distinctive visuo-spatial tasks critical for telerobotic tasks (Spatial Cognition battery), and affect brain structural changes and the neural circuitry of key brain networks expected to be relevant for spaceflight-related performance. At the conclusion of the research, we will have defined and demonstrated the use of a neuroscience-based, adaptive training integrative knowledge system to potentially mitigate visuo-spatial and sensorimotor brain changes associated with prolonged isolation and confinement to reduce the likelihood or impact of potential decrements in human performance capabilities during long-duration space missions. The expected significance of this 4-year project relates to its relevance for facilitating effective countermeasure tools to acquire and retain operational skills that are critical for exploration class missions. This will support the development of necessary countermeasures and technologies in support of human space exploration, focusing on mitigating operational performance risks.

A C Stahn↗
DOE OSTI2025

Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks

Physics-informed deep learning has emerged as a promising alternative for solving partial differential equations. However, for complex problems, training these networks can still be challenging, often resulting in unsatisfactory accuracy and efficiency. In this work, we demonstrate that the failure of plain physics-informed neural networks arises from the significant discrepancy in the convergence rate of residuals at different training points, where the slowest convergence rate dominates the overall solution convergence. Based on these observations, we propose a pointwise adaptive weighting method that balances the residual decay rate across different training points. The performance of our proposed adaptive weighting method is compared with current state-of-the-art adaptive weighting methods on benchmark problems for both physics-informed neural networks and physics-informed deep operator networks. In conclusion, through extensive numerical results we demonstrate that our proposed approach of balanced residual decay rates offers several advantages, including bounded weights, high prediction accuracy, fast convergence rate, low training uncertainty, low computational cost, and ease of hyperparameter tuning.

Balanced convergence rate↗
DOE OSTI2026

Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study

Neural scaling laws play a pivotal role in the performance of deep neural networks and have been observed in a wide range of tasks. However, a complete theoretical framework for understanding these scaling laws remains underdeveloped. In this paper, we explore the neural scaling laws for deep operator networks, which involve learning mappings between function spaces, with a focus on the Chen and Chen style architecture. These approaches, which include the popular Deep Operator Network (DeepONet), approximate the output functions using a linear combination of learnable basis functions and coefficients that depend on the input functions. We establish a theoretical framework to quantify the neural scaling laws by analyzing its approximation and generalization errors. We articulate the relationship between the approximation and generalization errors of deep operator networks and key factors such as network model size and training data size. Moreover, we address cases where input functions exhibit low-dimensional structures, allowing us to derive tighter error bounds. These results also hold for deep ReLU networks and other similar structures. Our results offer a partial explanation of the neural scaling laws in operator learning and provide a theoretical foundation for their applications.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1993

Space time neural networks for tether operations in space

A space shuttle flight scheduled for 1992 will attempt to prove the feasibility of operating tethered payloads in earth orbit. due to the interaction between the Earth's magnetic field and current pulsing through the tether, the tethered system may exhibit a circular transverse oscillation referred to as the 'skiprope' phenomenon. Effective damping of skiprope motion depends on rapid and accurate detection of skiprope magnitude and phase. Because of non-linear dynamic coupling, the satellite attitude behavior has characteristic oscillations during the skiprope motion. Since the satellite attitude motion has many other perturbations, the relationship between the skiprope parameters and attitude time history is very involved and non-linear. We propose a Space-Time Neural Network implementation for filtering satellite rate gyro data to rapidly detect and predict skiprope magnitude and phase. Training and testing of the skiprope detection system will be performed using a validated Orbital Operations Simulator and Space-Time Neural Network software developed in the Software Technology Branch at NASA's Lyndon B. Johnson Space Center.

Lea, Robert N.↗