SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This is the code developed for paper - Reducing Frequency Bias of Fourier Neural Operators in 3D Seismic Wavefield Simulations Through Multi-Stage Training (https://arxiv.org/abs/2503.02023), it contains training the multi-stage FNO model to generate the seismic forward simulations in 3D medium.
STFNO (Sparsified Time-dependent PDEs FNO code) is an extension of the popular Fourier Neural Operator (FNO) architecture to the solution of coupled systems of time-dependent partial differential equations. STFNO leverages the sparsified dependencies on the field quantities based on the semi-discretiezed form of the PDEs, enabling significant reduction in the number of model parameters. STFNO has been extensively tested on two fusion simulation codes, NIMROD and GTC, and can be easily tailored to other systems of PDEs.
Small PDE U-Net Solver (SPUS) is a compact and efficient foundation model (FM) designed as a unified neural operator for solving a wide range of partial differentialequations (PDEs). SPUS leverages a lightweight residual U-Net-based architecture as a foundation model architecture. To enable effective learning in this minimalist framework, SPUS utilizes a simple yet powerful auto-regressive pretraining strategy which closely replicates the behavior of numerical solvers to learn the underlying physics. SPUS is designed to be pretrained on a diverse set of fluid dynamics PDEs from public benchmark datasets.
Presentation at the 17th International Conference on Greenhouse Gas Control Technologies GHGT-17 held in Calgary, Canada, October 20-24, 2024. Carbon capture and storage (CCS) is a way to play a critical role in the global transition to a low-emission economy. Current progress is hampered by a number of factors, among which the lack of risk-informed design tools and decision support frameworks is seen as a major roadblock. Significant interest exists in using artificial intelligence to accelerate CCS site feasibility studies, as well as to facilitate the permit application process. Existing works commonly train a single deep learning model. This work investigates the feasibility of using a conventional ensemble learning (committee machine) technique to further improve prediction accuracy. Ensemble-based algorithms generally improve over individual base learners in terms of robustness and accuracy. Deep ensembles, however, are time-consuming to create and train. A pragmatic question is whether small-sized ensembles may lead to prediction improvement. Here we evaluated the efficacy of an ensemble learning technique using the latent spectral model (LSM), an efficient deep neural operator algorithm, as base learners. Preliminary results, obtained using the Illinois Basin-Decatur Project (IBDP) carbon sequestration data/model, show that small-sized ensembles can improve prediction over the base learners, achieving prediction accuracy of ~1.6 psi root mean square error (RMSE) on pressure (relative the average reservoir pressure of 3150 psi), and less than 1.3% for saturation.
This is the conference paper accompanying an oral presentation at the 17th International Conference on Greenhouse Gas Control Technologies GHGT-17 held in Calgary, Canada, October 20-24, 2024. Carbon capture and storage (CCS) is a way to play a critical role in the global transition to a low-emission economy. Current progress is hampered by a number of factors, among which the lack of risk-informed design tools and decision support frameworks is seen as a major roadblock. Significant interest exists in using artificial intelligence to accelerate CCS site feasibility studies, as well as to facilitate the permit application process. Existing works commonly train a single deep learning model. This work investigates the feasibility of using a conventional ensemble learning (committee machine) technique to further improve prediction accuracy. Ensemble-based algorithms generally improve over individual base learners in terms of robustness and accuracy. Deep ensembles, however, are time-consuming to create and train. A pragmatic question is whether small-sized ensembles may lead to prediction improvement. Here we evaluated the efficacy of an ensemble learning technique using the latent spectral model (LSM), an efficient deep neural operator algorithm, as base learners. Preliminary results, obtained using the Illinois Basin-Decatur Project (IBDP) carbon sequestration data/model, show that small-sized ensembles can improve prediction over the base learners, achieving prediction accuracy of ~1.6 psi root mean square error (RMSE) on pressure (relative the average reservoir pressure of 3150 psi), and less than 1.3% for saturation.
An evolvable synthetic neural system includes an evolvable neural interface operably coupled to at least one neural basis function. Each neural basis function includes an evolvable neural interface operably coupled to a heuristic neural system to perform high-level functions and an autonomic neural system to perform low-level functions. In some embodiments, the evolvable synthetic neural system is operably coupled to one or more evolvable synthetic neural systems in a hierarchy.
Improved method of adjoint-operator learning reduces amount of computation and associated computational memory needed to make electronic neural network learn temporally varying pattern (e.g., to recognize moving object in image) in real time. Method extension of method described in "Adjoint-Operator Learning for a Neural Network" (NPO-18352).
4 page paper submitted to ANS Student conference. Summary of paper similar to the following abstract: The ability to predict how a reactor will operate, understand when anomalous conditions arise, and ensure a reactor is being operated as expected is crucial for deploying new nuclear facilities. Digital twins serve as a unique solution to recognizing reactor behavior; however, they require data to be useful. For next-generation reactors, this data may not currently be available. To explore how synthetic physics-informed reactor data can be used to predict reactor operations, a recurrent neural network was implemented for the Idaho State University AGN-201 digital twin. The goal of this work is to determine how synthetic data can be used to train a recurrent neural network model for predicting the reactor power of the AGN-201. The recurrent neural network was validated using both synthetic and real operational data. We envision this approach will help bridge the gap between the virtual and physical sides of a digital twin, where reactor physics models based on as-built data can be corrected for actual operating parameters to ensure the virtual model mirrors reality.
Deep neural networks have been extensively used for the solution of both the forward and the inverse problem for dynamical systems. However, their implementation necessitates optimizing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, pose a barrier to achieving high numerical accuracy, but also interpretability. Here, to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections and tailor-made numerical analysis methods that learn accurately and fast linear and nonlinear operators. Building on previous works, we prove that RandOnets are universal approximators of linear and nonlinear operators. Due to their simplicity, RandONets provide a one-step transformation of the input space, facilitating interpretability. For the evaluation of their performance, we focus on operators of PDEs. We show, that RandONets outperform by several orders of magnitude, both in terms of numerical approximation accuracy and computational cost, the “vanilla” DeepONets. Hence, we believe that our method will trigger further developments in the field of scientific machine learning, for the development of new ‘’light”schemes that will provide high accuracy while reducing dramatically the computational cost. A MATLAB toolbox for RandONets, including demos, is available on GitHub at https://github.com/GianlucaFabiani/RandONets.
With prolonged mission durations, spaceflight crews will become increasingly dependent on onboard technologies for knowledge acquisition and maintenance. It is expected that not all skills and knowledge required for these missions can be retained and retrieved based on pre-mission training alone. Limited and delayed communication will significantly constrain support from Mission Control and crews will increasingly rely on autonomous onboard technologies to successfully perform post-landing operations. With the present project we will target NASA’s particular interest in developing and assessing an adaptive, just-in-time countermeasure that will consolidate and improve skills that are most relevant to space flight operations. To achieve this aim, NASA established a Virtual NASA Specialized Center of Research (VNSCOR)referred to as “Mars Adaptive Training Integrative Knowledge System (MATRIKS)”, comprising the following three projects: (1) “Trinity–Multi-Environment Virtual Training for Long Duration Exploration Missions”, PI: A. Anderson (UC Boulder); (2) “Morpheus–A Haptic Sensory Supplement to Optimize In-Flight Adaptive Training for Human Control of Spacecraft Robotic Arms”, PI: S. Robinson, UC Davis); and the present project “Neo–Adaptive Training integrative knowledge System to Improve Operational Performance and its Neural Basis for Spaceflight” (UPenn, PI: A.C. Stahn). Neo leverages a validated workstation called 6DF that simulates a rendezvous and docking maneuver using real spacecraft flight dynamics. It is designed to (1) train and improve sensorimotor skills relevant for inflight and post-landing operational tasks; (2) feature an autonomous and adaptive training approach that does not rely on feedback from flight operations on the ground; (3) maximize the transfer of mission-relevant motor skills; (4) allow the assessment of the neural circuitry underlying the task; and (5) deliver the training in a motivating and meaningful way to astronauts. Neocomprises two overarching aims: First, we will identify the neural circuitry underlying spaceflight relevant tasks by performing a subset of the 6DFtaskduring functional magnetic resonance imaging (MRI)in a total of up to N=30 subjects with varying levels of 6DF training experience. Second, as part of the above-mentioned VNSCOR MATRIKS the proposed 6DF autonomous intelligent tutor system will be integrated in an additive manner with a haptic feedback intervention (Morpheus), and a multi-environment virtual trainer(Trinity).It is expected that Neo, Morpheus and Trinity mutually complement each other to facilitate an effective countermeasure tool to acquire and retain operational skills that are critical for exploration class missions. To assess the efficacy of this combined effort, the VNSCOR MATRIKS will collect data inN=16 crew members in one HERA campaign of 45 days duration with N=16 crew members(four missions with N=4 crew member seach).The primary goal is to identify changes in operational performance as assessed by NASA’s simulator of Canadarm2 operations, i.e., Robotic On-board Trainer (ROBoT-r) in response to MATRIKS. As part of Neo we will also identify if, and to what extent MATRIKS will promote transfer to general cognitive performance (Cognition battery), distinctive visuo-spatial tasks critical for telerobotic tasks (Spatial Cognition battery), and affect brain structural changes and the neural circuitry of key brain networks expected to be relevant for spaceflight-related performance. At the conclusion of the research, we will have defined and demonstrated the use of a neuroscience-based, adaptive training integrative knowledge system to potentially mitigate visuo-spatial and sensorimotor brain changes associated with prolonged isolation and confinement to reduce the likelihood or impact of potential decrements in human performance capabilities during long-duration space missions. The expected significance of this 4-year project relates to its relevance for facilitating effective countermeasure tools to acquire and retain operational skills that are critical for exploration class missions. This will support the development of necessary countermeasures and technologies in support of human space exploration, focusing on mitigating operational performance risks.
Physics-informed deep learning has emerged as a promising alternative for solving partial differential equations. However, for complex problems, training these networks can still be challenging, often resulting in unsatisfactory accuracy and efficiency. In this work, we demonstrate that the failure of plain physics-informed neural networks arises from the significant discrepancy in the convergence rate of residuals at different training points, where the slowest convergence rate dominates the overall solution convergence. Based on these observations, we propose a pointwise adaptive weighting method that balances the residual decay rate across different training points. The performance of our proposed adaptive weighting method is compared with current state-of-the-art adaptive weighting methods on benchmark problems for both physics-informed neural networks and physics-informed deep operator networks. In conclusion, through extensive numerical results we demonstrate that our proposed approach of balanced residual decay rates offers several advantages, including bounded weights, high prediction accuracy, fast convergence rate, low training uncertainty, low computational cost, and ease of hyperparameter tuning.
Neural scaling laws play a pivotal role in the performance of deep neural networks and have been observed in a wide range of tasks. However, a complete theoretical framework for understanding these scaling laws remains underdeveloped. In this paper, we explore the neural scaling laws for deep operator networks, which involve learning mappings between function spaces, with a focus on the Chen and Chen style architecture. These approaches, which include the popular Deep Operator Network (DeepONet), approximate the output functions using a linear combination of learnable basis functions and coefficients that depend on the input functions. We establish a theoretical framework to quantify the neural scaling laws by analyzing its approximation and generalization errors. We articulate the relationship between the approximation and generalization errors of deep operator networks and key factors such as network model size and training data size. Moreover, we address cases where input functions exhibit low-dimensional structures, allowing us to derive tighter error bounds. These results also hold for deep ReLU networks and other similar structures. Our results offer a partial explanation of the neural scaling laws in operator learning and provide a theoretical foundation for their applications.
A space shuttle flight scheduled for 1992 will attempt to prove the feasibility of operating tethered payloads in earth orbit. due to the interaction between the Earth's magnetic field and current pulsing through the tether, the tethered system may exhibit a circular transverse oscillation referred to as the 'skiprope' phenomenon. Effective damping of skiprope motion depends on rapid and accurate detection of skiprope magnitude and phase. Because of non-linear dynamic coupling, the satellite attitude behavior has characteristic oscillations during the skiprope motion. Since the satellite attitude motion has many other perturbations, the relationship between the skiprope parameters and attitude time history is very involved and non-linear. We propose a Space-Time Neural Network implementation for filtering satellite rate gyro data to rapidly detect and predict skiprope magnitude and phase. Training and testing of the skiprope detection system will be performed using a validated Orbital Operations Simulator and Space-Time Neural Network software developed in the Software Technology Branch at NASA's Lyndon B. Johnson Space Center.
Electronic neural networks made to synthesize initially unknown mathematical models of time-dependent phenomena or to learn temporally evolving patterns by use of algorithms based on adjoint operators. Algorithms less complicated, involve less computation and solve learning equations forward in time possibly simultaneously with equations of evolution of neural network, thereby both increasing computational efficiency and making real-time applications possible.
A developmental approach to the study of the emergence of mental operational structures in neural networks is presented. Neural architectures proposed to underlie the six stages of the sensory-motor period are discussed.
Residual-based adaptive strategies are widely used in scientific machine learning yet remain largely heuristic. We introduce a variational framework that formalizes these methods through convex transformations of the residual, where different transformations correspond to distinct objective functionals. For instance, exponential weights target uniform error minimization, while linear weights recover quadratic error minimization. This perspective reveals adaptive weighting as a means of selecting sampling distributions that optimize a primal objective, directly linking discretization choices to error metrics. This principled approach yields three key benefits: it enables systematic design of adaptive schemes, reduces discretization error by lowering estimator variance, and enhances learning dynamics by improving gradient signal-to-noise ratio. Extending the framework to operator learning, we demonstrate substantial performance gains across diverse optimizers and architectures. Our results provide a theoretical perspective for residual-based adaptivity and establish a foundation for principled discretization and training.
Explore the source record for details and available documents.
A new class of equivariant neural networks is presented, hereby dubbed lattice-equivariant neural networks (LENNs), designed to satisfy local symmetries of a lattice structure. The approach develops within a recently introduced framework aimed at learning neural network-based surrogate models’ lattice Boltzmann collision operators. Whenever neural networks are employed to model physical systems, respecting symmetries and equivariance properties has been shown to be key for accuracy, numerical stability, and performance. Here, hinging on ideas from group representation theory, trainable layers are defined whose algebraic structure is equivariant with respect to the symmetries of the lattice cell. In this work, the presented method naturally allows for efficient implementations, in terms of both memory usage and computational costs, supporting scalable training/testing for lattices in two spatial dimensions and higher (in which the size of symmetry group grows). The approach is validated and tested considering 2D and 3D flowing dynamics, both in laminar and turbulent regimes. It is compared with group-averaged-based symmetric networks and with plain, nonsymmetric, networks, showing how the presented approach unlocks the (a posteriori) accuracy and training stability of the former models and the train/inference speed of the latter networks. (LENNs are about one order of magnitude faster than group-averaged networks in 3D.) The work in this paper opens toward practical use of machine learning-augmented lattice Boltzmann CFD in real-world simulations.