Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Operator Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

De-noising drift chambers in CLAS12 using convolutional auto encoders

Modern Nuclear Physics experimental setups run experiments with higher beam intensity resulting in increased noise in detector components used for particle track reconstruction. Increased uncorrelated signals (noise) result in decreased particle reconstruction efficiency. In this paper, we investigate the usage of Machine Learning, specifically Convolutional Neural Network Auto-Encoders (CAE), for de-noising raw hits from drift chambers in the CLAS12 detector. To the best of our knowledge, this is the first time CAE is employed to perform such an operation in this field. During the de-noising phase, it is important to remove as much noise as possible while retaining the valid hits to avoid losing crucial information about the experiment. Here, we show that using CAE, it is possible to remove noise hits while retaining up to 94% of valid tracks for a beam current of 110nA while for lower beam currents (45-55nA), we get up to 98% efficiency. Studies on experimental conditions with increasing noise show that CAE performs better than conventional tracking algorithms in isolating hits belonging to tracks. Specifically, the de-noising algorithm results in tracking efficiency improvements greater than 15%, in real data production procedures with nominal conditions, and up to two times better efficiency in synthetically generated data with high luminosity conditions (90-110nA), indicating that machine learning can lead to significantly shorter times for conducting physics experiments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Investigating Anomalies in Compute Clusters: An Unsupervised Learning Approach

As compute clusters continue to grow in scale and complexity, the frequency of detected anomalies in their operation significantly increases. Timely detection of anomalous events is vital to maintain system efficiency and availability. This study presents an attentionbased graph neural network (GNN) for detecting anomalies in clusters at the compute node level and for providing detailed root cause analysis. We show the effectiveness of attention-based GNNs to accurately detect and localize anomalies on real-world datasets.

Lu, Yiyang↗
NASA NTRS1991

Artificial neural networks and approximate reasoning for intelligent control in space

A method is introduced for learning to refine the control rules of approximate reasoning-based controllers. A reinforcement-learning technique is used in conjunction with a multi-layer neural network model of an approximate reasoning-based controller. The model learns by updating its prediction of the physical system's behavior. The model can use the control knowledge of an experienced operator and fine-tune it through the process of learning. Some of the space domains suitable for applications of the model such as rendezvous and docking, camera tracking, and tethered systems control are discussed.

Berenji, Hamid R.↗
DOE OSTI2024

Physics-constrained machine learning for electrodynamics without gauge ambiguity based on Fourier transformed Maxwell’s equations

We utilize a Fourier transformation-based representation of Maxwell’s equations to develop physics-constrained neural networks for electrodynamics without gauge ambiguity, which we label the Fourier–Helmholtz–Maxwell neural operator method. In this approach, both of Gauss’s laws and Faraday’s law are built in as hard constraints, as well as the longitudinal component of Ampère–Maxwell in Fourier space, assuming the continuity equation. An encoder–decoder network acts as a solution operator for the transverse components of the Fourier transformed vector potential, $\hat{A}_⟂(k,t)$, whose two degrees of freedom are used to predict the electromagnetic fields. This method was tested on two electron beam simulations. Among the models investigated, it was found that a U-Net architecture exhibited the best performance as it trained quicker, was more accurate and generalized better than the other architectures examined. We demonstrate that our approach is useful for solving Maxwell’s equations for the electromagnetic fields generated by intense relativistic charged particle beams and that it generalizes well to unseen test data, while being orders of magnitude quicker than conventional simulations. We show that the model can be re-trained to make highly accurate predictions in as few as 20 epochs on a previously unseen data set.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Physics-Informed Neural Networks for Heat Transfer Problems

Abstract Physics-informed neural networks (PINNs) have gained popularity across different engineering fields due to their effectiveness in solving realistic problems with noisy data and often partially missing physics. In PINNs, automatic differentiation is leveraged to evaluate differential operators without discretization errors, and a multitask learning problem is defined in order to simultaneously fit observed data while respecting the underlying governing laws of physics. Here, we present applications of PINNs to various prototype heat transfer problems, targeting in particular realistic conditions not readily tackled with traditional computational methods. To this end, we first consider forced and mixed convection with unknown thermal boundary conditions on the heated surfaces and aim to obtain the temperature and velocity fields everywhere in the domain, including the boundaries, given some sparse temperature measurements. We also consider the prototype Stefan problem for two-phase flow, aiming to infer the moving interface, the velocity and temperature fields everywhere as well as the different conductivities of a solid and a liquid phase, given a few temperature measurements inside the domain. Finally, we present some realistic industrial applications related to power electronics to highlight the practicality of PINNs as well as the effective use of neural networks in solving general heat transfer problems of industrial complexity. Taken together, the results presented herein demonstrate that PINNs not only can solve ill-posed problems, which are beyond the reach of traditional computational methods, but they can also bridge the gap between computational and experimental heat transfer.

Engineering↗
DOE OSTI2021

A data-driven operational model for traffic at the Dallas Fort Worth International Airport

Airports are on the front line of significant innovations, allowing the movement of more people and goods faster, cheaper, and with greater convenience. As air travel continues to grow, airports will face challenges in responding to increasing passenger vehicle traffic, which leads to lower operational efficiency, poor air quality, and security concerns. This paper evaluates methods for traffic demand forecasting combined with traffic microsimulation, which will allow airport operations staff to accurately predict traffic and congestion. Using two years of detailed data describing individual vehicle arrivals and departures, aircraft movements, and weather at Dallas-Fort Worth (DFW) International Airport, we evaluate multiple prediction methods including the Auto Regressive Integrated Moving Average (ARIMA) family of models, traditional machine learning models, and DeepAR, a modern recurrent neural network (RNN). We find that these algorithms are able to capture the diurnal trends in the surface traffic, and all do very well when predicting the next 30 minutes of demand. Longer forecast horizons are moderately effective, demonstrating the challenge of this problem and highlighting promising techniques as well as potential areas for improvement. Traffic demand is not the only factor that contributes to terminal congestion, because temporary changes to the road network, such as a lane closure, can make benign traffic demand highly congested. Combining a demand forecast with a traffic microsimulation framework provides a complete picture of traffic and its consequences. The result is an operational intelligence platform for exploring policy changes, as well as infrastructure expansion and disruption scenarios. To demonstrate the value of this approach, we present results from a case study at DFW Airport assessing the impact of a policy change for vehicle routing in high demand scenarios. This framework can assist airports like DFW as they tackle daily operational challenges, as well as explore the integration of emerging technology and expansion of their services into long term plans.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Data-Driven Modeling and Correction of Vehicle Dynamics

We develop a data-driven framework for learning and correcting nonautonomous vehicle dynamics. Physics-based vehicle models are often simplified for tractability and therefore exhibit inherent model-form uncertainty, motivating the need for data-driven correction. Moreover, nonautonomous dynamics are governed by time-dependent control inputs, which pose challenges in learning predictive models directly from temporal snapshot data. To address these, we reformulate the vehicle dynamics via a local parameterization of the time-dependent inputs, yielding a modified system composed ofa sequence of local parametric dynamical systems. Here, we approximate these parametric systems using two complementary approaches. First, we employ the dimension reduction and interpolation in parameter space (DRIPS) methodology to construct efficient linear surrogate models, equipped with lifted observable spaces and manifold-based operator interpolation. This enables data-efficient learning of vehicle models whose dynamics admit accurate linear representations in the lifted spaces. Second, for more strongly nonlinear systems, we employ flow map learning (FML), a deep neural network (DNN) approach that approximates the parametric evolution map without requiring special treatment of nonlinearities. We further extend FML with a transfer-learning-based model correction procedure, enabling the correction of misspecified prior models using only a sparse set of high-fidelity or experimental measurements, without assuming a prescribed form for the correction term. Through a suite of numerical experiments on unicycle, simplified bicycle, and slip-based bicycle models, we demonstrate that DRIPS offers robust and highly data-efficient learning of nonautonomous vehicle dynamics, while FML provides expressive nonlinear modeling and effective correction of model-form errors under severe data scarcity.

data-driven modeling↗
DOE OSTI2024

Real-time capable modeling of ICRF heating on NSTX and WEST via machine learning approaches

Abstract A real-time capable core Ion Cyclotron Range of Frequencies (ICRF) heating model on NSTX and WEST is developed. The model is based on two nonlinear regression algorithms, the random forest ensemble of decision trees and the multilayer perceptron neural network. The algorithms are trained on TORIC ICRF spectrum solver simulations of the expected flat-top operation scenarios in NSTX and WEST assuming Maxwellian plasmas. The surrogate models are shown to successfully capture the multi-species core ICRF power absorption predicted by the original model for the high harmonic fast wave and the ion cyclotron minority heating schemes while reducing the computational time by six orders of magnitude. Although these models can be expanded, the achieved regression scoring, computational efficiency and increased model robustness suggest these strategies can be implemented into integrated modeling frameworks for real-time control applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

On the feasibility of using physics-informed machine learning for underground reservoir pressure management

In this work, we evaluate the feasibility of using physics-informed machine learning (PIML) for underground energy-related pressure management. To this end, we develop a PIML framework to manage underground reservoir pressures by training neural networks to determine fluid extraction rates for dedicated extraction wells during fluid injection operations given a range of reservoir conditions (e.g., transmissivity and storativity). We implement an automatically-differentiable analytical physics model of fluid flow in porous media within the PIML framework as a proxy for more complicated models. This allows us to execute a sufficient number of training scenarios to fully evaluate the feasibility of using PIML to support pressure management activities. We quantify the number of physics-model parameters required for automatic differentiation to become more efficient than finite-difference gradient calculations. We use a simple scenario with a single injector, extractor, and critical location for our feasibility analysis. We evaluate the effect of the size of the training dataset (i.e., the number of reservoir condition samples) on the accuracy and efficiency of the PIML framework. For an equivalent number of model evaluations, the larger training dataset took less time to train and produced a neural network that was able to more accurately manage reservoir pressures. We also evaluate the effect of the training dataset batch size (i.e., number of reservoir condition samples used to update the neural network coefficients during training; i.e., how the training dataset is partitioned). While training ran faster with larger batch sizes, they produced neural networks that managed pressures less accurately. We demonstrate the approach on a more complex scenario involving 10 injectors, 10 extractors, and 4 critical locations (a relatively high well density of 20 wells/km2). We provide the number of forward and adjoint model evaluations required in each case as an indication of the feasibility of using PIML for pressure management when more complicated physics models with longer execution times are used.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Machine learning models of intermittent operation of RO wellhead water treatment for salinity reduction and nitrate removal

Machine learning models were developed for intermittent multi-mode operation of a wellhead reverse osmosis water purification and desalination system to predict salt passage, nitrate passage, and permeate flux. The models, based on long short-term memory (LSTM) recurrent neural network (RNN) architecture, included an attention mechanism to increase model performance in proximity of the regulatory limit for nitrate. Training and testing of the models for the Startup, Production, Shutdown and Flushing operational modes were based on operational data (consisting of 22 process variables per data sample) acquired every 2–5 s over a six-month period. The significant sets of model input attributes for the different operational modes were assessed via Spearman ranking correlation, Self-Organizing Map (SOM) analysis and feed forward feature selection (FFFS). Although the variability of nitrate passage, salt passage and permeate flux was significant over the four operational modes, prediction performance for the three outcomes were with R2 and Average Absolute Relative Error (AARE) of 0.78–0.95 and 2.96–6.16 %, respectively. Model updates post membrane elements replacement demonstrated similar levels of prediction accuracy. The study results suggest that there is merit in exploring the utility of multi-mode models for sensor fault detection, data imputation, and for potential use in model-predictive control.

Intermittent RO operation↗
NASA NTRS2021

DroughtCast: A Machine Learning Forecast of the United States Drought Monitor

Drought is one of the most ecologically and economically devastating natural phenomena affecting the United States, causing the U.S. economy billions of dollars in damage, and driving widespread degradation of ecosystem health. Many drought indices are implemented to monitor the current extent and status of drought so stakeholders such as farmers and local governments can appropriately respond. Methods toforecast drought conditions weeks to months in advance are less common but would provide a more effective early warning system to enhance drought response, mitigation, and adaptation planning. To resolve this issue, we introduce DroughtCast, a machine learning framework for forecasting the United States Drought Monitor (USDM). DroughtCast operates on the knowledge that recent anomalies in hydrology and meteorology drive future changes in drought conditions. We use simulated meteorology and satellite observed soil moisture as inputs into a recurrent neural network to accurately forecast the USDM between 1 and 12 weeks into the future. Our analysis shows that precipitation, soil moisture, and temperature are the most important input variables when forecasting future drought conditions. Additionally, a case study of the 2017 Northern Plains Flash Drought shows that DroughtCast was able to forecast a very extreme drought event up to 12 weeks before its onset. Given the favorable forecasting skill of the model, DroughtCast may provide a promising tool for land managers and local governments in preparing for and mitigating the effects of drought.

Machine Learning↗
DOE OSTI2022

Exact constraints and appropriate norms in machine-learned exchange-correlation functionals

Machine learning techniques have received growing attention as an alternative strategy for developing general-purpose density functional approximations, augmenting the historically successful approach of human-designed functionals derived to obey mathematical constraints known for the exact exchange-correlation functional. More recently, efforts have been made to reconcile the two techniques, integrating machine learning and exact-constraint satisfaction. We continue this integrated approach, designing a deep neural network that exploits the exact constraint and appropriate norm philosophy to de-orbitalize the strongly constrained and appropriately normed (SCAN) functional. The deep neural network is trained to replicate the SCAN functional from only electron density and local derivative information, avoiding the use of the orbital-dependent kinetic energy density. The performance and transferability of the machine-learned functional are demonstrated for molecular and periodic systems.

Artificial neural networks↗
DOE OSTI2023

Superconducting Hyperdimensional Associative Memory Circuit for Scalable Machine Learning

Here we propose a generalized architecture for the first rapid-single-flux-quantum (RSFQ) associative memory circuit. The circuit employs hyperdimensional computing (HDC), a machine learning (ML) paradigm utilizing vectors with dimensionality in the thousands to represent information. HDC designs have small memory footprints, simple computations, and simple training algorithms compared to superconducting neural network accelerators (SNNAs), making them a better option for scalable SFQ machine learning (ML) solutions. The proposed superconducting HDC (SHDC) circuit uses entirely on-chip RSFQ memory which is tightly integrated with logic, operates at 33.3 GHz, is applicable to general ML tasks, and is manufacturable at practically useful scales given current SFQ fabrication limits. Tailored to a language recognition task, SHDC consists of ~ 2-20 M Josephson junctions (JJs) and consumes up to three times less power than an analogous CMOS HDC circuit while achieving 78-84% higher throughput. SHDC is capable of outperforming the state of the art RSFQ SNNA, SuperNPU, by 48-99% for all benchmark NN architectures tested while occupying up to 90% less area and consuming up to nine times less power. To the best of the authors' knowledge, SHDC is currently the only superconducting ML approach feasible at practically useful scales for real-world ML tasks and capable of online learning.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Extending Parsimonious Bayesian Inference

Parsimonious Bayesian inference is a theoretical framework for efficient data assimilation that seeks to balance increased consistency between predictions and training data against corresponding increases in model complexity. Within this framework, over-training is understood as optimization that encodes excessive information within model parameters while only achieving small improvements between predictions and training data. This project aims to develop practical methods of limiting excess model information during optimization. One key observation is that practical heuristics for parsimonious learning in high-dimensions must balance expressivity, i.e. the ability of the model to capture diverse predictions with only a few non-zero parameters, against discoverability, i.e. the ability to train the model with gradient-based optimization and drive parameters to low information states. As such, we developed logical activation functions that are able to adaptively approximate arbitrary truth tables that define Boolean logic operations within a probabilistic framework. These functions have demonstrated the ability to learn exclusive disjunction (XOR) and conditioned disjunction (if [condition] then [result_if_true] else [result_if_false]) within a single layer of a neural network. To efficiently exploit these activation functions to drive parsimonious learning required several other advances within the domain of variational inference. The most efficient form of complexity suppression is structured sparsification, driving most model parameters to zero while achieving the structural coherence among nonzeros needed for bandwidth reduction. Such models are not only far more efficient at suppressing information-theoretic complexity, they also reduce the other forms of complexity (computations, communication, storage, and the number of dependencies needed to evaluate predictions). Aiming to support enhanced sparsification, this project examined new approaches to high-dimensional variational inference that allow us to calibrate and control parameter uncertainty during optimization. By identifying which parameters can sustain sparsifying perturbations with little impact on prediction quality, we can develop better pruning strategies by framing them as approximate Bayesian inference. These advances also open paths to mitigate concerns with deploying advanced learning methods in resource-constrained environments, such as running models on power-limited or communication-limited devices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Neural networks for modeling and control of particle accelerators

Charged particle accelerators support a wide variety of scientific, industrial, and medical applications. They range in scale and complexity from systems with just a few components for beam acceleration and manipulation, to large scientific user facilities that span many kilometers and have hundreds-to-thousands of individually-controllable components. Specific operational requirements must be met by adjusting the many controllable variables of the accelerator. Meeting these requirements can be challenging, both in terms of the ability to achieve specific beam quality metrics in a reliable fashion and in terms of the time needed to set up and maintain the optimal operating conditions. One avenue toward addressing this challenge is to incorporate techniques from the fields of machine learning (ML) and artificial intelligence (AI) into the way particle accelerators are modeled and controlled. While many promising approaches within AI/ML could be used for particle accelerators, this dissertation focuses on approaches based on neural networks. Neural networks are particularly well-suited to modeling, control, and diagnostic analysis of nonlinear systems, as well as systems with large parameter spaces. They are also very appealing for their ability to process high-dimensional data types, such as images and time series (both of which are ubiquitous in particle accelerators). In this work, key studies that demonstrated the potential utility of modern neural network-based approaches to modeling and control of particle accelerators are presented. The context for this work is important: at the start of this work in 2012, there was little interest in AI/ML in the particle accelerator community, and many of the advances in neural networks and deep learning that enabled its present success had not yet been made at that time. As such, this work was both an exploration of possible application areas and a generator of proof-of-concept demonstrations in these areas.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Learning subgrid-scale models with neural ordinary differential equations

We propose a new approach to learning the subgrid-scale model when simulating partial differential equations (PDEs) solved by the method of lines and their representation in chaotic ordinary differential equations, based on neural ordinary differential equations (NODEs). Solving systems with fine temporal and spatial grid scales is an ongoing computational challenge, and closure models are generally difficult to tune. Machine learning approaches have increased the accuracy and efficiency of computational fluid dynamics solvers. In this approach neural networks are used to learn the coarse- to fine-grid map, which can be viewed as subgrid-scale parameterization. We propose a strategy that uses the NODE and partial knowledge to learn the source dynamics at a continuous level. Our method inherits the advantages of NODEs and can be used to parameterize subgrid scales, approximate coupling operators, and improve the efficiency of low-order solvers. Numerical results with the two-scale Lorenz 96 ODE, the convection-diffusion PDE, and the viscous Burgers' PDE are used to illustrate this approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Deep Koopman learning of nonlinear time-varying systems

Here this paper presents a data-driven approach to approximate the dynamics of a nonlinear time-varying system (NTVS) by a linear time-varying system (LTVS), which results from the Koopman operator and deep neural networks. Analysis of the approximation error between states of the NTVS and the resulting LTVS is presented. Simulations on a representative NTVS show that the proposed method achieves small approximation errors, even when the system changes rapidly. Furthermore, simulations in an example of quadcopters demonstrate the computational efficiency of the proposed approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Convolutional Neural Network-Based Protection-Zone Classification of Faults in Distribution Feeders with Photovoltaics.

Fault detection and isolation is critical for reliable operation of distribution systems. The ride-through requirements for the distributed energy resources (DER), mandated by the IEEE 1547-2018 standard, makes it challenging to use undervoltage (UV) conditions for fault detection. In addition, with low fault current contribution from these inverter-based DERs, the time-overcurrent relays are also less effective. Thus motivated, this paper presents a learning-based approach for fault detection and localization. A convolutional neural network (CNN)-based model is proposed which uses local voltage and current waveforms from DER locations and feeder substations, for training a zonal classifier. The classifier can be adopted into any relay-like device for discriminating between faults originating from different protection zones. The performance of the proposed approach was tested on publicly available test feeders with distributed photovoltaics (PVs).

Fault localization, convolutional neural networks,↗