Search NASA⌕ Search

SEARCH · Search NASA

Results for “Deep Operator Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗
DOE OSTI2022

PRODeepSyn: predicting anticancer synergistic drug combinations by embedding cell lines with protein–protein interaction network

Abstract Although drug combinations in cancer treatment appear to be a promising therapeutic strategy with respect to monotherapy, it is arduous to discover new synergistic drug combinations due to the combinatorial explosion. Deep learning technology holds immense promise for better prediction of in vitro synergistic drug combinations for certain cell lines. In methods applying such technology, omics data are widely adopted to construct cell line features. However, biological network data are rarely considered yet, which is worthy of in-depth study. In this study, we propose a novel deep learning method, termed PRODeepSyn, for predicting anticancer synergistic drug combinations. By leveraging the Graph Convolutional Network, PRODeepSyn integrates the protein–protein interaction (PPI) network with omics data to construct low-dimensional dense embeddings for cell lines. PRODeepSyn then builds a deep neural network with the Batch Normalization mechanism to predict synergy scores using the cell line embeddings and drug features. PRODeepSyn achieves the lowest root mean square error of 15.08 and the highest Pearson correlation coefficient of 0.75, outperforming two deep learning methods and four machine learning methods. On the classification task, PRODeepSyn achieves an area under the receiver operator characteristics curve of 0.90, an area under the precision–recall curve of 0.63 and a Cohen’s Kappa of 0.53. In the ablation study, we find that using the multi-omics data and the integrated PPI network’s information both can improve the prediction results. Additionally, the case study demonstrates the consistency between PRODeepSyn and previous studies.

Wang, Xiaowen↗
DOE OSTI2024

Extraction of Drell-Yan Angular Parameters in $pp$ Collisions with a 120 GeV Beam Energy Using a Deep-Learning Unfolding Algorithm

Dilepton production in pp collisions through the Drell-Yan process provides a crucial tool for studying the internal quark-gluon structure of the nucleon. By precisely measuring the $\cos2\phi$ asymmetry, where $\phi$ represents the azimuthal angle of the $l^{+}l^{-}$ pair in the Collins-Soper frame, we can gain valuable insights into the proton’s structure and the transverse momentum ($q_{T}$) dependence of the $\cos2\phi$ asymmetry. SeaQuest, a fixed-target Drell-Yan experiment at Fermilab, involved an unpolarized proton beam colliding with unpolarized LH$_{2}$ and LD$_{2}$ targets. Measurements obtained from experiments typically require corrections for detector inefficiencies, smearing, and acceptance. Traditionally, these corrections involve “unfolding” the detector-level measurements through matrix operations. However, in higher-dimensional phase space, these conventional methods fail to scale effectively. To overcome these limitations, we employ an unbinned unfolding method that utilizes deep neural networks for unfolding higher-dimensional phase space. In this presentation, we will explain the design of the neural network architecture, our training strategies, and outline our plans to achieve conclusive results.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Permeability Prediction Using Vision Transformers

Accurate permeability predictions remain pivotal for understanding fluid flow in porous media, influencing crucial operations across petroleum engineering, hydrogeology, and related fields. Traditional approaches, while robust, often grapple with the inherent heterogeneity of reservoir rocks. With the advent of deep learning, convolutional neural networks (CNNs) have emerged as potent tools in image-based permeability estimation, capitalizing on micro-CT scans and digital rock imagery. This paper introduces a novel paradigm, employing vision transformers (ViTs)—a recent advancement in computer vision—for this crucial task. ViTs, which segment images into fixed-sized patches and process them through transformer architectures, present a promising alternative to CNNs. We present a methodology for implementing ViTs for permeability prediction, its results on diverse rock samples, and a comparison against conventional CNNs. The prediction results suggest that, with adequate training data, ViTs can match or surpass the predictive accuracy of CNNs, especially in rocks exhibiting significant heterogeneity. This study underscores the potential of ViTs as an innovative tool in permeability prediction, paving the way for further research and integration into mainstream reservoir characterization workflows.

58 GEOSCIENCES↗
DOE OSTI2023

Galaxy Morphology Classification Using Bayesian Neural Networks for LSST

Within the decade, many new ground and space-based observatories will become operational, generating massive amounts of data on short timescales. New surveys like Rubin Observatory's Legacy Survey of Space and Time (LSST) will be capable of observing objects with greater resolution than ever before, but processing and analyzing these datasets optimally will pose a significant challenge. In an effort to prepare for this, we explore how incorporating Deep Neural Networks can better support future data-intensive Astrophysics tasks such as galaxy morphology classification.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

Measurement of Long Baseline Neutrino Oscillations and Improvements from Deep Learning

NOvA is a long-baseline neutrino oscillation experiment which measures the oscillation of muon neutrinos from the NuMI beam at Fermilab after they travel through the Earth for 810 km. In this dissertation I describe the operations and monitoring of the detectors which make it possible to record over 98% of the delivered neutrino beam. I also present reconstruction and identification techniques using deep convolutional neural networks (CNNs), which are applicable to multiple analyses. Lastly, I detail the oscillation analyses in the $\nu_{\mu} \to \nu_{\mu}$ and $\nu_{\mu} \to\nu_e$ channels on a dataset of 8:85 x $10^{20}$ protons on target, with emphasis on the application of deep learning, energy reconstruction, and treatment of systematic uncertainties....

Psihas, Fernanda↗
DOE OSTI2025

Machine Learning for Anomaly Detection in Neural Network Security and SRF Cavities

This dissertation explores the development and deployment of machine learning approaches to address critical challenges in anomaly detection across two distinct domains: neural network security in federated learning settings and cavity behavior analysis in particle accelerator operations at Jefferson Lab in Newport News, Virginia. Anomaly detection identifies deviations from expected patterns, safeguarding systems in cybersecurity, industry, and research against malicious activities and failures. This dissertation demonstrates how our machine learning approaches enhance detection accuracy and efficiency in both neural network security and industrial applications. First, we investigate vulnerabilities in deep neural networks deployed in federated learning. Although federated learning preserves user privacy by training models locally, it remains vulnerable to backdoor attacks, in which malicious participants embed hidden triggers that induce targeted misbehavior. We propose a self-supervised contrastive learning framework to detect and mitigate such backdoor attacks. In our experiments, this method achieves higher detection accuracy and lower false positive rates than existing defenses, while operating without access to local model updates or original training data and thus preserving the privacy guarantees of the federated setting. Second, we address the operational reliability of superconducting radio-frequency (SRF) cavities at the Continuous Electron Beam Accelerator Facility (CEBAF). Our research leverages an unsupervised learning approach, combined with Principal Component Analysis (PCA) and k-means clustering, to identify anomalous behaviors in SRF cavities. Our method detects subtle anomalous behavior by analyzing SRF signal data. This knowledge allows for the early detection and resolution of potential faults, significantly improving the efficiency and reliability of operations. Third, we extend these insights to time-series anomaly detection more broadly. We design a contrastive-learning based model tailored to increasingly dynamic environments and academic research. This model improves detection accuracy in settings that require real-time monitoring and predictive maintenance. Our research underscores the broader applicability and impact of advanced machine learning techniques in anomaly detection. By extracting meaningful patterns from complex data, machine learning can significantly enhance security in distributed neural networks and improve the efficiency of particle accelerator operations. This dissertation serves as a stepping stone for future investigations into the vast possibilities of anomaly detection, inspiring further exploration and development of machine learning techniques in this field.

Ferguson, Hal [Old Dominion University]↗
DOE OSTI2021

Smart Methane Emission Detection System Development (Final Report)

Working with the Department of Energy’s National Energy Technology Laboratory, Southwest Research Institute® (SwRI®) developed a system to identify methane leaks reliably, accurately, and autonomously at critical midstream sections of the natural gas distribution network in real- time for the purpose of mitigating methane emissions using Optical Gas Imaging (OGI) cameras. SwRI’s Smart Leak Detection – Methane (SLED/M) adds a high degree of automation to the process of methane leak detection to minimize sources of human error, minimize response time to a leak event, and maximize midstream visibility. Furthermore, SwRI has been working towards integrating Quantitative OGI (QOGI) capabilities into this existing technology. By leveraging Deep Learning, SwRI now has the capability to estimate fugitive emission leak rates quickly and reliably, which allows operators to detect emissions, quantify leak rate, prioritize repairs, and validate the repairs in a single instrument. The next generation QOGI technology leverages the same cameras used in Leak Detection and Repair (LDAR) programs, with improvements in safety and speed for traditional quantification-based repairs, ultimately leading to less overhead cost for the operators. The goals for this research were to develop two types of models with the following goals: 1. Run in real-time on the edge (≥ 12 Hz) 2. Classification: Achieve less than 5% false positive detection 3. Classification: Achieve ≥ 95% methane plume detection rate 4. Regression: achieve ≤ 10 standard cubic feet per hour (scfh) prediction > 70% of the time In order to achieve these results, multiple infrared (IR) and other sensors were investigated in tandem with the midwave IR (MWIR) OGI to provide additional information to train the underlying models. Information on atmospheric conditions including humidity, temperature, pressure, and solar radiation was provided by a weather station. Several machine learning and deep learning architectures and methods, including looking at quantized classification networks and regressions networks, were explored. As further data was collected, curated, and labeled, it allowed for more refined regressive networks to be adequately trained, leading to better insight into the true flow rates being observed. An important valuable deliverable of this research effort was the development of an advanced network which underwent multiple iterations capable of giving a continuous output. The current network has a predicted mean average percentage error (MAPE) of 12.3% just outside our target goal of 10.00%, but an accuracy of 97.78% at ±50 scfh, well within the overall goal for the Department of Energy (DOE) program. Upon closer inspection, it was observed that more than 10% of datapoints contributing to the MAPE predictions were the result of low flow rate predictions and are beyond the sensitivity of instrument measurement as a result of normal operational variation and noise.

03 NATURAL GAS↗
DOE OSTI2017

Deep Packet/Flow Analysis using GPUs

Deep packet inspection (DPI) faces severe performance challenges in high-speed networks (40/100 GE) as it requires a large amount of raw computing power and high I/O throughputs. Recently, researchers have tentatively used GPUs to address the above issues and boost the performance of DPI. Typically, DPI applications involve highly complex operations in both per-packet and per-flow data level, often in real-time. The parallel architecture of GPUs fits exceptionally well for per-packet network traffic processing. However, for stateful network protocols such as TCP, their data stream need to be reconstructed in a per-flow level to deliver a consistent content analysis. Since the flow-centric operations are naturally antiparallel and often require large memory space for buffering out-of-sequence packets, they can be problematic for GPUs, whose memory is normally limited to several gigabytes. In this work, we present a highly efficient GPU-based deep packet/flow analysis framework. The proposed design includes a purely GPU-implemented flow tracking and TCP stream reassembly. Instead of buffering and waiting for TCP packets to become in sequence, our framework process the packets in batch and uses a deterministic finite automaton (DFA) with prefix-/suffix- tree method to detect patterns across out-of-sequence packets that happen to be located in different batches. In conclusion, evaluation shows that our code can reassemble and forward tens of millions of packets per second and conduct a stateful signature-based deep packet inspection at 55 Gbit/s using an NVIDIA K40 GPU.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Newton-Raphson AC Power Flow Convergence Based on Deep Learning Initialization and Homotopy Continuation

Power flow forms the basis of many power system studies. With the increased penetration of renewable energy, grid planners tend to perform multiple power flow simulations under various operating conditions and not just selected snapshots at peak or light load conditions. Getting a converged AC power flow (ACPF) case remains a significant challenge for grid planners especially in large power grid networks. This paper proposes a two-stage approach to improve Newton-Raphson ACPF convergence and was applied to a 6102 bus Electric Reliability Council of Texas (ERCOT) system. The first stage utilizes a deep learning-based initializer with data re-training. Here a deep neural network (DNN) initializer is developed to provide better initial voltage magnitude and angle guesses to aid in power flow convergence. This is because Newton-Raphson ACPF is quite sensitive to the initial conditions and bad initialization could lead to divergence. The DNN initializer includes a data re-training framework that improves the initializer's performance when faced with limited training data. The DNN initializer successfully solved 3,285 cases out of 3,899 non-converging dispatch and performed better than random forest and DC power flow initialization methods. ACPF cases not solved in this first stage are then passed through a hot-starting algorithm based on homotopy continuation with switched shunt control. The hot-starting algorithm successfully converged 416 cases out of the remaining 614 non-converging ACPF dispatch. In conclusion, the combined two-stage approach achieved a 94.9% success rate, by converging a total of 3,701 cases out of the initial 3,899 unsolved cases.

Deep learning↗
DOE OSTI2020

Fouling modeling and prediction approach for heat exchangers using deep learning

In this article, we develop a generalized and scalable statistical model for accurate prediction of fouling resistance using commonly measured parameters of industrial heat exchangers. This prediction model is based on deep learning where a scalable algorithmic architecture learns non-linear functional relationships between a set of target and predictor variables from large number of training samples. Here, the efficacy of this modeling approach is demonstrated for predicting fouling in an analytically modeled cross-flow heat exchanger, designed for waste heat recovery from flue-gas using room temperature water. The performance results of the trained models demonstrate that the mean absolute prediction errors are under 10 –4 KW –1 for flue-gas side, water side and overall fouling resistances. The coefficients of determination (R 2 ), which characterize the goodness of fit between the predictions and observed data, are over 99%. Even under varying levels of measurement noise in the inputs, we demonstrate that predictions over an ensemble of multiple neural networks achieves better accuracy and robustness to noise. We find that the proposed deep-learning fouling prediction framework learns to follow heat exchanger flow and heat transfer physics, which we confirm using locally interpretable model agnostic explanations around randomly selected operating points. Overall, we provide a robust algorithmic framework for fouling prediction that can be generalized and scaled to various types of industrial heat exchangers.

42 ENGINEERING↗
DOE OSTI2026

Parallel-in-Time Solution of Allen-Cahn Equations by Integrating Operator Learning into the Parareal Method

While recent advances in deep learning have shown promising efficiency gains in solving time-dependent partial differential equations (PDEs), matching the accuracy of conventional numerical solvers still remains a challenge. One strategy to improve the accuracy of deep learning-based solutions for time-dependent PDEs is to use the learned model as the coarse propagator in the Parareal method and a traditional numerical method as the fine solver. However, successful integration of deep learning into the Parareal method requires consistency between the coarse and fine solvers, particularly for PDEs exhibiting rapid changes such as sharp transitions. Here, to ensure this consistency, we propose using convolutional neural networks (CNNs) to learn the fully discrete time-stepping operator defined by the same numerical scheme employed as the fine solver. We demonstrate the effectiveness of the proposed method in solving the classical and mass-conservative Allen–Cahn (AC) equations. Through iterative updates in the Parareal algorithm, our approach achieves a significant computational speedup compared to traditional fine solvers while converging to high-accuracy solutions. Our results highlight that the proposed hybrid Parareal algorithm effectively accelerates simulations, particularly when implemented on multiple GPUs, and converges to the desired accuracy in only a few iterations. Another advantage of our method is that the CNN model is trained on trajectory-based data generated from random initial conditions, such that the trained model can be used to solve the AC equations with various initial conditions without retraining. This work demonstrates the potential of integrating neural network methods into parallel-in-time frameworks for efficient and accurate simulations of time-dependent PDEs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Smart Methane Emission Detection System Development (Final Report)

Working with the Department of Energy's National Energy Technology Laboratory, Southwest Research Institute® (SwRI®) developed a system to identify methane leaks reliably, accurately, and autonomously at critical midstream sections of the natural gas distribution network in real-time for the purpose of mitigating methane emissions using Optical Gas Imaging (OGI) cameras. SwRI's Smart Leak Detection – Methane (SLED/M) adds a high degree of automation to the process of methane leak detection to minimize sources of human error, minimize response time to a leak event, and maximize midstream visibility. Furthermore, SwRI has been working towards integrating Quantitative OGI (QOGI) capabilities into this existing technology. By leveraging Deep Learning, SwRI now has the capability to estimate fugitive emission leak rates quickly and reliably, which allows operators to detect emissions, quantify leak rate, prioritize repairs, and validate the repairs in a single instrument. The next generation QOGI technology leverages the same cameras used in Leak Detection and Repair (LDAR) programs, with improvements in safety and speed for traditional quantification-based repairs, ultimately leading to less overhead cost for the operators. The goals for this research were to develop two types of models with the following goals: Run in real-time on the edge (≥ 12 Hz), Classification: Achieve less than 5% false positive detection, Classification: Achieve ≥ 95% methane plume detection rate, Regression: achieve ≤ 10 standard cubic feet per hour (scfh) prediction > 70% of the time. In order to achieve these results, multiple infrared (IR) and other sensors were investigated in tandem with the midwave IR (MWIR) OGI to provide additional information to train the underlying models. Information on atmospheric conditions including humidity, temperature, pressure, and solar radiation was provided by a weather station. Several machine learning and deep learning architectures and methods, including looking at quantized classification networks and regressions networks, were explored. As further data was collected, curated, and labeled, it allowed for more refined regressive networks to be adequately trained, leading to better insight into the true flow rates being observed. An important valuable deliverable of this research effort was the development of an advanced network which underwent multiple iterations capable of giving a continuous output. The current network has a predicted mean average percentage error (MAPE) of 12.3% just outside our target goal of 10.00%, but an accuracy of 97.78% at ±50 scfh, well within the overall goal for the Department of Energy (DOE) program. Upon closer inspection, it was observed that more than 10% of datapoints contributing to the MAPE predictions were the result of low flow rate predictions and are beyond the sensitivity of instrument measurement as a result of normal operational variation and noise.

03 NATURAL GAS↗
DOE OSTI2023

ATLAS flavour-tagging algorithms for the LHC Run 2 $pp$ collision dataset

The flavour-tagging algorithms developed by the ATLAS Collaboration and used to analyse its dataset of $\sqrt{s}$ = 13 TeV $pp$ collisions from Run 2 of the Large Hadron Collider are presented. These new tagging algorithms are based on recurrent and deep neural networks, and their performance is evaluated in simulated collision events. These developments yield considerable improvements over previous jet-flavour identification strategies. At the 77% $b$-jet identification efficiency operating point, light-jet (charm-jet) rejection factors of 170 (5) are achieved in a sample of simulated Standard Model $t\bar{t}$ events; similarly, at a $c$-jet identification efficiency of 30%, a light-jet ($b$-jet) rejection factor of 70 (9) is obtained.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Adaptive Load Shedding for Grid Emergency Control via Deep Reinforcement Learning

Emergency control, typically such as under-voltage load shedding (UVLS), is broadly used to grapple with low voltage and voltage instability issues in real-world power systems under contingencies. However, existing emergency control schemes are rule-based and cannot be adaptively applied to uncertain and floating operating conditions. Here, we propose an adaptive UVLS algorithm for emergency control via deep reinforcement learning (DRL) and expert systems. We first construct dynamic components for picturing the power system operation as the environment. The transient voltage recovery criteria, which poses time-varying requirements to UVLS, is integrated into the states and reward function to advise the learning of deep neural networks. The proposed method has no tuning issue of coefficients in reward functions, and this issue was regarded as a deficiency in the existing DRL-based algorithms. Case studies illustrate that the proposed method outperforms the traditional UVLS relay in both the timeliness and efficacy for emergency control.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

DNN Approximation of Nonlinear Finite Element Equations

We investigate the potential of applying (D)NN ((deep) neural networks) for approximating nonlinear mappings arising in the finite element discretization of nonlinear PDEs (partial differential equations). As an application, we apply the trained DNN to replace the coarse nonlinear operator thus avoiding the need to visit the fine level discretization in order to evaluate the actions of the true coarse nonlinear operator. The feasibility of the studied approach is demonstrated in a two level FAS (full approximation scheme) used to solve a nonlinear diffusion-reaction PDE.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy

The functionality of ferroelastic domain walls in ferroelectric materials is explored in real-time via the in situ implementation of computer vision algorithms in scanning probe microscopy (SPM) experiment. The robust deep convolutional neural network (DCNN) is implemented based on a deep residual learning framework (Res) and holistically nested edge detection (Hed), and ensembled to minimize the out-of-distribution drift effects. The DCNN is implemented for real-time operations on SPM, converting the data stream into the semantically segmented image of domain walls and the corresponding uncertainty. Further the pre-defined experimental workflows perform piezoresponse spectroscopy measurement on thus discovered domain walls, and alternating high- and low-polarization dynamic (out-of-plane) ferroelastic domain walls in a PbTiO 3 (PTO) thin film and high polarization dynamic (out-of-plane) at short ferroelastic walls (compared with long ferroelastic walls) in a lead zirconate titanate (PZT) thin film is reported. This work establishes the framework for real-time DCNN analysis of data streams in scanning probe and other microscopies and highlights the role of out-of-distribution effects and strategies to ameliorate them in real time analytics.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Accelerating cavity fault prediction using deep learning at Jefferson Laboratory

Abstract Accelerating cavities are an integral part of the continuous electron beam accelerator facility (CEBAF) at Jefferson Laboratory. When any of the over 400 cavities in CEBAF experiences a fault, it disrupts beam delivery to experimental user halls. In this study, we propose the use of a deep learning model to predict slowly developing cavity faults. By utilizing pre-fault signals, we train a long short-term memory-convolutional neural network binary classifier to distinguish between radio-frequency (RF) signals during normal operation and RF signals indicative of impending faults. We optimize the model by adjusting the fault confidence threshold and implementing a multiple consecutive window criterion to identify fault events, ensuring a low false positive rate. Results obtained from analysis of a real dataset collected from the accelerating cavities simulating a deployed scenario demonstrate the model’s ability to identify normal signals with 99.99% accuracy and correctly predict 80% of slowly developing faults. Notably, these achievements were achieved in the context of a highly imbalanced dataset, and fault predictions were made several hundred milliseconds before the onset of the fault. Anticipating faults enables preemptive measures to improve operational efficiency by preventing or mitigating their occurrence.

43 PARTICLE ACCELERATORS↗