Search NASA⌕ Search

SEARCH · Search NASA

Results for “Deep Operator Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Non-Intrusive Appliance Identification with Appliance-Specific Networks

The problem of noninstrusive load monitoring (NILM) is usually formulated as a single-channel blind source separation task, whose successful solution enable fast and convenient load identification and energy disaggregation. When applied at test time, NILM algorithms aim to identify the operating characteristics of individual appliances from an aggregate power measurement of the entire house. Recent advances in deep learning gave rise to many methods that mostly focus on learning a direct mapping from aggregate measurement to individual appliance power. However, these methods are not only computationally expensive, but they often suffer from overfitting and do not generalize very well. In this article, we propose a novel NILM method that leverages advances in statistical learning that have not been properly applied in this domain before. The proposed method consists of three stages: first, a Bayesian nonparametric learning-based approach for appliance state extraction; second, synthetic minority oversampling technique for data augmentation and mitigating the heavy imbalance in switching events; and third, appliance-specific lightweight long short-term memory networks for status classification for each appliance. Here, we adopt a “differential” input (the difference before and after the switching event) to reduce the complexity of network training and make the proposed method robust to multiappliance switching events. Experiments are conducted to demonstrate the effectiveness of the proposed method, achieving superior performance when compared to recent methods. An ablation study is conducted to demonstrate the effectiveness of each module of our method. Finally, we investigate the quality of generated synthetic samples.

42 ENGINEERING↗
DOE OSTI2023

Postdisaster Routing of Movable Energy Resources for Enhanced Distribution System Resilience: A Deep Reinforcement Learning-Based Approach

The deployment of movable energy resources (MERs) can be an effective strategy to restore critical loads to enhance power system resilience when no other energy sources are available after the occurrence of an extreme event. Since the optimal locations of MERs following an extreme event are dependent on system operating states (e.g., the loads at each node, on/off status of system branches, and so on), existing analytical and population-based approaches must repeat the entire analysis and calculation when the system operating states change. On the contrary, if deep reinforcement learning (DRL)-based algorithms are sufficiently trained with a wide range of scenarios, they can quickly find optimal or near-optimal locations irrespective of changes in system states. A deep Q-learning-based approach is proposed for optimal MER deployment to enhance power system resilience. MERs can be also utilized to complement other types of resources, if available. The proposed approach operates in two stages after the occurrence of extreme events. In the first stage, the distribution network is represented as a graph, and the network is then reconfigured using tie switches by using Kruskal’s spanning forest search algorithm (KSFSA). To maximize critical load recovery, the optimal or near-optimal locations of MERs are chosen in the second stage. Further, case studies on a 33-node distribution system and a modified IEEE 123-node system demonstrate the effectiveness of the proposed approach for postdisaster routing of MERs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Multimodal Prediction of Tearing Instabilities in a Tokamak

Tokamak is a torus-shaped nuclear fusion device that uses magnetic fields to confine fusion fuel in the form of plasma. Tearing instability in plasma is a major issue in which the magnetic field breaks and recombines in tokamak. Here, this instability can lead to plasma disruption that terminates the fusion power generation and damages the plasma-facing wall materials. For a successful steady operation of a large-scale tokamak without disruption, it is required to predict and alarm the tearing instabilities well in advance to avoid them. In this work, we develop and validate a deep neural network-based multimodal prediction system that estimates the future tearing instability likelihood from multi-diagnostics signals in the DIII-D tokamak.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Data-Driven Modeling and Correction of Vehicle Dynamics

We develop a data-driven framework for learning and correcting nonautonomous vehicle dynamics. Physics-based vehicle models are often simplified for tractability and therefore exhibit inherent model-form uncertainty, motivating the need for data-driven correction. Moreover, nonautonomous dynamics are governed by time-dependent control inputs, which pose challenges in learning predictive models directly from temporal snapshot data. To address these, we reformulate the vehicle dynamics via a local parameterization of the time-dependent inputs, yielding a modified system composed ofa sequence of local parametric dynamical systems. Here, we approximate these parametric systems using two complementary approaches. First, we employ the dimension reduction and interpolation in parameter space (DRIPS) methodology to construct efficient linear surrogate models, equipped with lifted observable spaces and manifold-based operator interpolation. This enables data-efficient learning of vehicle models whose dynamics admit accurate linear representations in the lifted spaces. Second, for more strongly nonlinear systems, we employ flow map learning (FML), a deep neural network (DNN) approach that approximates the parametric evolution map without requiring special treatment of nonlinearities. We further extend FML with a transfer-learning-based model correction procedure, enabling the correction of misspecified prior models using only a sparse set of high-fidelity or experimental measurements, without assuming a prescribed form for the correction term. Through a suite of numerical experiments on unicycle, simplified bicycle, and slip-based bicycle models, we demonstrate that DRIPS offers robust and highly data-efficient learning of nonautonomous vehicle dynamics, while FML provides expressive nonlinear modeling and effective correction of model-form errors under severe data scarcity.

data-driven modeling↗
DOE OSTI2021

Predictive Data-driven Platform for Subsurface Energy Production

Subsurface energy activities such as unconventional resource recovery, enhanced geothermal energy systems, and geologic carbon storage require fast and reliable methods to account for complex, multiphysical processes in heterogeneous fractured and porous media. Although reservoir simulation is considered the industry standard for simulating these subsurface systems with injection and/or extraction operations, reservoir simulation requires spatio-temporal “Big Data” into the simulation model, which is typically a major challenge during model development and computational phase. In this work, we developed and applied various deep neural network-based approaches to (1) process multiscale image segmentation, (2) generate ensemble members of drainage networks, flow channels, and porous media using deep convolutional generative adversarial network, (3) construct multiple hybrid neural networks such as convolutional LSTM and convolutional neural network-LSTM to develop fast and accurate reduced order models for shale gas extraction, and (4) physics-informed neural network and deep Q-learning for flow and energy production. We hypothesized that physicsbased machine learning/deep learning can overcome the shortcomings of traditional machine learning methods where data-driven models have faltered beyond the data and physical conditions used for training and validation. We improved and developed novel approaches to demonstrate that physics-based ML can allow us to incorporate physical constraints (e.g., scientific domain knowledge) into ML framework. Outcomes of this project will be readily applicable for many energy and national security problems that are particularly defined by multiscale features and network systems.

58 GEOSCIENCES↗
DOE OSTI2022

Deep learning-based spatio-temporal estimate of greenhouse gas emissions using satellite data

Accurate estimation of greenhouse gases (GHGs) emissions is very important for developing mitigation strategies to climate change by controlling and reducing GHG emissions. This project aims to develop multiple deep learning approaches to estimate anthropogenic greenhouse gas emissions using multiple types of satellite data. NO2 concentration is chosen as an example of GHGs to evaluate the proposed approach. Two sentinel satellites (sentinel-2 and sentinel-5P) provide multiscale observations of GHGs from 10-60m resolution (sentinel-2) to ~kilometer scale resolution (sentinel-5P). Among multiple deep learning (DL) architectures evaluated, two best DL models demonstrate that key features of spatio-temporal satellite data and additional information (e.g., observation times and/or coordinates of ground stations) can be extracted using convolutional neural networks and feed forward neural networks, respectively. In particular, irregular time series data from different NO 2 observation stations limit the flexibility of long short-term memory architecture, requiring zero-padding to fill in missing data. However, deep neural operator (DNO) architecture can stack time-series data as input, providing the flexibility of input structure without zero-padding. As a result, the DNO outperformed other deep learning architectures to account for time-varying features. Overall, temporal patterns with smooth seasonal variations were predicted very well, while frequent fluctuation patterns were not predicted well. In addition, uncertainty quantification using conformal inference method is performed to account for prediction ranges. Overall, this research will lead to a new groundwork for estimating greenhouse gas concentrations using multiple satellite data to enhance our capability of tracking the cause of climate change and developing mitigation strategies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

A Novel LDPP-MADDPG Approach for Distributed Power Allocation in mmWave Cellular Networks

This paper considers the problem of distributed beam scheduling and power allocation problem in millimeter- Wave (mmWave) cellular networks, in which multiple Base Stations (BSs) operate as individual operators over a shared spectrum. We propose a novel learning-aided approach that integrates the Lyapunov Drift-Plus-Penalty (LDPP) framework and Multi-agent Deep Deterministic Policy Gradient (MADDPG) reinforcement learning algorithms. This offers a powerful approach to learning stable and constraint-aware policies, reaping the joint benefit of both LDPP and MADDPG, in complex multiagent environments. The major challenge for this approach is to integrate these two approaches in a meaningful and effective manner. The key idea to solve this problem is to introduce a novel feature of local observation that incorporates potential negative value of the reward function due to the stochastic constraints introduced by the LDPP framework. Empirical results demonstrate that our proposed scheme outperforms the baseline methods under various conditions.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2020

Novel Fault Location Method for Power Systems Based on Attention Mechanism and Double Structure GRU Neural Network

Fault location is one of the most essential techniques to maintain the stable operation of power systems. A fast and accurate fault location allows operators to restore power grids faster and avoid economic losses. Conventional methods rely on expert knowledge to extract the necessary features (e.g. DWT, DFT). For large systems, more coupling effects of transmission lines require more complex feature engineering, and incomplete features can easily introduce large errors. To overcome this, a deep learning approach without manual feature extraction is introduced to the fault location model under big data application. Towards this end, in the proposed method, the attention mechanism, the Bi-GRU and a dual structure network are applied to analyze the current data from different perspectives. Complete information for the fault features is extracted for the fault location. Based on a time series model and benefit from the ability to internally acquire the information architecture of faulty line, the established model is adaptive to the power grids with very complex topologies. Simulation results indicate that the proposed double-structure model reduces the maximum error and is less affected by noise. In comparison with different structures and different models, the proposed method shows better performance in IEEE 39-bus system.

42 ENGINEERING↗
DOE OSTI2022

A gradient-based deep neural network model for simulating multiphase flow in porous media

We report simulation of multiphase flow in porous media is crucial for the effective management of subsurface energy and environment-related activities. The numerical simulators used for modeling such processes rely on spatial and temporal discretization of the governing mass and energy balance partial-differential equations (PDEs) into algebraic systems via finite-difference/volume/element methods. These simulators usually require dedicated software development and maintenance, and suffer low efficiency from a runtime and memory standpoint for problems with multi-scale heterogeneity, coupled-physics processes or fluids with complex phase behavior. Therefore, developing cost-effective, data-driven models can become a practical choice, and in this work, we choose deep learning approaches as they can handle high dimensional data and accurately predict state variables with strong nonlinearity. In this paper, we describe a gradient-based deep neural network (GDNN) constrained by the physics related to multiphase flow in porous media. We tackle the nonlinearity of flow in porous media induced by rock heterogeneity, fluid properties, and fluid-rock interactions by decomposing the nonlinear PDEs into a dictionary of elementary differential operators. We use a combination of operators to handle rock spatial heterogeneity and fluid flow by advection. Since the augmented differential operators are inherently related to the physics of fluid flow, we treat them as first principles prior knowledge to regularize the GDNN training. We use the example of pressure management at geologic CO 2 storage sites, where CO 2 is injected in saline aquifers and brine is produced, and apply GDNN to construct a predictive model that is trained with physics-based simulation data and emulates the physics process. We demonstrate that GDNN can effectively predict the nonlinear patterns of subsurface responses, including the temporal and spatial evolution of the pressure and saturation plumes. We also successfully extend the GDNN to convolutional neural network (CNN), namely gradient-based CNN (GCNN), and validate its capability to improve the prediction accuracy. GDNN has great potential to tackle challenging problems that are governed by highly nonlinear physics and enable the development of data-driven models with higher fidelity.

42 ENGINEERING↗
DOE OSTI2024

CO 2 storage site characterization using ensemble-based approaches with deep generative models

Estimating spatially distributed properties such as permeability from available sparse measurements is a great challenge in efficient subsurface CO 2 storage operations. In this paper, a deep generative model that can accurately capture complex subsurface structure is tested with an ensemble-based inversion method for accurate and accelerated characterization of CO 2 storage sites. We chose Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) for its realistic reservoir property representation and Ensemble Smoother with Multiple Data Assimilation (ES-MDA) for its robust data fitting and uncertainty quantification capability. WGAN-GP are trained to generate high-dimensional permeability fields from a low-dimensional latent space and ES-MDA then updates the latent variables by assimilating available measurements. Several subsurface site characterization examples including Gaussian, channelized, and fractured reservoirs are used to evaluate the accuracy and computational efficiency of the proposed method and the main features of the unknown permeability fields are characterized accurately with reliable uncertainty quantification. Furthermore, the estimation performance is compared with a widely-used variational, i.e., optimization-based, inversion approach, and the proposed approach outperforms the variational inversion method in several benchmark cases. We explain such superior performance by visualizing the objective function in the latent space: because of nonlinear and aggressive dimension reduction via generative modeling, the objective function surface becomes extremely complex while the ensemble approximation can smooth out the multi-modal surface during the minimization. This suggests that the ensemble-based approach works well over the variational approach when combined with deep generative models at the cost of forward model runs unless convergence-ensuring modifications are implemented in the variational inversion.

42 ENGINEERING↗
DOE OSTI2026

Modeling Multi-View Impedance-Based Cross-Geometry SOH Estimator for Li-ion Batteries

Abstract: Accurately estimating battery’s State of Health (SOH) remains challenging when models must generalize across cell designs and operating conditions. Most Electrochemical Impedance Spectroscopy (EIS)-based approaches either (i) hand-engineer a few Nyquist-plot features for shallow models—fast but does not generalize across geometries—or (ii) learn directly from Nyquist plots with deep networks, which removes manual feature extraction, yet still limited to a single plot type. As a result, cross-geometry robustness and deployability on constrained Internet of Things (IoT) devices remain open problems. We propose a compact Convolutional Neural Network (CNN) (∼ 10k parameters) that takes multi-representation EIS inputs—Nyquist (real/imaginary) and phase–magnitude (|Z|/ϕ) stacked as four channels, so the model can learn complementary degradation signatures while remaining small enough for fast inference. We build a dataset from cyclic aging of two geometries (LG INR18650MJ1 cylindrical cells and LIR2032 coin cells), acquire EIS every ten cycles from 10 kHz to 10 mHz (10 points/decade), and evaluate with leave-one-cell-out testing strategy. We further study fusion vs. single-representation inputs and assess feasibility for on-device deployment (e.g., NVIDIA Jetson device). The results show that training on multiple EIS representations improves SOH estimation accuracy and cross-geometry generalization compared to single-representation models, which uses only Nyquist or phase–magnitude plots. This design targets accurate, generalizable SOH prediction without manual feature engineering while enabling practical real-time use.

Bakr, Ahmed [The University of Alabama (UA)]↗
DOE OSTI2026

Decentralised Reinforcement Learning for Dynamic Cyberattack Response in Microgrid Networks

Microgrids rely on communication networks for reliable operation, which makes them inherently vulnerable to cyberattacks. Such attacks can destabilise system dynamics and drive states away from their nominal operating trajectories. Although several physics-informed and machine learning-based strategies have been developed to counter these threats, the rapidly evolving cyber landscape enables adversaries to bypass static defences or rules-based mitigation approaches. This paper proposes a dynamic, online-trained and fully decentralised reinforcement learning (RL)-based cyberattack response framework to protect microgrids from evolving cyberattacks. The proposed framework deploys multiple deep Q-networks (DQNs), each associated with a distributed energy resource (DER), to enable localised and adaptive attack mitigation. In this framework, each DQN processes local voltage and frequency measurements—combined with intrusion detection system (IDS) alerts—as observations and rewards to guide decision-making. Extensive simulation studies demonstrate the robustness of the proposed framework under diverse attack scenarios and varying IDS-induced detection delays. Comparative analysis highlights its superiority over existing static or preexisting rules-based mitigation approaches. Finally, we present an analysis that shows the framework's scalability to real-life microgrids with more interacting agents.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2017

Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations

Recent advances in scanning transmission electron and scanning probe microscopies have opened unprecedented opportunities in probing the materials structural parameters and various functional properties in real space with an angstrom-level precision. This progress has been accompanied by exponential increase in the size and quality of datasets produced by microscopic and spectroscopic experimental techniques. These developments necessitate adequate methods for extracting relevant physical and chemical information from the large datasets, for which a priori information on the structures of various atomic configurations and lattice defects is limited or absent. Here we demonstrate an application of deep neural networks to extracting information from atomically resolved images including location of the atomic species and type of defects. We develop a “weakly-supervised” approach that uses information on the coordinates of all atomic species in the image, extracted via a deep neural network, to identify a rich variety of defects that are not part of an initial training set. We further apply our approach to interpret complex atomic and defect transformation, including switching between different coordination of silicon dopants in graphene as a function of time, formation of peculiar silicon dimer with mixed 3-fold and 4-fold coordination, and the motion of molecular “rotor”. In conclusion, this deep learning based approach resembles logic of a human operator, but can be scaled leading to significant shift in the way of extracting and analyzing information from raw experimental data.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Exploring Flexible Communications for Streamlining DNN Ensemble Training Pipelines

Parallel training of a Deep Neural Network (DNN) ensemble on a cluster of nodes is a common practice to train multiple models in order to construct a model with a higher prediction accuracy. Existing ensemble training pipelines can perform a great deal of redundant operations, resulting in unnecessary CPU usage, or even poor pipeline performance. In order to remove these redundancies, we need pipelines with more communication flexibility than existing DNN frameworks can provide. This project investigates a series of designs to improve pipeline flexibility and adaptivity, while also increasing performance. We implement our designs using Tensorflow with Horovod, and test it using several large DNNs in a large scale GPU cluster, Titan supercomputer at Oak Ridge National Lab. Our results show that the CPU time spent during training is reduced by 2-11X. Furthermore, our implementation can achieve up to 10X speedups when CPU core limits are imposed. Our best pipeline also reduces the average power draw of the ensemble training process by 5-16% when compared to the baseline.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Enabling Highly Efficient Capsule Networks Processing Through A PIM-Based Architecture Design

In recent years, the CNNs have achieved great successes in the image processing tasks, e.g., image recognition and object detection. Unfortunately, traditional CNN's classication is found to be easily misled by increasingly complex image features due to the usage of pooling operations, hence unable to preserve accurate position and pose information of the objects. To address this challenge, a novel neural network structure called Capsule Network has been proposed, which introduces equivariance through capsules to signicantly enhance the learning ability for image segmentation and object detection. Due to its requirement of performing a high volume of matrix operations, CapsNets have been generally accelerated on modern GPU platforms that provide highly optimized software library for common deep learning tasks. However, based on our performance characterization on modern GPUs, CapsNets exhibit low effciency due to the special program and execution features of their routing procedure, including massive unshareable intermediate variables and intensive syn- chronizations, which are very dicult to optimize at software level. To address these challenges, we propose a hybrid computing architecture design named PIM-CapsNet. It preserves GPU's on-chip computing capability for accelerating CNN types of layers in CapsNet, while pipelining with an off-chip in-memory acceleration solution that effectively tackles routing procedure's ineffciency by leveraging the processing-in-memory capability of today's 3D stacked memory. Using routing procedure's inherent parallellization feature, our design enables hierarchical improvements on CapsNet inference effciency through minimizing data movement and maximizing parallel processing in memory. Evaluation results demonstrate that our proposed design can achieve substantial improvement on both performance and energy savings for CapsNet inference, with almost zero accuracy loss. The results also suggest good performance scalability in optimizing the routing procedure with increasing network size.

Zhang, Xingyao↗
DOE OSTI2018

Efficiency Improvement in a Busy Radiology Practice: Determination of Musculoskeletal Magnetic Resonance Imaging Protocol Using Deep-Learning Convolutional Neural Networks

The purposes of this study are to evaluate the feasibility of protocol determination with a convolutional neural networks (CNN) classifier based on short-text classification and to evaluate the agreements by comparing protocols determined by CNN with those determined by musculoskeletal radiologists. Following institutional review board approval, the database of a hospital information system (HIS) was queried for lists of MRI examinations, referring department, patient age, and patient gender. These were exported to a local workstation for analyses: 5258 and 1018 consecutive musculoskeletal MRI examinations were used for the training and test datasets, respectively. The subjects for pre-processing were routine or tumor protocols and the contents were word combinations of the referring department, region, contrast media (or not), gender, and age. The CNN Embedded vector classifier was used with Word2Vec Google news vectors. The test set was tested with each classification model and results were output as routine or tumor protocols. The CNN determinations were evaluated using the receiver operating characteristic (ROC) curves. The accuracies were evaluated by a radiologist-confirmed protocol as the reference protocols. The optimal cut-off values for protocol determination between routine protocols and tumor protocols was 0.5067 with a sensitivity of 92.10%, a specificity of 95.76%, and an area under curve (AUC) of 0.977. The overall accuracy was 94.2% for the ConvNet model. All MRI protocols were correct in the pelvic bone, upper arm, wrist, and lower leg MRIs. Deep-learning-based convolutional neural networks were clinically utilized to determine musculoskeletal MRI protocols. CNN-based text learning and applications could be extended to other radiologic tasks besides image interpretations, improving the work performance of the radiologist.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2020

Deep learning model to detect various synchrophasor data anomalies

High-density synchrophasors provide valuable information for power grid situational awareness, operation and control. Unfortunately, due to factors including communication instability and hardware failure, their data quality can be greatly deteriorated by anomalies. Since the anomalies can impact the performance of the synchrophasor applications, it is of paramount significance to propose a model to detect anomalies in synchrophasor. In this study, a convolutional neural network model is established to detect and classify the anomalies in the synchrophasor measurements. Additionally, four types of anomalies observed in actual synchrophasors including erroneous patterns, random spikes, missing points and high-frequency interferences are considered in this study. The proposed model is extensively evaluated via field-collected measurements from the synchrophasor network in Jiangsu grid, China. The superior performance of the proposed model indicates the great potential of using deep learning for the detection of abnormal synchrophasor measurements.

42 ENGINEERING↗
DOE OSTI2023

Compute in‐Memory with Non‐Volatile Elements for Neural Networks: A Review from a Co‐Design Perspective

Abstract Deep learning has become ubiquitous, touching daily lives across the globe. Today, traditional computer architectures are stressed to their limits in efficiently executing the growing complexity of data and models. Compute‐in‐memory (CIM) can potentially play an important role in developing efficient hardware solutions that reduce data movement from compute‐unit to memory, known as the von Neumann bottleneck. At its heart is a cross‐bar architecture with nodal non‐volatile‐memory elements that performs an analog multiply‐and‐accumulate operation, enabling the matrix‐vector‐multiplications repeatedly used in all neural network workloads. The memory materials can significantly influence final system‐level characteristics and chip performance, including speed, power, and classification accuracy. With an over‐arching co‐design viewpoint, this review assesses the use of cross‐bar based CIM for neural networks, connecting the material properties and the associated design constraints and demands to application, architecture, and performance. Both digital and analog memory are considered, assessing the status for training and inference, and providing metrics for the collective set of properties non‐volatile memory materials will need to demonstrate for a successful CIM technology.

36 MATERIALS SCIENCE↗