Search NASA⌕ Search

SEARCH · Search NASA

Results for “Deep Operator Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Inter-Domain Fusion for Enhanced Intrusion Detection in Power Systems: An Evidence Theoretic and Meta-Heuristic Approach

False alerts due to misconfigured or compromised intrusion detection systems (IDS) in industrial control system (ICS) networks can lead to severe economic and operational damage. However, research using deep learning to reduce false alerts often requires the physical and cyber sensor data to be trustworthy. Implicit trust is a major problem for artificial intelligence or machine learning (AI/ML) in cyber-physical system (CPS) security, because when these solutions are most urgently needed is also when they are most at risk (e.g., during an attack). To address this, the Inter-Domain Evidence theoretic Approach for Inference (IDEA-I) is proposed that reframes the detection problem as how to make good decisions given uncertainty. Specifically, an evidence theoretic approach leveraging Dempster–Shafer (DS) combination rules and their variants is proposed for reducing false alerts. A multi-hypothesis mass function model is designed that leverages probability scores obtained from supervised-learning classifiers. Using this model, a location-cum-domain-based fusion framework is proposed to evaluate the detector’s performance using disjunctive, conjunctive, and cautious conjunctive rules. The approach is demonstrated in a cyber-physical power system testbed, and the classifiers are trained with datasets from Man-In-The-Middle attack emulation in a large-scale synthetic electric grid. For evaluating the performance, we consider plausibility, belief, pignistic, and general Bayesian theorem-based metrics as decision functions. To improve the performance, a multi-objective-based genetic algorithm is proposed for feature selection considering the decision metrics as the fitness function. Finally, we present a software application to evaluate the DS fusion approaches with different parameters and architectures.

42 ENGINEERING↗
DOE OSTI2023

Uncertainty Aware Deep Learning for Fault Prediction Using Multivariate Time Series Signals

The superconducting radio-frequency cavities are a crucial component of the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. When a cavity faults, beam delivery to experimental end users is disrupted. Prediction of cavity faults prior to onset is essential to reduce operation and maintenance costs. In this work, a parallel long short-term memory (LSTM)-convolution neural network (CNN)-based deep learning (DL) model is proposed to predict impending faults using pre-fault signals. Further, we introduce an uncertainty quantification approach using Monte Carlo dropout with the LSTM-CNN model to ascertain confidence in the prediction. The model was tested using multivariate time series signals from stable cavity operations and before faults. Initial results show that on the test dataset, the model can identify impending faults before their onset with an average 10-fold cross validation accuracy of 97.39% and a standard deviation of 0.12% using a 100-ms time window. It is also observed that the model performs better as the prediction time moves closer to the fault onset. For additional context, we compare the performance of the model with three machine-learning-based (ML) fault prediction models. Our proposed parallel LSTM-CNN-based DL method shows better performance than the ML-based methods.

Rahman, Md Monibor↗
DOE OSTI2024

Visibility-enhanced model-free deep reinforcement learning algorithm for voltage control in realistic distribution systems using smart inverters

Increasing integration of distributed solar photovoltaic (PV) into distribution networks could result in adverse effects on grid operation. Traditional model-based control algorithms require accurate model information that is difficult to acquire and thus are challenging to implement in practice. Here, this paper proposes a surrogate model-enabled grid visibility scheme to empower deep reinforcement learning (DRL) approach for distribution network voltage regulation using PV inverters with minimal system knowledge. In contrast to existing DRL methods, this paper presents and corroborates the adverse impact of missing load information on DRL performance and, based on this finding, proposes a surrogate model methodology to impute load information utilizing observable data. Additionally, a multi-fidelity neural network is utilized to construct the DRL training environment, chosen for its efficient data utilization and enhanced robustness to data uncertainty. The feasibility and effectiveness of the proposed algorithm are assessed by considering DRL testing across varying degrees of observable load information and diverse training environments on a realistic power system.

14 SOLAR ENERGY↗
DOE OSTI2022

Optimizing radar scan strategies for tracking isolated deep convection using observing system simulation experiments

Abstract. Optimizing radar observation strategies is one of the most important considerations in pre-field campaign periods. This is especially true for isolated convective clouds that typically evolve faster than the observations captured by operational radar networks. This study investigates uncertainties in radar observations of the evolution of the microphysical and dynamical properties of isolated deep convective clouds developing in clean and polluted environments. It aims to optimize the radar observation strategy for deep convection through the use of high-spatiotemporal cloud-resolving model simulations, which resolve the evolution of individual convective cells every 1 min, coupled with a radar simulator and a cell tracking algorithm. The radar simulation settings are based on the Tracking Aerosol Convection Interactions ExpeRiment (TRACER) and Experiment of Sea Breeze Convection, Aerosols, Precipitation and Environment (ESCAPE) field campaigns held in the Houston, TX, area but are generalizable to other field campaigns focusing on isolated deep convection. Our analysis produces the following four outcomes. First, a 5–7 m s−1 median difference in maximum updrafts of tracked cells is shown between the clean and polluted simulations in the early stages of the cloud lifetimes. This demonstrates the importance of obtaining accurate estimates of vertical velocity from observations if aerosol impacts are to be properly resolved. Second, tracking of individual cells and using vertical cross section scanning every minute capture the evolution of precipitation particle number concentration and size represented by polarimetric observables better than the operational radar observations that update the volume scan every 5 min. This approach also improves multi-Doppler radar updraft retrievals above 5 km above ground level for regions with updraft velocities greater than 10 m s−1. Third, we propose an optimized strategy composed of cell tracking by quick (1–2 min) vertical cross section scans from more than one radar in addition to the operational volume scans. We also propose the use of a single-RHI (range height indicator) updraft retrieval technique for cells close to the radars, for which multi-Doppler radar retrievals are still challenging. Finally, increasing the number of deep convective cells sampled by such observations better represents the median maximum updraft evolution with sample sizes of more than 10 deep cells, which decreases the error associated with sampling the true population to less than 3 m s−1.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Deep spectral CNN for laser induced breakdown spectroscopy

This work proposes a spectral convolutional neural network (CNN) operating on laser induced breakdown spectroscopy (LIBS) signals to learn to (1) disentangle spectral signals from the sources of sensor uncertainty (i.e., pre-process) and (2) get qualitative and quantitative measures of chemical content of a sample given a spectral signal (i.e., calibrate). Once the spectral CNN is trained, it can accomplish either task through a single feed-forward pass, with real-time benefits and without any additional side information requirements including dark current, system response, temperature and detector-to-target range. Our experiments demonstrate that the proposed method outperforms the existing approaches used by the Mars Science Lab for pre-processing and calibration for remote sensing observations from the Mars rover, ‘Curiosity’.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2019

Evolving Energy Efficient Convolutional Neural Networks

As deep neural networks have been deployed in more and more applications over the past half decade and are finding their way into an ever increasing number of operational systems, their energy consumption becomes a concern whether running in the datacenter or on edge devices. Hyperparameter optimization and automated network design for deep learning is a quickly growing field, but much of the focus has remained only on optimizing for the performance of the machine learning task. In this work, we demonstrate that the best performing networks created through this automated network design process have radically different computational characteristics (e.g. energy usage, model size, inference time), presenting the opportunity to utilize this optimization process to make deep learning networks more energy efficient and deployable to smaller devices. Optimizing for these computational characteristics is critical as the number of applications of deep learning continues to expand.

Young, Steven↗
DOE OSTI2020

Physics-Informed Deep Neural Network Method for Limited Observability State Estimation

The precise knowledge regarding the state of the power grid is important in order to ensure optimal and reliable grid operation. Specifically, knowing the state of the distribution grid becomes increasingly important as more renewable energy sources are connected directly into the distribution network, increasing the fluctuations of the injected power. In this paper, we consider the case when the distribution grid becomes partially observable, due to for example cyber attacks, and the state estimation problem is under-determined. We present a new methodology that leverages a deep neural network (DNN) to estimate the grid state. The standard DNN training method is modified to explicitly incorporate the physical information of the grid topology and line/shunt admittance. We show that our method leads to a superior accuracy of the estimation when compared to the case when no physical information is provided. Finally, we compare the performance of our method to the standard state estimation approach, which is based on the weighted least squares with pseudo-measurements, and show that our method performs significantly better with respect to the estimation accuracy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Femtojoule optical nonlinearity for deep learning with incoherent illumination

Optical neural networks (ONNs) are a promising computational alternative for deep learning due to their inherent massive parallelism for linear operations. However, the development of energy-efficient and highly parallel optical nonlinearities, a critical component in ONNs, remains an outstanding challenge. Here, we introduce a nonlinear optical microdevice array (NOMA) compatible with incoherent illumination by integrating the liquid crystal cell with silicon photodiodes at the single-pixel level. We fabricate NOMA with more than half a million pixels, each functioning as an optical analog of the rectified linear unit at ultralow switching energy down to 100 femtojoules per pixel. With NOMA, we demonstrate an optical multilayer neural network. Our work holds promise for large-scale and low-power deep ONNs, computer vision, and real-time optical image processing.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Randomized probe imaging through deep k-learning

Randomized probe imaging (RPI) is a single-frame diffractive imaging method that uses highly randomized light to reconstruct the spatial features of a scattering object. The reconstruction process, known as phase retrieval, aims to recover a unique solution for the object without measuring the far-field phase information. Typically, reconstruction is done via time-consuming iterative algorithms. In this work, we propose a fast and efficient deep learning based method to reconstruct phase objects from RPI data. The method, which we call deep k-learning, applies the physical propagation operator to generate an approximation of the object as an input to the neural network. This way, the network no longer needs to parametrize the far-field diffraction physics, dramatically improving the results. Deep k-learning is shown to be computationally efficient and robust to Poisson noise. The advantages provided by our method may enable the analysis of far larger datasets in photon starved conditions, with important applications to the study of dynamic phenomena in physical science and biological engineering.

Guo, Zhen (ORCID:0000000213473451)↗
DOE OSTI2023

Model-parallel Fourier neural operators as learned surrogates for large-scale parametric PDEs

Fourier neural operators (FNOs) are a recently introduced neural network architecture for learning solution operators of partial differential equations (PDEs), which have been shown to perform significantly better than comparable deep learning approaches. Once trained, FNOs can achieve speed-ups of multiple orders of magnitude over conventional numerical PDE solvers. However, due to the high dimensionality of their input data and network weights, FNOs have so far only been applied to two-dimensional or small three-dimensional problems. To remove this limited problem-size barrier, we propose a model-parallel version of FNOs based on domain-decomposition of both the input data and network weights. Here, we demonstrate that our model-parallel FNO is able to predict time-varying PDE solutions of over 2.6 billion variables on Perlmutter using up to 512 A100 GPUs and show an example of training a distributed FNO on the Azure cloud for simulating multiphase CO 2 dynamics in the Earth’s subsurface.

58 GEOSCIENCES↗
DOE OSTI2019

DynamiteNSM

DynamiteNSM is a free Network Security Monitor (NSM), built on top of several leading, enterprise-grade technologies. The tool provides network and cybersecurity operators with holistic insights into their networks while giving them the ability to deep-dive into lower-level activities. The solution presents powerful dashboards, giving comprehensive view into performance and threat-based metrics. Dynamite-NSM can be easily deployed in different environments including high-speed data centers, small-to-large enterprises, IoT & industrial networks, and even at home. Dynamite-NSM handles massive volumes of network traffic through scalable ingestion and optimized network sensors. The solution includes two key components: the agent and the monitor. The agent analyzes and forwards network events, while the monitor processes incoming events and displays analytic information. The monitor component builds upon the OpenSearch stack and is coupled with network packet inspection by Zeek and Suricata. Dynamite-NSM is designed to be deployed very quickly with minimal configuration. Unlike many other tools, it can be installed and managed with a standalone command-line utility. The system is inherently passive without disruption to the network. There is no need to install agents on every computer, perform network scans, or directly interact with network assets.

Becker, Jamin↗
DOE OSTI2024

Learning model combining convolutional deep neural network with a self-attention mechanism for AC optimal power flow

Alternating current optimal power flow (OPF) analysis is critical for efficient and reliable operation of power systems. For large systems or repetitive computations, the traditional methods such as the direct and gradient methods, or non-traditional methods, such as the genetic algorithm and simulating annealing, are time-consuming and unsuitable for real-time computing. The work in this paper proposes a novel framework to obtain the optimal solution of power flow in real-time using a combination of convolutional neural networks and a self-attention mechanism. All parameters of the power networks are rearranged in an image-like shape of a multi-channel image where each channel is a two-dimensional matrix. The proposed approach is adaptive with every input size of power systems as well as frequent variations of network topologies without intervention to the framework core. The encompassment of all power system contexts in which all parameters of internal elements, generation costs, and topology information are included, contributes to the higher accuracy of inference compared to other current machine-learning-based OPF-solving methods. Besides, the proposed framework established on ubiquitous platforms is effortlessly integrated into current infrastructures of power systems, and the great efficiency along with the computation speed may serve as a critical point for practical implications, such as enabling faster decision-making during real-time operations, predicting system contingencies, and remedial actions based on an offline pre-trained model. Furthermore, this supervised learning process is applied to the dataset of four case studies of meshed power systems: the IEEE 5-bus system (IEEE-5), the IEEE 30-bus system (IEEE-30), the IEEE 39-bus system (IEEE-39), and the IEEE 57-bus system (IEEE-57) to prove the efficacy of the proposed method.

42 ENGINEERING↗
DOE OSTI2022

Real-time deep-learning inversion of seismic full waveform data for CO 2 saturation and uncertainty in geological carbon storage monitoring

Deep-learning inversion has recently drawn attention in geological carbon storage research due to its potential of imaging and monitoring carbon storage in real time, significantly improving efficiency and safety of carbon storage operations. We present a deep-learning full waveform inversion method that after the neural network has been trained can image CO 2 saturation and its uncertainty in real time. Our deep-learning inversion method is based on the U-Net architecture with the neural network trained on pairs of synthetic seismic data and CO 2 saturation models. Accordingly, our training establishes a mapping relationship between seismic data and CO 2 saturation models and once fully trained directly estimates CO 2 saturation as a function of subsurface location. We further quantify uncertainties of CO 2 saturation estimates using the Monte Carlo dropout method and a bootstrap aggregating method. For this proof-of-concept study, the CO 2 training models and data are derived from the Kimberlina 1.2 model, a hypothetical 3D geological carbon storage model that is constructed based on various geological and hydrological data from the Southern San Joaquin Basin, California. We perform deep-learning inversion experiments using noise-free and noisy training and test data sets and compare the results. Our modelling experiments show that (1) the deep-learning inversion can estimate 2D distributions of CO 2 fairly well even in the presence of Gaussian random noise and (2) both CO 2 saturation imaging and uncertainty quantification can be done in real time. Our results suggest that the deep-learning inversion method can serve as a robust real-time monitoring tool for geological carbon storage and/or other time-varying reservoir/aquifer properties that result from injection, extraction, and/or other subsurface transport phenomena.

58 GEOSCIENCES↗
DOE OSTI2023

Abisko: Deep codesign of an architecture for spiking neural networks using novel neuromorphic materials

The Abisko project aims to develop an energy-efficient spiking neural network (SNN) computing architecture and software system capable of autonomous learning and operation. The SNN architecture explores novel neuromorphic devices that are based on resistive-switching materials, such as memristors and electrochemical RAM. Equally important, Abisko uses a deep codesign approach to pursue this goal by engaging experts from across the entire range of disciplines: materials, devices and circuits, architectures and integration, software, and algorithms. Here, the key objectives of our Abisko project are threefold. First, we are designing an energy-optimized high-performance neuromorphic accelerator based on SNNs. This architecture is being designed as a chiplet that can be deployed in contemporary computer architectures and we are investigating novel neuromorphic materials to improve its design. Second, we are concurrently developing a productive software stack for the neuromorphic accelerator that will also be portable to other architectures, such as field-programmable gate arrays and GPUs. Third, we are creating a new deep codesign methodology and framework for developing clear interfaces, requirements, and metrics between each level of abstraction to enable the system design to be explored and implemented interchangeably with execution, measurement, a model, or simulation. As a motivating application for this codesign effort, we target the use of SNNs for an analog event detector for a high-energy physics sensor.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Uncertainty quantification of a deep learning fuel property prediction model

Deep learning models are being widely used in the field of combustion. Given the black-box nature of typical neural network based models, uncertainty quantification (UQ) is critical to ensure the reliability of predictions as well as the training datasets, and for a principled quantification of noise and its various sources. Deep learning surrogate models for predicting properties of chemical compounds and mixtures have been recently shown to be promising for enabling data-driven fuel design and optimization, with the ultimate goal of improving efficiency and lowering emissions from combustion engines. In this study, UQ is performed for a multi-task deep learning model that simultaneously predicts the research octane number (RON), Motor Octane Number (MON), and Yield Sooting Index (YSI) of pure components and multicomponent blends. The deep learning model is comprised of three smaller networks: Extractor 1, Extractor 2, and Predictor, and a mixing operator. The molecular fingerprints of individual components are encoded via Extractor 1 and Extractor 2, the mixing operator generates fingerprints for mixtures/blends based on linear mixing operation, and the predictor maps the fingerprint to the target properties. Two different classes of UQ methods, Monte Carlo ensemble methods and Bayesian neural networks (BNNs), are employed for quantifying the epistemic uncertainty. Combinations of Bernoulli and Gaussian distributions with DropConnect and DropOut techniques are explored as ensemble methods. All the DropConnect, DropOut and Bayesian layers are applied to the predictor network. Aleatoric uncertainty is modeled by assuming that each data point has an independent uncertainty associated with it. The results of the UQ study are further analyzed to compare the performance of BNN and ensemble methods. Although this study is confined to UQ of fuel property prediction, the methodologies are applicable to other deep learning frameworks that are being widely used in the combustion community.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Advancing spatiotemporal forecasts of CO 2 plume migration using deep learning networks with transfer learning and interpretation analysis

Accurate and timely forecasts of CO 2 plume distribution throughout the injection and post-injection phases are crucial for detecting plume migration, assessing leakage risks, and supporting operational decisions in geologic carbon storage (GCS). Current convolutional neural network-based approaches primarily focus on spatial information and overlook temporal dependencies in plume distributions, thus limiting their ability to capture dynamic movement effects and provide accurate predictions of plume migration. In this work, we propose two deep learning models, Auto-Encoder (AE)-LSTM and Encoder-Decoder (ED)-ConvLSTM, each uniquely designed to capture both spatial and temporal features. We apply the proposed methods to forecast the dynamic distribution of CO 2 plumes based on 108 reservoir simulations over a 30-year injection and a 30-year post-injection period. The results indicate that the ED-ConvLSTM model outperforms the AE-LSTM model in accurately predicting the spatiotemporal dynamics of CO 2 plume migration, achieving R 2 values above 0.99. To provide a deeper understanding of these model predictions, we employ a gradient-based explanation method on the trained models. This approach provides insights into the influence of input variables on plume migration forecasts and uncovers the underlying prediction mechanisms of the proposed models. Furthermore, we introduce a transfer learning technique, enabling fast and accurate plume migration forecasting in the post-injection phase by leveraging the trained model during the injection phase. This reduces the necessity for extensive data collection or re-training. In conclusion, the methods proposed in our work enhances the performance and interpretability of CO 2 plume migration forecasts, thereby facilitating informed decision-making throughout the entire lifecycle of GCS applications.

58 GEOSCIENCES↗
DOE OSTI2025

Introduction: Neuromorphic Materials

The explosive growth in data collection and the need to process it efficiently, as well as the desire to automate increasingly complex tasks in transportation, medical care, manufacturing, security and many other fields have motivated a growing interest in neuromorphic computing. Unlike the binary, transistorbased ON/OFF logic gates and separate logic and memory functionalities employed in digital computing, neuromorphic computing is inspired by animal brains that use interconnected synapses and neurons to perform processing, storage and transmission of information at the same location, while only consuming ~20 W or less of power. Motivated by the brain’s efficiency, adaptability, self-learning and resiliency qualities, neuromorphic computing can be broadly defined as an approach to processing and storing information using hardware and algorithms inspired by models of biological neural systems. Present research in neuromorphic computing encompasses approaches that vary significantly in their degree of neuro-inspiration, from systems that only incorporate features such as asynchronous, event-driven operation or use crossbar arrays of non-volatile memory (NVM) elements to accelerate deep neural networks (DNNs), to designs that embrace the extreme parallelism, sparsity, reconfigurability, adaptability, complexity and stochasticity observed in nervous systems. The term ‘neuromorphic’ computing is often credited to Carver Mead, who in the 1980s investigated Si-based analog electronics to replicate functions of the animal retina. Earlier important advances in this field include the work of Frank Rosenblatt, who proposed the concept of the perceptron, Bernard Widrow, who used this concept to build one of the first analog neural networks, the Adaline and many other researchers (see ref. 6 for an historical perspective on neuromorphic computing). With the recent increase in the use of artificial intelligence and large language models, and rising concerns over the associated energy costs, interest in neuromorphic hardware has expanded rapidly. According to some estimates, driven largely by the drastic growth in the training use of artificial intelligence (AI) models using the current computing architectures, the energy cost of computing is projected to reach the energy supply worldwide by 2045. Furthermore, while this is not a realistic outcome, it means that, if more efficient computing technologies are not developed -- soon -- the world will soon become one where demand for energy and market constraints limit the continued increase of societal access to AI and cloud services from data centers. Data centers used for training and use of these models consume hundreds of terawatt hours of electricity, already past 4% of the US electricity demand.

Circuits↗
DOE OSTI2022

Soft Actor Critic Based Volt-VAR Co-optimization in Active Distribution Grids

Modern distribution networks are undergoing several technical challenges, such as voltage fluctuations, because of high penetration of distributed energy resources (DERs). This paper proposes a deep reinforcement learning (DRL)-based Volt VAR co-optimization technique for reducing voltage fluctuations as well as power loss under high penetration of DERs. In addition, the proposed approach minimizes the operational cost of the grid. A stochastic policy optimization based soft actor critic (SAC) agent is proposed to configure the optimal set-points of the reactive power outputs of the inverters. The performance of the proposed model is verified on the modified IEEE 34- and 123-bus systems and compared with a base case scenario with no reactive supply by inverters, and a local droop control approach. The results demonstrate that the proposed framework outperforms the conventional droop control method in improving the voltage profile, minimizing the network power loss, and reducing grid operational cost.

—Distribution grids, deep reinforcement learning, ↗