Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Operator Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Initial testing of Alfvén eigenmode feedback control with machine-learning observers on DIII-D

A first of its kind fully data-driven system has been developed and implemented into the DIII-D plasma control system to detect and control Alfvén eigenmodes (AE) in real-time. Susceptibility to fast ion-induced AE is a challenge in fully non-inductive tokamak operation, which significantly reduces fast-particle confinement and results in degraded fusion gain. Controlling AEs in real-time to improve fast-ion confinement is, hence, important for future advanced tokamak fusion reactors. The models were implemented and tested in experiments which showed that neural networks (NN) are highly effective in detecting 5 types of AE (BAE, EAE, LFM, RSAE, TAE) using high resolution ECE. To estimate the neutron deficit, a NN has been trained that outputs the classical neutron rate using similar inputs to NUBEAM. Also a preliminary ML-based proportional control has been designed and gone through initial testing in experiment to use feedback-control on the neutral beam power to achieve desired amplitude of AE modes and neutron deficits. The effect of AEs on fast-ion confinement is measured by analysing the gap in classical neutron rate from the proposed NN-based NUBEAM and the measured neutron rate.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Data-driven cyber-attack detection for photovoltaic systems: A transfer learning approach

With increasing exposure to software-based sensing and control, power systems are facing higher risks of cyber/physical attacks. Here, to ensure system stability and minimize the potential economic losses, it is imperative to monitor the operating states and detect those attacks at the early stage. In this paper, a transfer learning method is proposed to detect cyber-attacks in photovoltaic (PV) systems with much less training data. First of all, two PV systems with a different number of PV inverters and power ratings are analyzed and their attack models are studied. Next, an attack detection Convolutional Neural Network (CNN) model was trained with rich amount of data from PV #1. Then, transfer learning was proposed to transfer the well-trained features from PV #1 to PV #2. Lastly, the attack detection model on PV #2 was trained based on the transferred CNN model. The experiment results show that the proposed transfer learning method achieves better accuracy and a faster convergence rate with a much less training dataset than conventional deep learning.

14 SOLAR ENERGY↗
DOE OSTI2021

Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters

In this report, an adaptive critic design (ACD) approach is proposed to control the phase and voltage of a grid-connected virtual synchronous generator (VSG). The penetration of fast responding inertia-less power converters significantly affect the stability of the power system, especially weak systems such as micro grids. The concept of virtual inertia addresses this concern by virtually emulating the behavior of a synchronous generator. However, the conventional VSG is designed based on two conditions: (i) fixed operating point and (ii) inductive grid connections. The performance of VSGs in low-voltage semi-resistive microgrids is far from optimal. To overcome the aforementioned concerns, a heuristic dynamic programing (HDP) approach is proposed to optimally control grid-connected VSGs. The neural-network-based inherence of the HDP enables the proposed technique to adapt to any impedance angle. The HDP controller includes two subnetworks: (i) the action network that controls the system optimally and (ii) the critic network, which evaluates the effectiveness of the action network. The simulation and experimental results are provided to evaluate the effectiveness of the proposed technique. As shown, the HDP-based approach illustrates a better performance in comparison with the conventional PI-based VSG in various operating conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Next-Level Energy Management in Manufacturing: Facility-Level Energy Digital Twin Framework Based on Machine Learning and Automated Data Collection

This research introduces an energy prediction framework at the facility level supported by automated data collection and machine learning models. It investigates whether reducing the prediction time scale allows for applying more complex machine learning techniques and if those techniques improve the prediction accuracy. The primary advantages of this framework lie in its automation of the energy prediction process and its provision of real-time energy data suitable for use in energy dashboards or digital twins. A sitewide dataset was created by combining 15 min energy and daily production data of five shops—assembly, battery, body (electric), body (gas), and paint—from a globally recognized electric vehicle manufacturer. Various machine learning models were evaluated on daily, weekly, and monthly datasets, including, in increasingly complex order: naïve, simple linear regression, net regularized generalized linear regression, principal component regression, k-nearest neighbor, random forest, and Bayesian regularized neural network. Compared to the current state-of-the-art energy consumption prediction for the industrial facility level, this research investigates more complex models and smaller time intervals for higher accuracy. The findings revealed that the more complex monthly models require a minimum of a year and a half of data to operate, while weekly models demand a year of data to achieve improved accuracy. Daily models can operate with only six months of data but exhibit poor performance due to reduced prediction accuracy of production. Key challenges identified include access to reliable, high-quality energy and production data and the initial demand for human labor.

digital twin↗
DOE OSTI2026

HydraGNN_OPF_GFM_2026 - Ensemble of predictive graph foundation models for power grid applications

This dataset supports research on graph foundation models for optimal power flow (OPF) on electric grids using HydraGNN. It contains heterogeneous graph representations of PGLib-OPF cases spanning systems from 14 to 13,659 buses, together with packed HDF5 datasets for pretraining, feasibility classification, and N-1 contingency analysis. The release includes OPF solution data, downstream fine-tuning datasets, pretrained HeteroSAGE and HeteroHEAT model checkpoints, hyperparameter-optimization summaries across multiple heterogeneous GNN architectures, and aggregated fine-tuning results for sample-efficiency studies. The dataset is designed to enable scalable training, evaluation, and transfer-learning studies for OPF surrogate modeling, including node-level AC-OPF solution prediction, graph-level prediction, feasibility classification, operating-condition generalization, and contingency-response tasks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Robustness of Deep Learning Classification to Adversarial Input on GPUs: Asynchronous Parallel Accumulation Is a Source of Vulnerability

The ability of machine learning (ML) classification models to resist small, targeted input perturbations—known as adversarial attacks—is a key measure of their safety and reliability. We show that floating-point non associativity (FPNA) coupled with asynchronous parallel programming on GPUs is sufficient to result in misclassification, without any perturbation to the input. Additionally, we show that this misclassification is particularly significant for inputs close to the decision boundary and that standard adversarial robustness results may be overestimated up to 4.6 when not considering machine-level details. We first study a linear classifier, before focusing on standard Graph Neural Network (GNN) architectures and datasets used in robustness assessments. We develop a novel black-box attack using Bayesian optimization to discover external workloads that can change the instruction scheduling which bias the output of reductions on GPUs and reliably lead to misclassification. Motivated by these results, we present a new learnable permutation (LP) gradient-based approach to learning floating-point operation orderings that lead to misclassifications. The LP approach provides a worst-case estimate in a computationally efficient manner, avoiding the need to run identical experiments tens of thousands of times over a potentially large set of possible GPU states or architectures. Finally, using instrumentation-based testing, we investigate parallel reduction ordering across different GPU architectures under external background workloads, when utilizing multi-GPU virtualization, and when applying power capping. Our results demonstrate that parallel reduction ordering varies significantly across architectures under the first two conditions, substantially increasing the search space required to fully test the effects of this parallel scheduler-based vulnerability. These results and the methods developed here can help to include machine-level considerations into adversarial robustness assessments, which can make a difference in safety and mission critical applications.

Shanmugavelu, Sanjif [Maxeler Technologies, a Groq↗
DOE OSTI2023

PyTorch Circuit-Aware Bit-Cell Modeling

SAND2024-08549O PyTorch Circuit-Aware Bit-Cell Modeling, a Python library, demonstrates circuit-aware training for analog machine learning accelerators. Built upon PyTorch, this software provides hardware-aware versions of common neural network layers to mitigate non-ideal behavior from novel hardware. The software provides accurate forward and backward pass estimates for hardware-mapped neural networks that are implemented in crossbars (arrays) that contain a non-linear transistor selector device. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Bennett, Christopher↗
DOE OSTI2024

Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units

Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs such as social networks. We demonstrate a novel hardware-software co-design approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations including a planner and vectorization for the gather-scatter operations targeted for Graphcore’s Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed co-design approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes and sparsity. Here, we demonstrate that such a co-design approach can reduce the training time of atomistic GNNs and can improve the performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Classification and Localization of Fracture-Hit Events in Low-Frequency Distributed Acoustic Sensing Strain Rate with Convolutional Neural Networks

Summary Distributed acoustic sensing (DAS) has been used in the oil and gas industry as an advanced technology for surveillance and diagnostics. Operators use DAS to monitor hydraulic fracturing activities, examine well stimulation efficacy, and estimate complex fracture system geometries. Particularly, low-frequency DAS can detect geomechanical events such as fracture hits because hydraulic fractures propagate and create strain rate variations in the rock. Analysis of DAS data today is mostly done post-job and subject to interpretation methods. However, the continuous and dense data stream generated live by DAS poses the opportunity for more efficient and accurate real-time data-driven analysis. The objective of this study is to develop a machine learning-based workflow that can identify and locate fracture-hit events in simulated strain rate responses correlated with low-frequency DAS data. In this paper, “fracture hit” refers to a hydraulic fracture originating from a stimulated well intersecting an offset well. We start with building a single fracture propagation model to produce strain rate patterns observed at a hypothetical monitoring well. This model is used to generate two sets of strain rate responses with one set containing fracture-hit events. The labeled synthetic data are then used to train a custom convolutional neural network (CNN) model for identifying the presence of fracture-hit events. The same model is trained again for locating the event with the output layer of the model replaced with linear units. We achieved near-perfect predictions for both event classification and localization. These promising results prove the feasibility of using CNN for real-time event detection from fiber-optic sensing data. Additionally, we use edge detection techniques to recognize fracture-hit event patterns in strain rate images. The fracture-hit location can be identified using recognized pixels in the image. The accuracy of edge detection-based location identification is also plausible, but edge detection is dependent on the assumption of pattern shape and image quality, hence it is less robust compared to CNN models. This comparison further supports the need for CNN applications in image-based real-time fiber-optic sensing event detection.

Engineering↗
DOE OSTI2020

Novel Fault Location Method for Power Systems Based on Attention Mechanism and Double Structure GRU Neural Network

Fault location is one of the most essential techniques to maintain the stable operation of power systems. A fast and accurate fault location allows operators to restore power grids faster and avoid economic losses. Conventional methods rely on expert knowledge to extract the necessary features (e.g. DWT, DFT). For large systems, more coupling effects of transmission lines require more complex feature engineering, and incomplete features can easily introduce large errors. To overcome this, a deep learning approach without manual feature extraction is introduced to the fault location model under big data application. Towards this end, in the proposed method, the attention mechanism, the Bi-GRU and a dual structure network are applied to analyze the current data from different perspectives. Complete information for the fault features is extracted for the fault location. Based on a time series model and benefit from the ability to internally acquire the information architecture of faulty line, the established model is adaptive to the power grids with very complex topologies. Simulation results indicate that the proposed double-structure model reduces the maximum error and is less affected by noise. In comparison with different structures and different models, the proposed method shows better performance in IEEE 39-bus system.

42 ENGINEERING↗
DOE OSTI2025

Neutrino interaction vertex reconstruction in DUNE with Pandora deep learning

The Pandora Software Development Kit and algorithm libraries perform reconstruction of neutrino interactions in liquid argon time projection chamber detectors. Pandora is the primary event reconstruction software used at the Deep Underground Neutrino Experiment, which will operate four large-scale liquid argon time projection chambers at the far detector site in South Dakota, producing high-resolution images of charged particles emerging from neutrino interactions. While these high-resolution images provide excellent opportunities for physics, the complex topologies require sophisticated pattern recognition capabilities to interpret signals from the detectors as physically meaningful objects that form the inputs to physics analyses. A critical component is the identification of the neutrino interaction vertex. Subsequent reconstruction algorithms use this location to identify the individual primary particles and ensure they each result in a separate reconstructed particle. A new vertex-finding procedure described in this article integrates a U-ResNet neural network performing hit-level classification into the multi-algorithm approach used by Pandora to identify the neutrino interaction vertex. The machine learning solution is seamlessly integrated into a chain of pattern-recognition algorithms. The technique substantially outperforms the previous BDT-based solution, with a more than 20% increase in the efficiency of sub-1 cm vertex reconstruction across all neutrino flavours.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Process prediction and detection of faults using probabilistic bidirectional recurrent neural networks on real plant data

Attaining Industry 4.0 for manufacturing operations requires advanced monitoring systems and real-time data analytics of plant data, among other topics. We propose a Probabilistic Bidirectional Recurrent Network (PBRN) for industrial process monitoring for the early detection of faults. The model is based on a Gated Recurrent Unit (GRU) neural network that allows the model to retain long-term dependencies between sensor data along a time horizon, hence learning the dynamic behavior of the process. To reduce the false-positive detection rate of the model, we compel the model to learn from a highly noisy sensor reading while outputting noise-free sensor outputs. The performance of the proposed model is compared to other data-driven statistical process monitoring schemes using real plant data from an industrial Air Separations Unit (ASU) containing noisy sensor readings. We show that the model can learn from noisy data without reducing its performance. Using two different fault cases, we demonstrate the model’s ability to carry out early fault detection with average false-positive rates of 2.9% and 4.9% for both fault cases. The missed detection rates are 0.1% and 0.2%, respectively.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Attack on Grid Event Cause Analysis: An Adversarial Machine Learning Approach

With the ever-increasing reliance on data for data-driven applications in power grids, such as event cause analysis, the authenticity of data streams has become crucially important. The data can be prone to adversarial stealthy attacks aiming to manipulate the data such that residual-based bad data detectors cannot detect them, and the perception of system operators or event classifiers changes about the actual event. This paper investigates the impact of adversarial attacks on convolutional neural network-based event cause analysis frameworks. We have successfully verified the ability of adversaries to maliciously misclassify events through stealthy data manipulations. The vulnerability assessment is studied with respect to the number of compromised measurements. Furthermore, a defense mechanism to robustify the performance of the event cause analysis is proposed. The effectiveness of adversarial attacks on changing the output of the framework is studied using the data generated by real-time digital simulator (RTDS) under different scenarios such as type of attacks and level of access to data.

Niazazari, Iman↗
DOE OSTI2021

Learning the structure of wind: A data-driven nonlocal turbulence model for the atmospheric boundary layer

In this work, we develop a novel data-driven approach to modeling the atmospheric boundary layer. This approach leads to a nonlocal, anisotropic synthetic turbulence model which we refer to as the deep rapid distortion (DRD) model. Our approach relies on an operator regression problem that characterizes the best fitting candidate in a general family of nonlocal covariance kernels parameterized in part by a neural network. This family of covariance kernels is expressed in Fourier space and is obtained from approximate solutions to the Navier–Stokes equations at very high Reynolds numbers. Each member of the family incorporates important physical properties such as mass conservation and a realistic energy cascade. The DRD model can be calibrated with noisy data from field experiments. After calibration, the model can be used to generate synthetic turbulent velocity fields. To this end, we provide a new numerical method based on domain decomposition which delivers scalable, memory-efficient turbulence generation with the DRD model as well as others. We demonstrate the robustness of our approach with both filtered and noisy data coming from the 1968 Air Force Cambridge Research Laboratory Kansas experiments. Using these data, we witness exceptional accuracy with the DRD model, especially when compared to the International Electrotechnical Commission standard.

17 WIND ENERGY↗
DOE OSTI2022

Semantic Regression for Disentangling Beam Losses in the Fermilab Main Injector and Recycler

Fermilab’s Main Injector enclosure houses two accelerators: the Main Injector (MI) and the Recycler (RR). In periods of joint operation, when both machines contain high intensity beam, radiative beam losses from MI and RR overlap on the enclosure’s beam loss monitoring (BLM) system, making it difficult to attribute those losses to a single machine. Incorrect diagnoses result in unnecessary downtime that incurs both financial and experimental cost. In this work, we introduce a novel neural approach for automatically disentangling each machine’s contributions to those measured losses. Using a continuous adaptation of the popular UNet architecture in conjunction with a novel data augmentation scheme, our model accurately infers the machine of origin on a per-BLM basis in periods of joint and independent operation. Crucially, by extracting beam loss information at varying receptive fields, the method is capable of learning both local and global machine signatures and producing high quality inferences using only raw BLM loss measurements.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Energy and Emission Prediction for Mixed-Vehicle Transit Fleets Using Multi-task and Inductive Transfer Learning

Public transit agencies are focused on making their fixed-line bus systems more energy efficient by introducing electric (EV) and hybrid (HV) vehicles to their fleets. However, because of the high upfront cost of these vehicles, most agencies are tasked with managing a mixed-fleet of internal combustion vehicles (ICEVs), EVs, and HVs. In managing mixed-fleets, agencies require accurate predictions of energy use for optimizing the assignment of vehicles to transit routes, scheduling charging, and ensuring that emission standards are met. The current state-of-the-art is to develop separate neural network models to predict energy consumption for each vehicle class. Although different vehicle classes’ energy consumption depends on a varied set of covariates, we hypothesize that there are broader generalizable patterns that govern energy consumption and emissions. In this paper, we seek to extract these patterns to aid learning to address two problems faced by transit agencies. First, in the case of a transit agency which operates many ICEVs, HVs, and EVs, we use multi-task learning (MTL) to improve accuracy of forecasting energy consumption. Second, in the case where there is a significant variation in vehicles in each category, we use inductive transfer learning (ITL) to improve predictive accuracy for vehicle class models with insufficient data. As this work is to be deployed by our partner agency, we also provide an online pipeline for joining the various sensor streams for fixed-line transit energy prediction. Here, we find that our approach outperforms vehicle-specific baselines in both the MTL and ITL settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Parallel Alternative for Energy-Efficient Neural Network Training and Inferencing

Energy efficiency of training and inferencing with large neural network models is a critical challenge facing the future of sustainable large-scale machine learning workloads. This paper introduces an alternative strategy, called phantom parallelism, to minimize the net energy consumption of traditional tensor (model) parallelism, the most energy-inefficient component of large neural network training. The approach is presented in the context of feed-forward network architectures as a preliminary, but comprehensive, proof-of-principle study of the proposed methodology. We derive new forward and backward propagation operators for phantom parallelism, implement them as custom autograd operations within an end-to-end phantom parallel training pipeline and compare its parallel performance and energy-efficiency against those of conventional tensor parallel training pipelines. Formal analyses that predict lower bandwidth and FLOP counts are presented with supporting empirical results on up to 256 GPUs that corroborate these gains. Experiments are shown to deliver ∼50% reduction in the energy consumed to train FFNs using the proposed phantom parallel approach when compared with conventional tensor parallel methods. Additionally, the proposed approach is shown to train smaller phantom models to the same model loss on smaller GPU counts as larger tensor parallel models on larger GPU counts offering the possibility for even greater energy savings.

Seal, Sudip [ORNL] (ORCID:0000000332330656)↗
DOE OSTI2022

Dimensionally Reduced Model for Rapid and Accurate Prediction of Gas Saturation, Pressure, and Brine Production in a CO 2 Storage Application: Case Study Using the SACROC Field as Part of SMART Task 5

This technical report presents work conducted by the sub surface analysis team of the Strategic Systems Analysis & Engineering group at NETL for Task 5 of SMART Phase 1. This study involved the development of deep learning models for CO 2 geologic storage that are capable of accurate prediction of spatio-temporal outputs of CO 2 saturation, pressure, and brine production in three dimensional space over a storage operation's injection and post-injection timeframes. The model framework involves ensembling multi-layer encoder networks that provide dimesionality reduction of geologic inputs with fully connected long short-term memory (LSTM) neural networks that generate time-series prediction This approach offers a means to maximize training time efficiency, reduce computational memory burden, and minimize prediction turnaround.

54 ENVIRONMENTAL SCIENCES↗