Search NASA⌕ Search

SEARCH · Search NASA

Results for “Autoencoder neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Temporally-consistent koopman autoencoders for forecasting dynamical systems

Absence of sufficiently high-quality data often poses a key challenge in data-driven modeling of high-dimensional spatio-temporal dynamical systems. Koopman Autoencoders (KAEs) harness the expressivity of deep neural networks (DNNs), the dimension reduction capabilities of autoencoders, and the spectral properties of the Koopman operator to learn a reduced-order feature space with simpler, linear dynamics. However, the effectiveness of KAEs is hindered by limited and noisy training datasets, leading to poor generalizability. To address this, we introduce the Temporally-Consistent Koopman Autoencoder (tcKAE), designed to generate accurate long-term predictions even with limited and noisy training data. This is achieved through a consistency regularization term that enforces prediction coherence across different time steps, thus enhancing the robustness and generalizability of tcKAE over existing models. We provide analytical justification for this approach based on Koopman spectral theory and empirically demonstrate tcKAE’s superior performance over state-of-the-art KAE models across a variety of test cases, including simple pendulum oscillations, kinetic plasma, and fluid flow data.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Emulation With Uncertainty Quantification of Regional Sea‐Level Change Caused by the Antarctic Ice Sheet

Abstract Projecting regional sea‐level change under various climate‐change scenarios typically involves running forward simulations of the Earth's gravitational, rotational and deformational (GRD) response to ice‐mass change, which requires substantial computational cost if applied to probabilistic frameworks requiring thousands to millions of samples. Here we build emulators of regional sea‐level change at 27 coastal locations, due to the GRD effects associated with future Antarctic Ice Sheet mass change over the 21st century. The emulators are evaluated against a numerical sea‐level model applied to an ensemble of ice‐sheet model simulations of the Antarctic Ice Sheet through 2100. We build a physics‐based emulator using a recent sensitivity kernel approach and compare it to machine learning based emulators (neural network and conditional variational autoencoder methods). In order to quantify uncertainty, we derive well‐calibrated prediction intervals for regional sea‐level change via split‐conformal inference and linear regression, and show that Monte Carlo dropout does not yield well‐calibrated uncertainties in this instance. We also demonstrate substantial gains in computational efficiency using both the physics‐based emulator and neural networks in comparison to the numerical model for the complete regional sea‐level solution. Overall, we find the physics‐based emulator modestly outperforms the machine learning emulators for this problem.

58 GEOSCIENCES↗

AI Denoising to Accelerate Detector Simulation

Detector simulation is critical to experimental HEP; however this simulation (commonly done through toolkits such as Geant4) is computationally intensive. Performance can be improved somewhat through technical optimization, but more is needed. Using machine learning (ML) to accelerate simulation is a promising field, however efforts to use generative adversarial networks (GANs) or optimized autoencoders have faced issues. Using convolutional neural networks (CNNs) for denoising has been successful in non-HEP applications such as image processing. This poster investigates the efficacy of using CNNs to denoise Geant4 simulations. This could increase the accuracy of simulations performed under settings designed to increase computational efficiency.

Franklin, Lena↗

Sensor Fault Detection in Smart Extraterrestrial Habitats Using Unsupervised Learning

Various types of sensors are needed to monitor the health state of smart deep-space habitats. However, measured data can be affected by sensor faults, which influence the health management system and consequently the decision-making. In this paper, an unsupervised learning approach based on convolutional autoencoders (CAEs) is developed to detect anomalies in temperature and pressure sensors. The proposed method is systematically investigated using a habitat simulator (HabSim). Several illustrative examples are demonstrated in the nominal and hazardous states of the habitat, including micrometeorite impact and fire scenarios. The performance of the proposed method using CAEs is compared with that of existing methods using auto-associative neural networks (AANNs) and variational autoencoders. This comparison is based on typical evaluation metrics, including precision, recall, F1 score, training time, and testing time. The effect of temperature–pressure coupling on the detection performance of CAEs and AANNs is explored by training different data-driven models, including one with temperature sensors, one with pressure sensors, and one with both temperature and pressure sensors. The effect of the number of faulty sensors on the performance of CAEs is studied, as with an increase in the number of faulty sensors, redundant information among the sensors is reduced. The capability of CAEs to change the number of sensors without redesigning the network architecture and retraining the neural network is investigated and demonstrated. The capabilities and limitations of the proposed solution are discussed.

Zixin Wang↗

Virtual Battery Parameter Identification using Transfer Learning based Stacked Autoencoder

Recent studies have shown that the aggregated dynamic flexibility of an ensemble of thermostatic loads can be modeled in the form of a virtual battery. The existing methods for computing the virtual battery parameters require the knowledge of the first-principle models and parameter values of the loads in the ensemble. In real-world applications, however, it is likely that the only available information are end-use measurements such as power consumption, room temperature, device on/off status, etc., while very little about the individual load models and parameters are known. We propose a transfer learning based deep network framework for calculating virtual battery state of a given ensemble of flexible thermostatic loads, from the available end-use measurements. This proposed framework extracts first order virtual battery model parameters for the given ensemble. We illustrate the effectiveness of this novel framework on different ensembles of ACs and WHs.

Stacked Autoencoder, Convolution Neural Network, v↗

Scaling kinetic Monte-Carlo simulations of grain growth with combined convolutional and graph neural networks

Graph neural networks (GNN) have emerged as a promising machine learning method for microstructure simulations such as grain growth. However, accurate modeling of realistic grain boundary networks requires large simulation cells, which GNN has difficulty scaling up to. To alleviate the computational costs and memory footprint of GNN, we suggest a hybrid architecture combining a convolutional neural network (CNN) based bijective autoencoder to compress the spatial dimensions, and a GNN that evolves the microstructure in the latent space of reduced spatial sizes. Our results demonstrate that the new design significantly reduces computational costs with using fewer message passing layer (from 12 down to 3) compared with GNN alone. The reduction in computational cost becomes more pronounced as the spatial size increases, indicating strong computational scalability. For the largest mesh evaluated (160 3 ), our method reduces memory usage and runtime in inference by 117× and 115×, respectively, compared with GNN-only baseline. More importantly, it shows higher accuracy and stronger spatiotemporal capability than the GNN-only baseline, especially in long-term testing. Such combination of scalability and accuracy is essential for simulating realistic material microstructures over extended time scales. The improvements can be attributed to the bijective autoencoder’s ability to compress information losslessly from spatial domain into a high dimensional feature space, thereby producing more expressive latent features for the GNN to learn from, while also contributing its own spatiotemporal modeling capability. Training data are generated from stochastic grain growth simulations, providing realistic variability for learning robust microstructure evolution. Comprehensive system validation confirms that the model is accurate, robust, and scalable.

36 MATERIALS SCIENCE↗

INSPIRED: Inelastic neutron scattering prediction for instantaneous results and experimental design

Inelastic neutron scattering (INS) has unique advantages in probing how atoms vibrate and how the vibrations propagate and interact. Such dynamic information is crucial in understanding various material properties, from heat capacity, thermal conductivity, phase transitions, and chemical reactions to more exotic quantum behavior. The analysis and interpretation of the INS spectra often start from a model structure of the sample, followed by a series of calculations to obtain the simulated spectra to compare with experiments. The conventional way to perform such calculations usually requires significant time, computing resources, and specialized expertise. Here, we present a new program named INSPIRED (Inelastic Neutron Scattering Prediction for Instantaneous Results and Experimental Design), which enables users to perform rapid INS simulations in several different ways on their personal computers in just a few clicks, with the crystal structure as the only input file. Specifically, the users can choose a pre-trained symmetry-aware neural network (coupled with an autoencoder) to predict the phonon density of states (DOS), 1D S(E) and 2D S(|Q|,E) spectra for any given structure. One can also choose an existing density functional theory (DFT) calculation from a database (containing over 12,000 crystals), and quickly obtain the simulated INS spectra for single crystals and powders. It is also possible to use pre-trained universal machine learning force fields to relax a given crystal structure, calculate the phonon dispersion and DOS, and, subsequently, the INS spectra. All these functions are implemented with a PyQt graphic user interface. Finally, we expect these new tools will benefit broad user communities and significantly improve the efficiency of experiment design, execution, and data analysis for INS.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

RX-ADS: Interpretable Anomaly Detection Using Adversarial ML for Electric Vehicle CAN Data

Recent year has brought considerable advancements in Electric Vehicles (EVs) and associated infrastructures/ communications. Intrusion Detection Systems (IDS) are widely deployed for anomaly detection in such critical infrastructures. This paper presents an Interpretable Anomaly Detection System (RX-ADS) for intrusion detection in CAN protocol communication in EVs. Contributions include: 1) window based feature extraction method; 2) deep Autoencoder based anomaly detection method; and 3) adversarial machine learning based explanation generation methodology. The presented approach was tested on two benchmark CAN datasets: OTIDS and Car Hacking. The anomaly detection performance of RX-ADS was compared against the state-of-the-art approaches on these datasets: HIDS and GIDS. The RX-ADS approach presented performance comparable to the HIDS approach (OTIDS dataset) and has outperformed HIDS and GIDS approaches (Car Hacking dataset). Further, the proposed approach was able to generate explanations for detected abnormal behaviors arising from various intrusions. Furthermore, these explanations were later validated by information used by domain experts to detect anomalies. Other advantages of RX-ADS include: 1) the method can be trained on unlabeled data; 2) explanations help experts in understanding anomalies and root course analysis, and also help with AI model debugging and diagnostics, ultimately improving user trust in AI systems.

42 ENGINEERING↗

Neural Image Compression: Generalization, Robustness, and Spectral Biases

Recent advances in neural image compression (NIC) have resulted in models which are starting to outperform traditional codecs. While this has led to growing excitement about using these methods in real-world applications, the successful adoption of any machine learning system (including NIC) in the wild requires it to generalize (and be robust) to unseen distribution shifts at deployment time. Unfortunately, current research lacks comprehensive datasets and informative tools to evaluate and understand compression performance in real-world settings. To bridge this crucial gap, first, this paper presents a comprehensive benchmark suite to evaluate the out-of-distribution (OOD) performance of image compression methods. Specifically, we design CLIC-C and Kodak-C by introducing 15 common corruptions to popular CLIC and Kodak benchmarks. Next, we propose spectrally inspired introspection tools to gain a deeper understanding of errors introduced by image compression methods as well as their OOD performance. To this end, we carry out a detailed performance comparison of the classical codec with various variants of NIC (e.g., original, variable rate, pruned), revealing intriguing findings that challenge our current understanding of the strengths and limitations of NIC. Finally, we corroborate our empirical findings with theoretical analysis, providing an in-depth view of the OOD performance of NIC. Our benchmarks, spectral introspection tools, and findings provide a crucial bridge to the real-world adoption of NIC. We hope that our work will propel future efforts in designing more robust and generalizable NIC methods.

neural networks, Variational Autoencoder, robustne↗

Spatio-Temporal Deep Graph Network for Event Detection, Localization, and Classification in Cyber-Physical Electric Distribution System

This work proposes a deep graph learning framework to identify, locate, and classify power, cyber, and cyber power events at the distribution system level. The proposed algorithm jointly exploits spatial, temporal, and node-level cyber and physical data features. The developed graph neural network, together with a deep autoencoder, utilizes physical measurements from distribution level phasor measurement units and cyber data from communication network logs. The spatial structure of the synchrophasor measurements and network is incorporated through a weighted adjacency matrix. The temporal structure is incorporated by defining a spatial operation in the gated recurrent unit. This spatio-temporal learning element resides inside a power event detection, localization, and classification module that provides the degree of confidence for an event label. To accurately pinpoint the location of an event to the nearest bus equipped with a measurement unit, a combination of squared error and proximity score is utilized. Also included is a cyber event detection module that employs heteroskedasticity to analyze the significance of various cyber features during different types of attacks. Finally, a dual-bit cyber-power decision table determines the nature of the event. The proposed method is validated on two distribution systems modeled in OPAL-RT/Hypersim with limited phasor measurement units for different possible physical and cyber events. Further analyses include comparison with other state-of-the-art methods and validation in the presence of measurement noise. As a result, our method outperforms existing approaches and achieves an average detection accuracy of 97.97%, F1-score of 96.88%, precision of 96.53%, and recall of 98.57%.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data Using Unstructured Spatial Discretizations

In this work, we propose a nonlinear manifold learning technique based on deep convolutional autoencoders that is appropriate for model order reduction of physical systems in complex geometries. Convolutional neural networks have proven to be highly advantageous for compressing data arising from systems demonstrating a slow-decaying Kolmogorov n-width. However, these networks are restricted to data on structured meshes. Unstructured meshes are often required for performing analyses of real systems with complex geometry. Our custom graph convolution operators based on the available differential operators for a given spatial discretization effectively extend the application space of deep convolutional autoencoders to systems with arbitrarily complex geometry that are typically discretized using unstructured meshes. We propose sets of convolution operators based on the spatial derivative operators for the underlying spatial discretization, making the method particularly well suited to data arising from the solution of partial differential equations. We demonstrate the method using examples from heat transfer and fluid mechanics and show better than an order of magnitude improvement in accuracy over linear methods.

97 MATHEMATICS AND COMPUTING↗

Time series anomaly detection in power electronics signals with recurrent and ConvLSTM autoencoders

The anomalies in the high voltage converter modulator (HVCM) remain a major down time for the spallation neutron source facility, that delivers the most intense neutron beam in the world for scientific materials research. In this work, we propose neural network architectures based on Recurrent AutoEncoders (RAE) to detect anomalies ahead of time in the power signals coming from the HVCM. Bi-directional gated recurrent unit, bi-directional long-short term memory (LSTM), and convolutional LSTM (ConvLSTM) are developed, trained, and tested using real experimental signals from the HVCM module. The results show a good performance of the proposed RAE models, achieving precision up to 91%, recall up to 88%, false omission rate as low as 20% (i.e. 80% of the anomalies were detected), and area under the ROC curve up to 0.9. The three RAE models provide very comparable performance, with LSTM showing slightly better performance than GRU and ConvLSTM. The RAE models are benchmarked against other anomaly detection methods, including isolation forest, support vector machine, local outlier factor, feedforward and convolutional autoencoders, and others; showing a better performance. Here, the results of this study demonstrate the promising potential of RAE in anomaly detection for real-world power systems, and for increasing the reliability of the HVCM modules in the spallation neutron source.

42 ENGINEERING↗

Statistical and Neural Network for Real Sensor-Data-Driven Anomaly Detection in Nuclear Applications

Anomaly detection (AD) in sensor data is critical to ensure uninterrupted functionality of nuclear power plants (NPPs). Consequently, AD model validation through real-world sensor data is important for applications in nuclear facilities. In this paper, we propose an Autoencoder (AE)—a multi-layered neural network, for AD in sensor data from an operational NPP testbed. Since the dataset lacks labels for irregularities, we introduce random noise and label them to effectively train our model. The proposed AE model assigns a higher reconstruction error to the abnormal samples that deviate from those encountered during the training phase and uses the reconstruction loss to detect anomalies in a representative imbalanced dataset. We also introduce an analytical solution—seasonal trend decomposition (STD)—as another AD scheme for identifying irregularities withinthe same time-series dataset. In contrast to the AE model which relies on reconstruction loss, the STD scheme decomposes the entire dataset into its trend, seasonality, and residual components to pinpoint irregularities. Our findings indicate that the proposed AE and STD models individually achieve recall scores of 97% and 92%, respectively. We validate the performance of the two models on both balanced and imbalanced data. We further solidify the results by picking the combined selected anomalies of the two solutions with an "AND" operator for more reliable predictions.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Statistical and Neural Network for Real Sensor-Data-Driven Anomaly Detection in Nuclear Applications

Anomaly detection (AD) in sensor data is critical to ensure uninterrupted functionality of nuclear power plants (NPPs). Consequently, validation of AD models through real-world sensor data is important for their application in nuclear facilities. In this paper, we propose an Autoencoder (AE)— a multi-layered neural network, for AD in sensor data from an operational NPP testbed. Since the dataset lacks labels for irregularities, we introduce random noise and label them to effectively train our model. The proposed AE model assigns a higher reconstruction error to the abnormal samples that deviate from those encountered during the training phase and uses the reconstruction loss to detect anomalies in a representative imbalanced dataset. We also introduce an analytical solution—seasonal trend decomposition (STD) — as another AD scheme for identifying irregularities within the same time-series dataset. In contrast to the AE model which relies on reconstruction loss, the STD scheme decomposes the entire dataset into its trend, seasonality, and residual components to pinpoint irregularities. Our findings indicate that the proposed AE and STD models individually achieve recall scores of 97% and 92%, respectively. We also validate the performance of the two models on both balanced and imbalanced data. We further solidify the results by picking the combined selected anomalies of the two solutions with an "AND" operator for more reliable predictions.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Cause identification of electromagnetic transient events using spatiotemporal feature learning

This paper presents a spatiotemporal feature learning method for cause identification of electromagnetic transient events in power grids. The proposed method is formulated based on the availability of time-synchronized high-frequency measurements and using the convolutional neural network as the spatiotemporal feature representation along with softmax function for the classification. Despite the existing threshold-based, or energy-based events analysis methods, such as support vector machine autoencoder, and tapered multi-layer perceptron neural network, the proposed feature learning is carried out with respect to both time and space. The effectiveness of the proposed feature learning and the subsequent cause identification is validated through the Electromagnetic Transients Program (EMTP) simulation of different events such as line energization, capacitor bank energization, lightning, fault, and high-impedance fault in the IEEE 30-bus, and the real-time digital simulation of the Western System Coordinating Council (WSCC) 9-bus system.

42 ENGINEERING↗

Real-time Intrusion Detection for High-bandwidth Research Networks using Unsupervised Deep Learning

Nowadays, data generated by large-scale scientific experiments are on the scale of petabytes per month. These data are transferred through dedicated high-bandwidth networks across distributed collaboration sites for processing, storage, and analysis. Like general purpose networks, research networks experience attacks. However, monitoring traffic in such high-bandwidth network environments is challenging with conventional deep packet inspection (DPI)- based network intrusion detection systems (NIDS). Recently, machine learning (ML) algorithms have shown potential in detecting anomalous patterns with better generalizability and faster speed. Compared to the general purpose networks, traffic movement in research network is less volatile and more likely to benefit from the ML-based anomaly analysis. We present ThunderSecure, an efficient AI-powered system for detecting anomalies in 100G research networks. ThunderSecure implements a high-throughput packet processing infrastructure using multi-cores and GPUs. It takes time-series statistics from a variety of network measurements and uses 2D Convolutional Neural Network (CNN) to explore both spatial and temporal dependencies in network data stream. Patterns of science data traffic are learned with a one-class neural network, which blends an Adversarial Autoencoder (AAE) with Gaussian mixture density estimation. Testing traffic flows exhibiting significant deviations from the learned baseline of normality are marked as anomalies. We trained ThunderSecure on hundreds of billions of science data packets recorded from a 100G research network at Fermi National Accelerator Laboratory. The detection performance was evaluated on traffic captured from the same research network days and weeks after the training with different types of attack flows injected. To the best of our knowledge, this is the first ML work in the area of network anomaly detection that has been validated on such extreme scale datasets. Results show that ThunderSecure can recognize science data traffic that are captured long after the training and make nearly certain detection on those with anomalous flows injected, even in the case when the anomaly-to-normal mixing ratio is 0.1%.

Gong, Qian↗

Investigation of process history and underlying phenomena associated with the synthesis of plutonium oxides using Vector Quantizing Variational Autoencoder

Accurate, high throughput, and unbiased analysis of plutonium oxide particles is needed for analysis of the phenomenology associated with process parameters in their synthesis. Compared to qualitative and taxonomic descriptors, quantitative descriptors of particle morphology through scanning electron microscopy (SEM) have shown success in analyzing process parameters of uranium oxides. Among other candidates, a neural network called a Vector Quantizing Variational Autoencoder (VQ-VAE) has shown the ability to quantitatively describe particle morphology to attain >85% accuracy in identifying uranium oxide processing routes. We utilize a VQ-VAE to quantitatively describe plutonium dioxide (PuO 2 ) particles created in a designed experiment and investigate their phenomenology and prediction of their process parameters. PuO 2 was calcined from Pu(III) oxalates that were precipitated under varying synthetic conditions that related to concentrations, temperature, addition and digestion times, precipitant feed, and strike order; the surface morphology of the resulting PuO 2 powders were analyzed by SEM. A pipeline was developed to extract and quantify useful image representations for individual particles with the VQ-VAE, then further reduce the dimensionality of the feature space using a bottlenecking neural network fit to perform multiple classification tasks simultaneously. The reduced feature space could predict process parameters with greater than 80% accuracies for some parameters with a single particle. They also showed utility for grouping particles with similar surface morphology characteristics together. Both the clustering and classification results reveal valuable information regarding which chemical process parameters chiefly influence the PuO 2 particle morphologies: strike order and oxalic acid feedstock. Doing the same analysis with multiple particles was shown to improve the classification accuracy on each process parameter over the use of a single particle, with statistically significant results generally seen with as few as four particles in a sample.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Data reduction in deterministic neutron transport calculations using machine learning

Neutron cross section matrices for fission and scattering data are required for each material, temperature, and enrichment level to calculate the neutron transport equation accurately. Here, this information can be a limiting factor when using the multigroup discrete ordinates (S N ) method when the number of energy groups is large. Machine Learning (ML) can be used to replace the need for the cross section matrices by reproducing the function that maps the scalar flux to the scattering and fission sources. Through the use of autoencoders and Deep Jointly-Informed Neural Networks (DJINN), the data storage requirements are reduced by 94% of the original data for a 618 group problem. This is accomplished while preserving the scalar flux, maintaining generality, and decreasing wall clock times.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗