Search NASA⌕ Search

SEARCH · Search NASA

Results for “recurrent neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Woven ceramic matrix composite surrogate model based on physics-informed recurrent neural network

A recurrent neural network (RNN) based surrogate model is developed to emulate the nonlinear constitutive behavior of woven ceramic matrix composites (CMCs) driven by matrix damage at multiple length scales. Physics-informed constraints are introduced into the surrogate model through regularization to ground the prediction in physics and improve its predictive capabilities. Training data is generated using the multiscale generalized method of cells (MSGMC) approach coupled with a matrix damage model. This coupling permits simulating the nonlinear behavior of woven CMCs based on constituent response at the micro-, meso-, and macroscales. The multiscale repeating unit cell is loaded under non-monotonic conditions including multiple load / unload cycles and tension / compression. The fiber volume fraction as well as the intra- and intertow void volume fractions are also varied in the generation of training data. Therefore, the RNN-based surrogate model is tasked with predicting, as a function of variable input strain sequence and fiber and void volume fractions, the resulting stress versus strain response while satisfying physical constraints such as positive semi-definiteness of the tangent stiffness matrix and linear elastic unloading. Further, the trained surrogate model effectively matches the stress versus strain response and successfully predicts the tangent modulus throughout the loading regime. Neural network based surrogate models can offer efficient alternatives to running computationally intensive multiscale material models to simulate the nonlinear response of large structural models. Therefore the presented work provides evidence towards the feasibility of developing, training, and running such models for CMCs with complex architectures, nonlinear multiaxial material response, and under non-monotonic loading conditions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Hybrid Recurrent Neural Network Modeling for Traffic Delay Prediction at Signalized Intersections Along an Urban Arterial

This paper studies the traffic delay prediction modeling for multiple signalized intersections along the Ala Moana Boulevard and Nimitz Highway in Hawaii. Several machine learning (ML) based approaches have been studied in the literature, and most of them focused on prediction accuracy rather than the end use of real-time control and implementation. These ML models tend to be very complex and non-linear in nature, making it challenging to achieve fast inferences and are computationally heavy for real-time signal control implementation. As such, in this paper, a simple yet accurate hybrid modeling method is proposed to predict traffic delay one-step ahead with the model made suitable for real-time implementation to control traffic flow. Since real-time road-side measurements are recorded in unstructured form, the paper also discusses other issues related to data extraction and the pre-processing process. Finally, a simple signal control loop is developed to demonstrate the proposed modeling approach, which has shown advantages in model accuracy and computation efficiency compared against several existing modeling methods.

42 ENGINEERING↗
DOE OSTI2022

Path sampling of recurrent neural networks by incorporating known physics

Recurrent neural networks have seen widespread use in modeling dynamical systems in varied domains such as weather prediction, text prediction and several others. Often one wishes to supplement the experimentally observed dynamics with prior knowledge or intuition about the system. While the recurrent nature of these networks allows them to model arbitrarily long memories in the time series used in training, it makes it harder to impose prior knowledge or intuition through generic constraints. In this work, we present a path sampling approach based on principle of Maximum Caliber that allows us to include generic thermodynamic or kinetic constraints into recurrent neural networks. We show the method here for a widely used type of recurrent neural network known as long short-term memory network in the context of supplementing time series collected from different application domains. These include classical Molecular Dynamics of a protein and Monte Carlo simulations of an open quantum system continuously losing photons to the environment and displaying Rabi oscillations. Our method can be easily generalized to other generative artificial intelligence models and to generic time series in different areas of physical and social sciences, where one wishes to supplement limited data with intuition or theory based corrections.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1993

Discrete Recurrent Neural Networks for Grammatical Inference

Recurrent neural networks have recently been shown to have the ability to learn regular and context-free grammars from examples. We show that while conventional analog recurrent networks try to form clusters in activation space to represent discrete states of the grammars during learning, and can be successful in doing so, the clusters so formed tend to become unstable as longer and longer test input strings are presented to the network.

Neural↗
DOE OSTI2020

Time-Sequenced Flow Field Prediction in an Optima Spark-Ignition Direct-Injection Engine Using Bidirectional Recurrent Neural Network (bi-RNN) with Long Short-Term Memory

To further improve the energy conversion efficiency of internal combustion engine, the transient and complex air flow movement inside the cylinder needs to be better understood and controlled. Although the in-cylinder flow fields are highly stochastic with strong cycle-to-cycle fluctuations, machine learning can still provide an efficient way to learn and regress the complex flow movement process inside the cylinder. In this work, a bidirectional recurrent neural network (bi-RNN) model with long short-term memory was applied to predict the in-cylinder flow fields at different time steps using training data from mull-cycle particle image velocimetry (PIV) measurements. To evaluate the agreement between the true and predicted flow fields, structure and magnitude comparison indices are calculated both globally and locally. The comparison results show that the bi-RNN model can accurately predict the bulk flow and vortex motions from early intake stroke to compression stroke. This work demonstrates that the machine learning model has the potential to predict the underlying dynamics of the interaction between in-cylinder flows and provides a reliable way to improve temporal resolution in PIV flow data to better reveal transient in-cylinder flow features.

Bi-RNN model↗
DOE OSTI2024

Secure authentication using recurrent neural networks

A computer-implemented method of user authentication is provided. The method comprises combining, by a computer system, a user recurrent neural network with a system recurrent neural network to form a unique combined recurrent neural network. The user recurrent neural network is configured to generate a unique user key, and the system recurrent neural network is configured to generate a system key. The computer system inputs a predetermined input into the combined recurrent neural network, and the combined recurrent neural network generates a unique combined key from the input, wherein the combined key differs from both the user key and system key. The computer system then associates the combined key with a unique access authorization to authenticate a user.

Aimone, James Bradley↗
DOE OSTI2018

Understanding Selection And Diversity For Evolution Of Spiking Recurrent Neural Networks

Evolutionary optimization or genetic algorithms have been used to optimize a variety of neural network types, including spiking recurrent neural networks, and are attractive for many reasons. However, a key impediment to their widespread use is the potential for slow training times and failure to converge to a good fitness value in a reasonable amount of time. In this work, we evaluate the effect of different selection algorithms on the performance of an evolutionary optimization method for designing spiking recurrent neural networks, including those that are meant to be deployed in a neuromorphic system. We propose a selection approach that utilizes a richer understanding of the fitness of an individual network to inform the selection process and to promote diversity in the population. We show that including this feature can provide a significant increase in performance over utilizing a standard selection approach.

Schuman, Catherine↗
DOE OSTI2023

Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4ml

Abstract Recurrent neural networks have been shown to be effective architectures for many tasks in high energy physics, and thus have been widely adopted. Their use in low-latency environments has, however, been limited as a result of the difficulties of implementing recurrent architectures on field-programmable gate arrays (FPGAs). In this paper we present an implementation of two types of recurrent neural network layers—long short-term memory and gated recurrent unit—within the hls4ml framework. We demonstrate that our implementation is capable of producing effective designs for both small and large models, and can be customized to meet specific design requirements for inference latencies and FPGA resources. We show the performance and synthesized designs for multiple neural networks, many of which are trained specifically for jet identification tasks at the CERN Large Hadron Collider.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Solving differential equations with unknown constitutive relations as recurrent neural networks

We solve a system of ordinary differential equations with an unknown functional form of a sink (reaction rate) term. We assume that the measurements (time series) of state variables are partially available, and use a recurrent neural network to “learn” the reaction rate from this data. This is achieved by including discretized ordinary differential equations as part of a recurrent neural network training problem. We extend TensorFlow’s recurrent neural network architecture to create a simple but scalable and effective solver for the unknown functions, and apply it to a fedbatch bioreactor simulation problem. Use of techniques from recent deep learning literature enables training of functions with behavior manifesting over thousands of time steps. Our networks are structurally similar to recurrent neural networks, but differ in purpose, and require modified training strategies.

deep learning↗
DOE OSTI2025

Short-Term Forecasting of Thermostatic and Residential Loads Using Long Short-Term Memory Recurrent Neural Networks

Internet of Things (IoT) devices in smart grids enable intelligent energy management for grid managers and personalized energy services for consumers. Investigating a smart grid with IoT devices requires a simulation framework with IoT devices modeling. However, there lack comprehensive study on the modeling of IoT devices in smart grids. This paper investigates the IoT device modeling of a thermostatic load and implements the recurrent neural networks model for short-term load forecasting in this IoT-based thermostatic load. The recurrent neural network structure is leveraged to build a load forecasting model on temporal correlation. The temporal recurrent neural network layers including long short-term memory cells are employed to learn the data from both the simulation platform and New South Wales residential datasets. The simulation results are provided for demonstration.

electric load forecasting↗
DOE OSTI2020

Long term predictions of coal fired power plant data using evolved recurrent neural networks

Abstract This work presents an investigation into the ability of recurrent neural networks (RNNs) to provide long term predictions of time series data generated by coal fired power plants. While there are numerous studies which have used artificial neural networks (ANNs) to predict coal plant parameters, to the authors’ knowledge these have almost entirely been restricted to predicting values at the next time step, and not farther into the future. Using a novel neuro-evolution strategy called Evolutionary eXploration of Augmenting Memory Models (EXAMM), we evolved RNNs with advanced memory cells to predict per-minute plant parameters and per-hour boiler parameters up to 8 hours into the future. These data sets were challenging prediction tasks as they involve spiking behavior in the parameters being predicted. While the evolved RNNs were able to successfully predict the spikes in the hourly data they did not perform very well in accurately predicting their severity. The per-minute data proved even more challenging as medium range predictions miscalculated the beginning and ending of spikes, and longer range predictions reverted to long term trends and ignored the spikes entirely. We hope this initial study will motivate further study into this highly challenging prediction problem. The use of fuel properties data generated by a new Coal Tracker Optimization (CTO) program was also investigated and this work shows that their use improved predictive ability of the evolved RNNs.

Automation & Control Systems↗
DOE OSTI2025

River Dissolved Oxygen Prediction Using Machine Learning Models and Wireless Sensor Measurements

Simultaneous flooding&heat and droughts&heat events can potentially destabilize hydro-meteorological conditions to deteriorate the water quality of Neches River. Machine learning (ML) models utilizing wireless sensor measurements have been applied to predict water quality and optimize various water management strategies. This study aims to develop ML models to predict dissolved oxygen (DO) prediction under various hydro-meteorological conditions and enhance water management decision-making. Wireless sensor measurements of DO, water temperature, sample depth, conductivity, turbidity, and pH, along with discharge from the United States Geological Survey stations, are collected for model inputs at the Pine Island Bayou C749 station (PIB-C749) and Neches River Saltwater Barrier (SWB). Multilayer perceptron neural networks, recurrent neural networks, long short-term memory (LSTM), and bidirectional LSTM (BiLSTM) with and without attention mechanism (AT) are tested to determine the best model, which is applied the rolling forecast method to predict 14-day DO. Traditional and recurrent transfer learning (TL and RTL) methods are adopted to overcome insufficient data at the SWB. The input feature importance analysis using the integrated gradients (IG) algorithm is applied to determine dominant inputs. The results show LSTM-based models are capable handling long sequential data. AT-BiLSTM and RTL-LSTM demonstrate the best performance at the PIB-C749 (RMSE=0.054) and the SWB (RMSE=0.028), respectively. TL and RTL methods significantly improve model performance at the SWB. DO, temperature, and pH show higher importance, consistent with hydrodynamics and water chemistry. Both best models are applied to predict 14-day DO and demonstrate reasonable performance for decision-making. Hydro-meteorological conditions of 2017 flood and 2012 drought events are simulated and reveal that possible hypoxia occurs after flooding due to increasing temperature and turbidity, and DO concentration decreases significantly under heat and drought conditions. In conclusion, LSTM-based models utilizing wireless sensor data can be a timely and effective approach to make appropriate decisions on water resource management.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions

Here we propose a new type of neural networks, Kronecker neural networks (KNNs), that form a general framework for neural networks with adaptive activation functions. KNNs employ the Kronecker product, which provides an efficient way of constructing a very wide network while keeping the number of parameters low. Our theoretical analysis reveals that under suitable conditions, KNNs induce a faster decay of the loss than that by the feed-forward networks. This is also empirically verified through a set of computational examples. Furthermore, under certain technical assumptions, we establish global convergence of gradient descent for KNNs. As a specific case, we propose the Rowdy activation function that is designed to get rid of any saturation region by injecting sinusoidal fluctuations, which include trainable parameters. The proposed Rowdy activation function can be employed in any neural network architecture like feed-forward neural networks, Recurrent neural networks, Convolutional neural networks etc. The effectiveness of KNNs with Rowdy activation is demonstrated through various computational experiments including function approximation using feed-forward neural networks, solution inference of partial differential equations using the physics-informed neural networks, and standard deep learning benchmark problems using convolutional and fully-connected neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Deep convolutional neural networks for multi-scale time-series classification and application to tokamak disruption prediction using raw, high temporal resolution diagnostic data

In this paper we discuss recent advances in deep convolutional neural networks (CNN) for sequence learning, which allow identifying long-range, multi-scale phenomena in long sequences, such as those found in fusion plasmas. We point out several benefits of these deep CNN architectures, such as not requiring experts such as physicists to hand-craft input data features, the ability to capture longer range dependencies compared to the more common sequence neural networks (recurrent neural networks like long short-term memory (LSTM) networks), and the comparative computational efficiency. We apply this neural network architecture to the popular problem of disruption prediction in fusion energy tokamaks, utilizing raw data from a single diagnostic, the Electron Cyclotron Emission imaging (ECEi) diagnostic from the DIII-D tokamak. Initial results trained on a large ECEi dataset show promise, achieving an F 1 -score of ~91% on individual time-slices using only the ECEi data. This indicates the ECEi diagnostic by itself can be sensitive to a number of pre-disruption markers useful for predicting disruptions on timescales not only for mitigation but also avoidance. Future opportunities for utilizing these deep CNN architectures with fusion data are outlined, including impact of recent upgrades to the ECEi diagnostic.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

AI-Based EMT Dynamic Model of PV Systems

Several electromagnetic transient (EMT) dynamic modeling methods are available to model systems like photovoltaic (PV) plants, wind power plants, variable-speed drives, among others. The methods include: (a) physics-based models and (b) data-driven models. The physics-based dynamic models may include high-fidelity switched system model and average-value model that both require the control algorithms included in the models. However, manufacturers typically prefer to provide black-box models to avoid disclosing proprietary. One of the solutions to prevent disclosing control algorithms is the use of data-driven dynamic EMT models of PV systems. In this paper, data-driven dynamic EMT model based on artificial intelligence (AI) algorithms are presented. The AI algorithms evaluated include convolutional neural networks, recurrent neural networks, and nonlinear auto-regressive exogenous model. Automation in generating data and training these models is also discussed in this paper. The results generated by the best AI algorithms have been observed to be greater than 95 % accurate.

Debnath, Suman↗
DOE OSTI2025

Physics-Informed Recurrent Neural Networks to Predict Reactor Operations of the AGN-201 Nuclear Reactor

4 page paper submitted to ANS Student conference. Summary of paper similar to the following abstract: The ability to predict how a reactor will operate, understand when anomalous conditions arise, and ensure a reactor is being operated as expected is crucial for deploying new nuclear facilities. Digital twins serve as a unique solution to recognizing reactor behavior; however, they require data to be useful. For next-generation reactors, this data may not currently be available. To explore how synthetic physics-informed reactor data can be used to predict reactor operations, a recurrent neural network was implemented for the Idaho State University AGN-201 digital twin. The goal of this work is to determine how synthetic data can be used to train a recurrent neural network model for predicting the reactor power of the AGN-201. The recurrent neural network was validated using both synthetic and real operational data. We envision this approach will help bridge the gap between the virtual and physical sides of a digital twin, where reactor physics models based on as-built data can be corrected for actual operating parameters to ensure the virtual model mirrors reality.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗