Search NASA⌕ Search

SEARCH · Search NASA

Results for “Long Short Term Memory Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Subsurface Characterization and Machine Learning Predictions at Brady Hot Springs: Preprint

Subsurface data analysis, reservoir modeling, and machine learning (ML) techniques have been applied to the Brady Hot Springs (BHS) geothermal field in Nevada, USA to further characterize the subsurface and assist with optimizing reservoir management. Hundreds of reservoir simulations have been conducted in TETRAD-G and CMG STARS to explore different injection and production fluid flow rates and allocations and to develop a training data set for ML. This process included simulating the historical injection and production since 1979 and prediction of future performance through 2040. ML networks were created and trained using TensorFlow based on multilayer perceptron (MLP), long short-term memory (LSTM), and convolutional neural network (CNN) architectures. These networks took as input selected flow rates, injection temperatures, and historical field operation data and produced estimates of future production temperatures. This approach was first successfully tested on a simplified single fracture doublet system, followed by the application to the BHS reservoir. Using an initial BHS dataset with 37 simulated scenarios, the trained and validated network predicted the production temperature for 6 production wells with the mean absolute percentage error of less than 8%. In a complementary analysis effort, the principal component analysis applied to 13 BHS geological parameters revealed that vertical fracture permeability shows the strongest correlation with fault density and fault intersection density. A new BHS reservoir model was developed considering the fault intersection density as proxy for permeability. This new reservoir model helps to explore under-exploited zones in the reservoir. A data gathering plan to obtain additional subsurface data was developed; it includes temperature surveying for three idle injection wells, at which the reservoir simulations indicate high bottom-hole temperatures. The collected data assist with calibrating the reservoir model and may lead to converting these wells to producers to access under-exploited zones in the reservoir. Data gathering activities are planned for the first quarter of 2021.

40 EE - Geothermal Technologies Office (EE-4G)↗
DOE OSTI2021

StressNet - Deep learning to predict stress with fracture propagation in brittle materials

Abstract Catastrophic failure in brittle materials is often due to the rapid growth and coalescence of cracks aided by high internal stresses. Hence, accurate prediction of maximum internal stress is critical to predicting time to failure and improving the fracture resistance and reliability of materials. Existing high-fidelity methods, such as the Finite-Discrete Element Model (FDEM), are limited by their high computational cost. Therefore, to reduce computational cost while preserving accuracy, a deep learning model, StressNet, is proposed to predict the entire sequence of maximum internal stress based on fracture propagation and the initial stress data. More specifically, the Temporal Independent Convolutional Neural Network (TI-CNN) is designed to capture the spatial features of fractures like fracture path and spall regions, and the Bidirectional Long Short-term Memory (Bi-LSTM) Network is adapted to capture the temporal features. By fusing these features, the evolution in time of the maximum internal stress can be accurately predicted. Moreover, an adaptive loss function is designed by dynamically integrating the Mean Squared Error (MSE) and the Mean Absolute Percentage Error (MAPE), to reflect the fluctuations in maximum internal stress. After training, the proposed model is able to compute accurate multi-step predictions of maximum internal stress in approximately 20 seconds, as compared to the FDEM run time of 4 h, with an average MAPE of 2% relative to test data.

36 MATERIALS SCIENCE↗
DOE OSTI2025

sPHENIX heavy flavor jet tagging studies in p+p at $\sqrt{s_{NN}}=200~GeV$

Heavy-flavor jets, which are initiated from heavy quarks, are ideal probes for studying flavor dependent parton energy loss. We report on the performance of jet flavor tagging using two Neural Network Machine Learning (ML) models: the Long Short-Term Memory (LSTM) model and an Attention-based Neural Network, in simulations of 200 GeV p + p collisions. The tagging performance of bottom quark initiated jets with both ML models surpasses that of the traditional cut-based method. Technical details, including sample and kinematic variable selections, the machine learning training and testing setup with parameter tuning, and outcome comparisons, will be discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2018

Network Scale Travel Time Prediction using Deep Learning

In recent years, deep learning models have been receiving increased attention within the artificial intelligence (AI) community because of their high prediction accuracy. In this paper, two deep learning models, long short-term memory (LSTM) and convolutional neural network (CNN), are proposed to predict travel time in a road network. One major advantage of using deep learning for travel time prediction is that it can make accurate predictions for all the segments in the transportation network with a single model structure, instead of building customized models for each segment separately. The proposed models were evaluated on a transportation network in the City of Saint Louis, Missouri. The prediction results show that deep learning can provide accurate prediction for both congested and uncongested traffic conditions, and can successfully capture the traffic dynamics of unexpected incidents or special events. The study findings show that deep learning offers a promising approach to real-time prediction of travel times on a network scale.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2021

Short-term apartment-level load forecasting using a modified neural network with selected auto-regressive features

Residential electricity load profiles and their diversity have become increasingly important to realize the benefits of Smart or Transactive Energy Networks (TENs). An important element of TENs will be practical, accurate, and implementable residential load forecasting techniques. While there have been many approaches to short-term load forecasting, few have included forecasting for individual households, partly because the high volatility and idiosyncrasies present in individual household load data can pose significant challenges. In this study, we develop a Convolutional Long Short-Term Memory-based neural network with Selected Autoregressive Features (termed a CLSAF model) to improve short-term household electricity load forecasting accuracy by employing three strategies: autoregressive features selection, exogenous features selection, and a “default” state to avoid overfitting at times of high load volatility. We include aggregations of apartments to floor and building level, because utilities may favor transactive approaches that rely on aggregator models, e.g., a cluster of consumers as opposed to an individual. We demonstrate that the CLSAF model, by virtue of its enhanced feature representation and modest computational resources, can accomplish load forecasting in a multi-family residential building across three spatial granularities (individual apartment/household, floor, and building levels), with an accuracy improvement of up to 25% compared to a persistence model. We propose a data screening technique to characterize time-series electricity-load data. This technique is suitable for integration into a TEN ecosystem and allows one to estimate confidence levels of the load forecasts to optimize computational resources and the risks associated with uncertain forecasts.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A Comparative Study of Physics‐Informed and Data‐Driven Neural Networks for Compound Flood Simulation at River‐Ocean Interfaces: A Case Study of Hurricane Irene

Simulating compound flooding (CF) at the river-ocean interface within large-scale Earth System Models (ESMs) presents significant challenges due to complex interactions between river discharge, storm surge, and tides. This study assesses the comparative advantages of physics-informed and data-driven machine learning (ML) approaches for enhancing local ESM performance. We systematically compare data-driven neural network models (i.e., CNNs, U-Net, Long Short-Term Memory (LSTM), Gated Recurrent Unit), and physics-informed neural network (PINN) models, including vanilla PINN and a finite-difference-based PINN (FD-PINN). Specifically, FD-PINN is introduced to enhance computational efficiency, accelerating vanilla PINNs by ∼6.5 times while improving accuracy. To enhance data-driven model training, a new data-generation approach is developed to sample historical fluvial and coastal flood events, which ensures a robust data set for extreme event prediction. The models are evaluated using a realistic one-dimensional river domain extracted from an ESM's river mesh and the Hurricane Irene event as an independent test case. Results show that FD-PINN achieves accurate predictions with significantly reduced computational costs relative to vanilla PINNs. Among data-driven models, the best overall performance is achieved by a CNN-LSTM hybrid, which balances accuracy and efficiency. While a fully connected CNN (CNN-FC) provides the best accuracy, it incurs high computational cost. Architectures lacking strong temporal modeling tend to underperform on unseen events. These findings highlight the importance of sequence-aware designs for robust generalization. This study reveals the trade-offs between physics-informed and data-driven models and proposes an adaptive hybrid framework for integrating ML into ESMs to enhance local flood simulations.

Earth Systems Modeling↗
DOE OSTI2023

A novel transfer learning framework for sorghum biomass prediction using UAV-based remote sensing data and genetic markers

Yield for biofuel crops is measured in terms of biomass, so measurements throughout the growing season are crucial in breeding programs, yet traditionally time- and labor-consuming since they involve destructive sampling. Modern remote sensing platforms, such as unmanned aerial vehicles (UAVs), can carry multiple sensors and collect numerous phenotypic traits with efficient, non-invasive field surveys. However, modeling the complex relationships between the observed phenotypic traits and biomass remains a challenging task, as the ground reference data are very limited for each genotype in the breeding experiment. In this study, a Long Short-Term Memory (LSTM) based Recurrent Neural Network (RNN) model is proposed for sorghum biomass prediction. The architecture is designed to exploit the time series remote sensing and weather data, as well as static genotypic information. As a large number of features have been derived from the remote sensing data, feature importance analysis is conducted to identify and remove redundant features. A strategy to extract representative information from high-dimensional genetic markers is proposed. To enhance generalization and minimize the need for ground reference data, transfer learning strategies are proposed for selecting the most informative training samples from the target domain. Consequently, a pre-trained model can be refined with limited training samples. Field experiments were conducted over a sorghum breeding trial planted in multiple years with more than 600 testcross hybrids. The results show that the proposed LSTM-based RNN model can achieve high accuracies for single year prediction. Further, with the proposed transfer learning strategies, a pre-trained model can be refined with limited training samples from the target domain and predict biomass with an accuracy comparable to that from a trained-from-scratch model for both multiple experiments within a given year and across multiple years.

36 MATERIALS SCIENCE↗
DOE OSTI2022

In-Situ Sensing and Dynamics Predictions for Electrothermally-Actuated Soft Robot Limbs

Untethered soft robots that locomote using electrothermally-responsive materials like shape memory alloy (SMA) face challenging design constraints for sensing actuator states. At the same time, modeling of actuator behaviors faces steep challenges, even with available sensor data, due to complex electrical-thermal-mechanical interactions and hysteresis. This article proposes a framework for in-situ sensing and dynamics modeling of actuator states, particularly temperature of SMA wires, which is used to predict robot motions. A planar soft limb is developed, actuated by a pair of SMA coils, that includes compact and robust sensors for temperature and angular deflection. Data from these sensors are used to train a neural network-based on the long short-term memory (LSTM) architecture to model both unidirectional (single SMA) and bidirectional (both SMAs) motion. Predictions from the model demonstrate that data from the temperature sensor, combined with control inputs, allow for dynamics predictions over extraordinarily long open-loop timescales (10 min) with little drift. Prediction errors are on the order of the soft deflection sensor’s accuracy. This architecture allows for compact designs of electrothermally-actuated soft robots that include sensing sufficient for motion predictions, helping to bring these robots into practical application.

Sabelhaus, Andrew P.↗
DOE OSTI2026

Monitoring of Liquid Metal Reactor Heater Zones with Recurrent Neural Network Learning of Temperature Time Series

Advanced high-temperature fluid reactors (ARs), such as sodium fast reactors (SFRs) and molten salt cooled reactors (MSCRs) utilize high-temperature fluids at ambient pressure. To melt the fluid during reactor startup and prevent fluid freezing during cooldown, the thermal–hydraulic systems of such ARs include heater zones consisting of specific heaters with controllers, temperature sensors, and thermal insulation. The failure of heater zones due to insulation material degradation or improper installation, resulting in parasitic heat losses, can lead to fluid freezing. The detection of faults using a heat-transfer model is difficult because of a lack of knowledge of the experimental details. Data-driven machine learning of heater zone temperature time series offers a viable alternative. In this study, we benchmarked the performance of recurrent neural networks (RNNs) in an analysis of heat-up transient temperature time series of heater zones installed on a liquid sodium vessel. The RNN models include long short-term memory (LSTM) and gated recurrent unit (GRU) networks, as well as their bi-directional variants, BiLSTM and BiGRU. Anomalous temperature points were designated using a percentile-based threshold applied to residual fluctuations in the detrended temperature time series. Additionally, the impact of the exponentially weighted moving average (EWMA) method on detection accuracy was examined. The RNN models’ performance was assessed using precision, recall, and F 1 score metrics. Results demonstrated that RNN models effectively detect anomalies in temperature time series with the best models for each heater zone achieving F 1 scores of over 93%. To explain the variations in RNN model performance across different heater zones, we used Kullback–Leibler (KL) divergence to quantify the relative entropy between training and testing data, and the Detrended Fluctuation Analysis (DFA) to assess long-range temporal correlations. For datasets with strong long-range correlations and minimal relative entropy between training and testing data, GRU is the best-performing model. When the data exhibits weaker long-term correlations and a significant relative entropy between training and testing distributions, BiGRU shows the best performance. For the data sets with intermediate values of both KL divergence and DFA, the best performance is obtained with LSTM and BiLSTM, respectively.

gated recurrent unit↗
DOE OSTI2019

Learning Temporal and Spatial Correlations Jointly: A Unified Framework for Wind Speed Prediction

Leveraging both temporal and spatial correlations to predict wind speed remains one of the most challenging and less studied areas of wind speed prediction. In this paper, the problem of predicting wind speeds for multiple sites is investigated by using the spatio-temporal correlation. We proposed a deep architecture termed predictive spatio-temporal network (PSTN), which is a unified framework integrating a convolutional neural network (CNN) and a long short-term memory (LSTM). Initially, the spatial features are extracted from the spatial wind speed matrices (SWSMs) by the CNN at the bottom of the model. Then, the LSTM captures the temporal dependencies among the spatial features extracted from contiguous time points. Lastly, the predicted wind speeds are given by the last state of the top layer of the LSTM, which are generated by using the spatial features and temporal dependencies. Though composed of two kinds of architectures, PSTN is trained with one loss function in an end-to-end (E2E) manner, which can learn temporal and spatial correlations jointly. Experiments for short-term predictions are conducted on real-world data, whose results demonstrate that PSTN outperforms prior methods.

42 ENGINEERING↗
DOE OSTI2024

Machine learning models of intermittent operation of RO wellhead water treatment for salinity reduction and nitrate removal

Machine learning models were developed for intermittent multi-mode operation of a wellhead reverse osmosis water purification and desalination system to predict salt passage, nitrate passage, and permeate flux. The models, based on long short-term memory (LSTM) recurrent neural network (RNN) architecture, included an attention mechanism to increase model performance in proximity of the regulatory limit for nitrate. Training and testing of the models for the Startup, Production, Shutdown and Flushing operational modes were based on operational data (consisting of 22 process variables per data sample) acquired every 2–5 s over a six-month period. The significant sets of model input attributes for the different operational modes were assessed via Spearman ranking correlation, Self-Organizing Map (SOM) analysis and feed forward feature selection (FFFS). Although the variability of nitrate passage, salt passage and permeate flux was significant over the four operational modes, prediction performance for the three outcomes were with R2 and Average Absolute Relative Error (AARE) of 0.78–0.95 and 2.96–6.16 %, respectively. Model updates post membrane elements replacement demonstrated similar levels of prediction accuracy. The study results suggest that there is merit in exploring the utility of multi-mode models for sensor fault detection, data imputation, and for potential use in model-predictive control.

Intermittent RO operation↗
DOE OSTI2023

Explainable machine learning model for multi-step forecasting of reservoir inflow with uncertainty quantification

We propose an explainable machine learning (ML) model with uncertainty quantification (UQ) to improve multi-step reservoir inflow forecasting. Traditional ML methods have challenges in forecasting inflows multiple days ahead, and lack explainability and UQ. To address these limitations, we introduce an encoder–decoder long short-term memory (ED-LSTM) network for multi-step forecasting, employ the SHapley Additive exPlanation (SHAP) technique for understanding the influence of hydrometeorological factors on inflow prediction, and develop a novel UQ method for prediction trustworthiness. We apply these methods to forecast 7-day inflow in snow-dominant and rain-driven reservoirs. The results demonstrate the effectiveness of the ED-LSTM model, with high forecasting accuracy for short lead times. Our UQ method provides reliable uncertainty estimates, covering 90% of data with a 90% confidence level. The SHAP analysis reveals the importance of historical inflow and precipitation as influential factors. These findings and methods may support reservoir operators in optimizing water resources management decisions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Accelerating cavity fault prediction using deep learning at Jefferson Laboratory

Abstract Accelerating cavities are an integral part of the continuous electron beam accelerator facility (CEBAF) at Jefferson Laboratory. When any of the over 400 cavities in CEBAF experiences a fault, it disrupts beam delivery to experimental user halls. In this study, we propose the use of a deep learning model to predict slowly developing cavity faults. By utilizing pre-fault signals, we train a long short-term memory-convolutional neural network binary classifier to distinguish between radio-frequency (RF) signals during normal operation and RF signals indicative of impending faults. We optimize the model by adjusting the fault confidence threshold and implementing a multiple consecutive window criterion to identify fault events, ensuring a low false positive rate. Results obtained from analysis of a real dataset collected from the accelerating cavities simulating a deployed scenario demonstrate the model’s ability to identify normal signals with 99.99% accuracy and correctly predict 80% of slowly developing faults. Notably, these achievements were achieved in the context of a highly imbalanced dataset, and fault predictions were made several hundred milliseconds before the onset of the fault. Anticipating faults enables preemptive measures to improve operational efficiency by preventing or mitigating their occurrence.

43 PARTICLE ACCELERATORS↗
DOE OSTI2019

Nonintrusive reduced order modeling framework for quasigeostrophic turbulence

In this study, we present a nonintrusive reduced order modeling (ROM) framework for large-scale quasistationary systems. The framework proposed in this work exploits the time series prediction capability of long short-term memory (LSTM) recurrent neural network architecture such that (1) in the training phase, the LSTM model is trained on the modal coefficients extracted from the high-resolution data snapshots using proper orthogonal decomposition (POD) transform, and (2) in the testing phase, the trained model predicts the modal coefficients for the total time recursively based on the initial time history. Hence, no prior information about the underlying governing equations is required to generate the ROM. To illustrate the predictive performance of the proposed framework, the mean flow fields and time series response of the field values are reconstructed from the predicted modal coefficients by using an inverse POD transform. As a representative benchmark test case, we consider a two-dimensional quasigeostrophic ocean circulation model which, in general, displays an enormous range of fluctuating spatial and temporal scales. We first demonstrate that the conventional Galerkin projection-based reduced order modeling of such systems requires a high number of POD modes to obtain a stable flow physics. In addition, ROM-Galerkin projection (ROM-GP) does not seem to capture the intermittent bursts appearing in the dynamics of the first few most energetic modes. However, the proposed nonintrusive ROM framework based on LSTM (ROM-LSTM) yields a stable solution even for a small number of POD modes. We also observe that the ROM-LSTM model is able to capture quasiperiodic intermittent bursts accurately, and yields a stable and accurate mean flow dynamics using the time history of a few previous time states, denoted as the lookback time window in this paper. We show several features of ROM-LSTM framework such as significantly higher accuracy than ROM-GP, and faster performance using larger time step size. Throughout the paper, we demonstrate our findings in terms of time series evolution of the field values and mean flow patterns, which suggest that the proposed fully nonintrusive ROM framework is robust and capable of predicting chaotic nonlinear fluid flows in an extremely efficient way compared to the conventional projection-based ROM framework.

42 ENGINEERING↗
DOE OSTI2023

Uncertainty Aware Deep Learning for Fault Prediction Using Multivariate Time Series Signals

The superconducting radio-frequency cavities are a crucial component of the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. When a cavity faults, beam delivery to experimental end users is disrupted. Prediction of cavity faults prior to onset is essential to reduce operation and maintenance costs. In this work, a parallel long short-term memory (LSTM)-convolution neural network (CNN)-based deep learning (DL) model is proposed to predict impending faults using pre-fault signals. Further, we introduce an uncertainty quantification approach using Monte Carlo dropout with the LSTM-CNN model to ascertain confidence in the prediction. The model was tested using multivariate time series signals from stable cavity operations and before faults. Initial results show that on the test dataset, the model can identify impending faults before their onset with an average 10-fold cross validation accuracy of 97.39% and a standard deviation of 0.12% using a 100-ms time window. It is also observed that the model performs better as the prediction time moves closer to the fault onset. For additional context, we compare the performance of the model with three machine-learning-based (ML) fault prediction models. Our proposed parallel LSTM-CNN-based DL method shows better performance than the ML-based methods.

Rahman, Md Monibor↗
DOE OSTI2024

Utilizing Physics-Informed Synthetic Data to Train a Digital Twin for Predicting Reactor Operations

Understanding techniques to strengthen the nuclear nonproliferation regime is crucial in reducing the creation of nuclear weaponry on the basis of advancements in the nuclear energy industry. Prior to construction of a nuclear power plant, it is necessary to understand the proliferation potential of the plant’s reactor. Digital twins serve as a unique solution to recognizing reactor behavior indicative of nuclear proliferation. The following research conducted serves as a validation to training a digital twin on synthetic data fabricated via means of reactor physics simulations based on parameters of Idaho State University’s AGN-201 reactor. The synthetic data is utilized to train a long short-term memory (LSTM) recurrent neural network model. The accuracy of the predicted data is measured against real operational data to verify the reliability of the synthetic data creation methods and if these methods should be used in the future to inform inspectors of a reactor’s proliferation capabilities.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2022

Tracing and Forecasting Metabolic Indices of Cancer Patients Using Patient-Specific Deep Learning Models

We develop a patient-specific dynamical system model from the time series data of the cancer patient’s metabolic panel taken during the period of cancer treatment and recovery. The model consists of a pair of stacked long short-term memory (LSTM) recurrent neural networks and a fully connected neural network in each unit. It is intended to be used by physicians to trace back and look forward at the patient’s metabolic indices, to identify potential adverse events, and to make short-term predictions. When the model is used in making short-term predictions, the relative error in every index is less than 10% in the L ∞ norm and less than 6.3% in the L 1 norm in the validation process. Once a master model is built, the patient-specific model can be calibrated through transfer learning. As an example, we obtain patient-specific models for four more cancer patients through transfer learning, which all exhibit reduced training time and a comparable level of accuracy. This study demonstrates that this modeling approach is reliable and can deliver clinically acceptable physiological models for tracking and forecasting patients’ metabolic indices.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Digital Twin Empowered PV Power Prediction

The accurate prediction of photovoltaic (PV) power generation is significant to ensure the economic and safe operation of power systems. To this end, the paper proposes a new digital twin (DT) empowered PV power prediction framework that is capable of ensuring reliable data transmission and employing the DT to achieve high accuracy of power prediction. With this framework, considering potential data contamination in the collected PV data, a generative adversarial network is employed to restore the historical data set, which offers a prerequisite to ensure accurate mapping from the physical space to the digital space. Further, a new DT empowered PV power prediction method is proposed. Therein, we model a DT that encompasses a digital physical model for reflecting the physical operation mechanism and a neural network model (i.e., a parallel network of convolution and bidirectional long-short-term memory model) for capturing the hidden spatial-temporal features. The proposed method enables the use of the DT to take advantages of the digital physical model and the neural network model, resulting in enhanced prediction accuracy. Finally, a real data set is conducted to access the effectiveness of the proposed method.

14 SOLAR ENERGY↗