Search NASA⌕ Search

SEARCH · Search NASA

Results for “Long Short Term Memory Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Early Information Parameter-Set Analysis for Satellite Close Approaches using Machine Learning

Understanding orbital mechanics is essential in space flight and navigation applications, and leveraging modern force models for flight path projection remains an important aspect in space mission design and operation. However, force models do not capture all the dynamics or perturbations in the space environment and thus are subject to errors in predicting the state vectors. The further out the predicted miss distance between spacecraft is from the time of closest approach (TCA), the larger the propagated errors in the predicted miss distance at TCA is. The dependency on these force models for spacecraft flight state prediction calls for a more reliable method that can quantify, or even reduce, these propagated errors. With recent advances in the field artificial intelligence, specifically in machine and deep learning algorithms, a model that implements these approaches can improve on the modern force model approach. The goal for this work is to provide an early-information decision-making threshold, in order to prioritize risk assessment implementation, given the ongoing increase of space objects. In analyzing the relationship of several parameters from conjunction data messages(CDMs) and solar information, early information becomes viable in miss distance prediction with unsupervised learning techniques, which learn the parameters that are linked together with miss distance and probability of collision (Pc) variables. Another approach implemented for identifying relationships within CDMs is supervised learning, in which a shallow neural network binary classifier learns to distinguish events with Pc values¡108. These parameters detected in the unsupervised process are then applied to a regression neural network, which predicts the miss distance at TCA for a specific event within a given uncertainty bound. For the regression neural network, a Long Short Term Memory (LSTM) neural network is implemented, which yields memory about each time step in an event. Using an LSTM network, the model learns to predict miss distance within 0.2km of the value measured at TCA. Although there is a limited amount of "close miss" data to train a network, the network learns to associate parameters, like large energy dissipation rates with the secondary object, with an elevated Pc

Brianna I. Robertson↗
DOE OSTI2024

Deep Multitask Learning Models for Radiation Estimation at High Energy Accelerator Facility

Controlling the dose of radiation exposure in potential radioactive facilities is critical for ensuring the safety of staff and the public. Here, in this paper, we developed machine learning models to estimate radiation exposure efficiently at the Thomas Jefferson National Accelerator Facility (JLab), aiming to enhance safety in both accelerator facilities and public areas. Multiple sensors were deployed around the three experimental halls at JLab. Data on single-beam currents, energy levels, and radiation values at the sensor locations were collected during accelerator operation. We proposed a multi-task learning model for radiation estimation, utilizing either one-dimensional convolutional neural networks (1-D CNNs) or long short-term memory networks (LSTMs) as the backbone. The proposed model was trained to simultaneously estimate radiation levels at the sensor locations. Experimental results demonstrated that the proposed model with LSTM backbone achieved the best estimation performance, with an average R 2 score of 0.7557 for estimation within the same year and 0.7157 for estimation across different years. These results significantly surpassed those of competing models.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Path sampling of recurrent neural networks by incorporating known physics

Recurrent neural networks have seen widespread use in modeling dynamical systems in varied domains such as weather prediction, text prediction and several others. Often one wishes to supplement the experimentally observed dynamics with prior knowledge or intuition about the system. While the recurrent nature of these networks allows them to model arbitrarily long memories in the time series used in training, it makes it harder to impose prior knowledge or intuition through generic constraints. In this work, we present a path sampling approach based on principle of Maximum Caliber that allows us to include generic thermodynamic or kinetic constraints into recurrent neural networks. We show the method here for a widely used type of recurrent neural network known as long short-term memory network in the context of supplementing time series collected from different application domains. These include classical Molecular Dynamics of a protein and Monte Carlo simulations of an open quantum system continuously losing photons to the environment and displaying Rabi oscillations. Our method can be easily generalized to other generative artificial intelligence models and to generic time series in different areas of physical and social sciences, where one wishes to supplement limited data with intuition or theory based corrections.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Exploring Randomly Wired Neural Networks for Climate Model Emulation

Exploring the climate impacts of various anthropogenic emissions scenarios is key to making informed decisions for climate change mitigation and adaptation. State-of-the-art Earth system models can provide detailed insight into these impacts but have a large associated computational cost on a per-scenario basis. This large computational burden has driven recent interest in developing cheap machine learning models for the task of climate model emulation. In this paper, we explore the efficacy of randomly wired neural networks for this task. We describe how they can be constructed and compare them with their standard feedforward counterparts using the ClimateBench dataset. Specifically, we replace the serially connected dense layers in multilayer perceptrons, convolutional neural networks, and convolutional long short-term memory networks with randomly wired dense layers and assess the impact on model performance for models with 1 million and 10 million parameters. We find that models with less-complex architectures see the greatest performance improvement with the addition of random wiring (up to 30.4% for multilayer perceptrons). Furthermore, of 24 different model architecture, parameter count, and prediction task combinations, only one had a statistically significant performance deficit in randomly wired networks relative to their standard counterparts, with 14 cases showing statistically significant improvement. We also find no significant difference in prediction speed between networks with standard feedforward dense layers and those with randomly wired layers. These findings indicate that randomly wired neural networks may be suitable direct replacements for traditional dense layers in many standard models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Data fusion in predicting internal heat gains for office buildings through a deep learning approach

Heating, Ventilation, and Air Conditioning (HVAC) is a major energy consumer in buildings. The predictive control has demonstrated a potential to reduce HVAC energy use. To facilitate predictive HVAC control, internal heat gains prediction is required. Here, we applied Long Short-Term Memory Networks, a special form of deep neural network, to predict miscellaneous electric loads, lighting loads, occupant counts and internal heat gains in two United States office buildings. Compared with the predetermined schedules used in American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) standard 90.1, the Long Short-Term Memory Networks method could reduce the prediction errors of internal heat gains from 12% to 8% in Building A, and from 26% to 16% in Building B. It was also found that for internal heat gains prediction, miscellaneous electric loads is a more important feature than occupant counts for two reasons. First, miscellaneous electric loads is the best proxy variable for internal heat gains, as it is the major component of and has the highest correlation coefficient with the internal heat gains. Second, miscellaneous electric loads contain valuable information to predict occupant count, while occupant count could not help improve miscellaneous electric loads prediction. Lastly, these findings could help researchers and practitioners select the most relevant features to more accurately predict internal heat gains for the implementation of predictive HVAC control in buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Normal or abnormal? Machine learning for the leakage detection in carbon sequestration projects using pressure field data

The international commitments for atmospheric carbon reduction will require a rapid increase in carbon capture and storage (CCS) projects. The key to any successful CCS project lies in the long term storage and prevention of leakage of stored carbon dioxide (CO 2 ). In addition to being a greenhouse gas, CO 2 leaks reaching the surface can accumulate in low-lying areas resulting in a serious health risk. Among several alternatives, some of the more promising CCS storage formations are depleted oil and gas reservoirs, where the reservoirs had good geological seals prior to hydrocarbon extraction. With more CCS wells coming online, it is imperative to implement permanent, automated monitoring tools. We apply machine learning models to automate the leakage detection process in carbon storage reservoirs using rates of (CO 2 ) injection and pressure data measured by simple harmonic pulse testing (HPT). To validate the feasibility of this machine learning based workflow, we use data from HPT experiments carried out in the Cranfield oil field, Mississippi, USA. The data consist of a series of pulse tests conducted with baseline parameters and with an artificially introduced leak. Here, in this study, we pose the leakage detection task as an anomaly detection problem where deviation from the predicted behavior indicates leaks in the reservoir. Results show that different machine learning architectures such as multi-layer feed forward network, Long Short-Term Memory, and convolutional neural network are able to identify leakages and can provide early warning. These warnings can then be used to take remedial measures.

58 GEOSCIENCES↗
DOE OSTI2024

Prediction of the SYM-H Index Using a Bayesian Deep Learning Method With Uncertainty Quantification

We propose a novel deep learning framework, named SYMHnet, which employs a graph neural network and a bidirectional long short-term memory network to cooperatively learn patterns from solar wind and interplanetary magnetic field parameters for short-term forecasts of the SYM-H index based on 1- and 5-min resolution data. SYMHnet takes, as input, the time series of the parameters' values provided by NASA's Space Science Data Coordinated Archive and predicts, as output, the SYM-H index value at time point t + w hours for a given time point t where w is 1 or 2. By incorporating Bayesian inference into the learning framework, SYMHnet can quantify both aleatoric (data) uncertainty and epistemic (model) uncertainty when predicting future SYM-H indices. Experimental results show that SYMHnet works well at quiet time and storm time, for both 1- and 5-min resolution data. The results also show that SYMHnet generally performs better than related machine learning methods. For example, SYMHnet achieves a forecast skill score (FSS) of 0.343 compared to the FSS of 0.074 of a recent gradient boosting machine (GBM) method when predicting SYM-H indices (1 hr in advance) in a large storm (SYM-H = -393 nT) using 5-min resolution data. When predicting the SYM-H indices (2 hr in advance) in the large storm, SYMHnet achieves an FSS of 0.553 compared to the FSS of 0.087 of the GBM method. In addition, SYMHnet can provide results for both data and model uncertainty quantification, whereas the related methods cannot.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Development of novel dynamic machine learning-based optimization of a coal-fired power plant

The increasing fraction of intermittent renewable energy in the electrical grid is resulting in coal-fired boilers now routinely ramp up and down. The current state-of-the-art operation for such boilers is to apply steady-state, neural network-based optimization to make control decisions in real-time, and this report demonstrates the feasibility of extending this to dynamic, neural network-based optimization using a long short-term memory neural network. A simplified numerical simulation of a t-fired coal boiler and supporting equipment is used to represent a real plant subjected to both steady-state, neural network-based optimization and dynamic, neural network-based optimization. Using the same intervals and a particle swarm optimization algorithm, the dynamic optimization outperforms the steady-state optimization and realizes up to 4.58% improvement in thermal efficiency. Dynamic optimization with a long short-term memory neural network is shown to both be feasible and beneficial for operation of a coal-fired boiler under changing load.

42 ENGINEERING↗
DOE OSTI2025

A Mass Conservation Relaxed (MCR) LSTM Model for Streamflow Simulation Across CONUS

The recent development of the physics-aware Mass-Conserving Long Short-Term Memory network (MC-LSTM) provides an alternative to other data-driven Deep Learning (DL) models in hydrology. Mass-Conserving Long Short-Term Memory incorporates mass conservation directly into the LSTM architecture. Despite the theoretical advancements, studies have reported a surprisingly limited performance of the MC-LSTM in streamflow simulation. We hypothesize that such a limitation is due to the unrealistic mass conservation scheme in MC-LSTM, which overlooks unobserved incoming water fluxes beyond precipitation. As an attempt to verify this hypothesis, we propose a Mass Conservation Relaxed LSTM (MCR-LSTM), which incorporates a bi-directional mass relaxation (MR) component to account for potential incoming water fluxes beyond precipitation. We train and test the proposed MCR-LSTM model across 531 watersheds in the contiguous United States (CONUS) against three baseline models: the Sacramento Soil Moisture Accounting, LSTM, and MC-LSTM. Our results show that MCR-LSTM outperforms MC-LSTM despite its underperformance compared to LSTM. Specifically, MCR-LSTM's advantage over MC-LSTM is mainly seen in the Plains and Western U.S., where the newly incorporated MR component better simulates water loss and suggests the likely existence of additional incoming water fluxes beyond precipitation, respectively. The novelty and contribution of this study are twofold: firstly, it introduces an alternative physics-aware DL tool (i.e., MCR-LSTM) in hydrology with higher accuracy in specific regions compared to MC-LSTM. Secondly, it provides a diagnosis of regions where strict, precipitation-based mass conservation constraints may be unrealistic in streamflow simulation.

deep learning↗
DOE OSTI2025

Estimating soybean yields from high-temporal-resolution multi-source data using deep learning

Accurate and timely crop yield prediction is crucial for ensuring food security and maintaining stable agricultural markets. In recent years, there has been a surge in interest in leveraging high-temporal-resolution, multi-source data for effective crop growth monitoring and yield estimation. A notable challenge arises from the difficulty in capturing the intricate interactions between variables across different time steps within these high-temporal-resolution time series datasets. This complexity hinders the reliable extraction of yield information from voluminous and often noisy datasets, especially during periods of extreme weather events. Here, in this study, we propose an Attention and Graph Isomorphism Network-enhanced Bi-directional Long Short-Term Memory network (AGB-LSTM) for estimating county-level soybean yield in the United States. This model integrates a diverse set of remote sensing data, including Near-Infrared Reflectance of Vegetation (NIRv), Sun-Induced chlorophyll Fluorescence (SIF), and Gross Primary Productivity (GPP), along with environmental covariates. The AGB-LSTM effectively leverages information related to crop yield from high-temporal-resolution time series data (5-days), achieving an accuracy of R²= 0.67 and rRMSE = 14.46%. This approach significantly outperforms traditional machine learning methods such as Random Forest (RF) (R²= 0.52, rRMSE = 17.36%) and Bi-LSTM (R²= 0.58, rRMSE = 16.17%). Sensitivity experiments with different time steps and ranges demonstrated that our model could accurately and stably predict yields 1 to 2 months before harvest. Moreover, data with a finer temporal resolution consistently improved prediction performance, resulting in an approximately 20% increase in and an approximately 20% decrease in rRMSE compared to using monthly composites. We also evaluated the robustness of the model under extreme climate events and observed strong performance (R²= 0.50, rRMSE = 21.32%). Finally, yield mapping for major soybean-producing regions in North America in 2023 revealed spatial patterns that closely matched USDA yield reports. Our findings suggest that the AGB-LSTM model is a promising and effective method for estimating yield and has notable potential for global crop yield forecasting.

Deep learning↗
DOE OSTI2019

A data-driven CO 2 leakage detection using seismic data and spatial–temporal densely connected convolutional neural networks

In carbon capture and sequestration (also known as carbon capture and storage, or CCS), developing effective monitoring methods is needed to detect and respond to CO 2 leakage. CO 2 leakage detection methods rely on geophysical observations and monitoring sensor network. However, traditional methods usually require the development of site-specific physical models and expert interpretation, and the effectiveness of these methods can be limited to different application locations, operational scenarios, and conditions. Here, we developed a novel data-driven leakage detection method based on densely connected convolutional neural networks. Our method is an end-to-end detection approach, that differs from conventional leakage monitoring methods by directly learning a mapping relationship between seismic data and the CO 2 leakage mass. To account for the spatial and temporal characteristics of seismic data, our novel networks architecture combines 1D and 2D convolutional neural networks. To overcome the computational expense of solving optimization problems, we apply a densely-connecting strategy in our network architecture that reduces the number of network parameters. Based on the features generated by our convolutional neural networks, we further incorporate a long short-term memory network to utilize time-sequential information, which further improves the detection accuracy. Lastly, we employ our detection method to synthetic seismic datasets generated based on flow simulations of a hypothetical CO 2 storage scenario with injection into a partially compartmentalized sandstone storage reservoir. To evaluate method performance, we conducted multiple experiments including a random leakage test, a sequential test, and a robustness test. Numerical results show that our CO 2 leakage detection method successfully detects the leakage and accurately predicts the leakage mass, suggesting that it has the potential for application in monitoring of real CO 2 storage sites.

58 GEOSCIENCES↗
DOE OSTI2020

Deep convolutional neural networks for multi-scale time-series classification and application to tokamak disruption prediction using raw, high temporal resolution diagnostic data

In this paper we discuss recent advances in deep convolutional neural networks (CNN) for sequence learning, which allow identifying long-range, multi-scale phenomena in long sequences, such as those found in fusion plasmas. We point out several benefits of these deep CNN architectures, such as not requiring experts such as physicists to hand-craft input data features, the ability to capture longer range dependencies compared to the more common sequence neural networks (recurrent neural networks like long short-term memory (LSTM) networks), and the comparative computational efficiency. We apply this neural network architecture to the popular problem of disruption prediction in fusion energy tokamaks, utilizing raw data from a single diagnostic, the Electron Cyclotron Emission imaging (ECEi) diagnostic from the DIII-D tokamak. Initial results trained on a large ECEi dataset show promise, achieving an F 1 -score of ~91% on individual time-slices using only the ECEi data. This indicates the ECEi diagnostic by itself can be sensitive to a number of pre-disruption markers useful for predicting disruptions on timescales not only for mitigation but also avoidance. Future opportunities for utilizing these deep CNN architectures with fusion data are outlined, including impact of recent upgrades to the ECEi diagnostic.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2022

Revisiting the Ground Magnetic Field Perturbations Challenge: A Machine Learning Perspective

Forecasting ground magnetic field perturbations has been a long-standing goal of the space weather community. The availability of ground magnetic field data and its potential to be used in geomagnetically induced current studies, such as risk assessment, have resulted in several forecasting efforts over the past few decades. One particular community effort was the Geospace Environment Modeling (GEM) challenge of ground magnetic field perturbations that evaluated the predictive capacity of several empirical and first principles models at both mid- and high-latitudes in order to choose an operative model. In this work, we use three different deep learning models-a feed-forward neural network, a long short-term memory recurrent network and a convolutional neural network-to forecast the horizontal component of the ground magnetic field rate of change (dB H /dt) over 6 different ground magnetometer stations and to compare as directly as possible with the original GEM challenge. We find that, in general, the models are able to perform at similar levels to those obtained in the original challenge, although the performance depends heavily on the particular storm being evaluated. We then discuss the limitations of such a comparison on the basis that the original challenge was not designed with machine learning algorithms in mind.

Victor A. Pinto↗
DOE OSTI2019

Physics-Guided Deep Learning for Time-Series State Estimation Against False Data Injection Attacks

The modern power grid is a cyber-physical system. While the grid is becoming more intelligent with emerging sensing and communication techniques, new vulnerabilities are introduced and cyber security becomes a major concern. One type of cyber-attacks - False Data Injection Attacks (FDIAs) - exploits the limitations in traditional power system state estimation, and modifies system states without being detected. In this paper, we propose a physics-guided deep learning (PGDL) approach to defend against FDIAs. The PGDL takes real-time measurements as inputs to neural networks, outputs the estimated states, and reconstructs measurements considering power system physics. A deep recurrent neural network - Long Short-Term Memory (LSTM) - is employed to learn the temporal correlations among states. This hybrid learning model leads to a time-series state estimation method to defend against FDIAs. The simulation results using IEEE 14-bus test system demonstrate the accuracy and robustness of the proposed time-series state estimation under FDIAs.

Cyber security, state estimation, physics-guided d↗
DOE OSTI2023

Identifying quenched jets in heavy ion collisions with machine learning

Measurements of jet substructure in ultra-relativistic heavy ion collisions suggest that the jet showering process is modified by the interaction with the quark-gluon plasma. Modifications of the hard substructure of jets can be explored with modern data-driven techniques. In this study, a machine learning approach to the identification of quenched jets is designed. Jet showering processes are simulated with a jet quenching model JEWEL and a non-quenching model PYTHIA 8. Sequential substructure variables are extracted from the jet clustering history following an angular-ordered sequence and are used in the training of a neural network built on top of a long short-term memory network. We show that this approach successfully identifies the quenching effect in the presence of the large uncorrelated background of soft particles created in heavy-ion collisions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Predicting plug loads with occupant count data through a deep learning approach

Predictive control has gained increasing attention for its ability to reduce energy consumption and improve occupant comfort in buildings. The plug loads prediction is a key component for the predictive building controls, as plug loads is a major source of internal heat gains in buildings. This study proposed a novel method to apply the Long-Short-Term-Memory (LSTM) Network, a special form of Recurrent Neural Network, to predict plug loads. The occupant count and the time have been confirmed to drive the plug load profile and thus selected as the features for the plug load prediction. The LSTM network was trained and tested with ground truth occupant count data collected from a real office building in Berkeley, California. Results from the LSTM network markedly improve the prediction accuracy compared with traditional linear regression methods and the classical Artificial Neural Network. 95% of 1-hour predictions from LSTM network are within ±1 kW of the actual plug loads, given the average plug loads during the office hour is 8.6 kW. The CV(RMSE) of the predicted plug load is 11% for the next hour, and 20% for the next eight hours. Lastly, we compared four prediction approaches with the office building we monitored: LSTM vs. ARIMA, with occupant counts vs. without occupant counts. It was found, the prediction error of the LSTM approach is around 4% less than the ARIMA approach. Using occupant counts as an exogenous input could further reduce the prediction error by 5%-6%. The findings of this paper could shed light on the plug load prediction for building control optimizations such as model-predictive control.

Occupant count↗
DOE OSTI2023

Deployment of Dynamic Neural Network Optimization to Minimize Heat Rate During Ramping for Coal Power Plants (Final Technical Report)

Much success was achieved throughout the course of this project. A successful implementation of Dynamic Neural Network Optimization (D-NNO) was coupled with Adaptive Predictive Controls (APC) and a novel hardware installation comprised of an advanced sensor network (ASN) measuring mass-weighted averages of flue gas constituents above the horizontal superheater of a coal-fired utility boiler. From 2019 through 2023 (including an extension due to COVID delays), the team was able to prototype, evaluate, deploy, iterate, and ultimately finalize an advanced closed-loop control D-NNO system which demonstrated the ability to: •improve unit efficiency ~2.0% relative to unoptimized operation (represented as total fuel fired per MWh generated) •improve unit NOx emission rates 10%+ beyond static optimization baselines •improve unit temperature stability as much as 58% and on average 12% •improve operating load stability as much as 35% The culmination of this project has generated an advanced methodology of deploying specially designed recurrent neural networks (long short-term memory, gated recurrent unit, encoder-decoder networks, transformers, etc.), customized trajectory planning and closed-loop optimization modules capable of adapting to live electric grid responses and demands, self-tuning and adaptive expert controls constantly adjusting prediction parameters to real-time unit behavior, and a hardware/software package able to reliably calculate net unit heat rate (NUHR) in real-time using flue gas constituents, machine learning, and known combustion relationships. Through this real-time NUHR value, immediate feedback on system adjustments relative to operating efficiency was available, allowing for rapid improvements to system performance. In addition to development and deployment of the advanced D-NNO system, the approach methodology has been readily commercialized through the project platform Griffin Open Systems, LLC, the D-NNO software platform host. Similar methodologies to those developed by this project have already been deployed at 5 other units across the United States, with another 6 implementations scheduled, and more expected. Over the course of the project, multiple academic papers were submitted and accepted for publication within esteemed academic journals, and PhD students were trained and graduated, as well as undergraduate students becoming involved and participating to project objectives.

01 COAL, LIGNITE, AND PEAT↗