Search NASA⌕ Search

SEARCH · Search NASA

Results for “LSTM modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Predicting Short-Term Deformation in the Central Valley Using Machine Learning

Land subsidence caused by excessive groundwater pumping in Central Valley, California, is a major issue that has several negative impacts such as reduced aquifer storage and damaged infrastructures which, in turn, produce an economic loss due to the high reliance on crop production. This is why it is of utmost importance to routinely monitor and assess the surface deformation occurring. Two main goals that this paper attempts to accomplish are deformation characterization and deformation prediction. The first goal is realized through the use of Principal Component Analysis (PCA) applied to a series of Interferomtric Synthetic Aperture Radar (InSAR) images that produces eigenimages displaying the key characteristics of the subsidence. Water storage changes are also directly analyzed by the use of data from the Gravity Recovery and Climate Experiment (GRACE) twin satellites and the Global Land Data Assimilation System (GLDAS). The second goal is accomplished by building a Long Short-Term Memory (LSTM) model to predict short-term deformation after developing an InSAR time series using LiCSBAS, an open-source InSAR time series package. The model is applied to the city of Madera and produces better results than a baseline averaging model and a one dimensional convolutional neural network (CNN) based on a mean squared error metric showing the effectiveness of machine learning in deformation prediction as well as the potential for incorporation in hazard mitigation models. The model results can directly aid policy makers in determining the appropriate rate of groundwater withdrawal while maintaining the safety and well-being of the population as well as the aquifers’ integrity.

58 GEOSCIENCES↗
DOE OSTI2025

Structure‐Aware Representation Learning for Effective Performance Prediction

ABSTRACT Application performance is a function of several unknowns stemming from the interactions between the application, runtime, OS, and underlying hardware, making it challenging to model performance using deep learning techniques, especially without a large labeled dataset. Collecting such labeled longitudinal datasets can take weeks. Intuitively, developers could save analysis time during code development by taking a comparative approach between multiple applications. However, the unknown dynamic interactions between applications and execution environments make it difficult for deep learning‐based models to predict the performance of new applications. In this paper, we address these problems by presenting a labeled dataset for the community and taking a comparative analysis approach to explore the source code differences between different correct implementations of the same problem. This paper assesses the feasibility of using purely static information, for example, Abstract Syntax Tree (AST), of applications to predict performance change based on code structure. We evaluate several deep learning‐based representation learning techniques for source code and propose an architecture for the tree‐based Long Short‐Term Memory (LSTM) models to discover latent representations for a source code's hierarchical structure. We demonstrate that our proposed architecture enables feed‐forward predictive models to predict change in performance using source code with up to 84% accuracy.

Ramadan, Tarek [Department of Computer Science Tex↗
NASA NTRSDate not supplied

Fast Machine Learning Lidar Surrogate Simulator: Pristine Clear Sky

The simulations of lidar signals and retrievals rely on a range of optic-physical models, such as radiative transfer models, particle scattering and absorption models, along with the output data from atmospheric physical models. Integrating these different models to represent signals of a lidar system is computationally expensive, and performing backward retrievals can be complex and ambiguous. However, with the advantages of Machine Learning, there is a new potential for building effective lidar signal database linked to corresponding atmospheric profiles. For this project, we are developing a fast pre-trained neural network as the lidar surrogate simulator using simulated data for a CALIPSO-like lidar (355 nm, 532nm, and 1064nm), and a CO2 differential absorption lidar (DIAL) near 1571nm. Specifically, we utilize a long short-term memory (LSTM) model to map the relationships between atmospheric profiles (pressure, temperature, air density and CO2 mixing ratio) and lidar signals. This approach allows us to build machine learning based simulators that can reconstruct lidar signals at specific bands from MERRA reanalysis data, and perform retrievals of atmospheric profiles using lidar signals at various wavelengths. As a first step, the results show the potential of this method to establish a foundational model for sensor signals. This model offers the promise of enabling both accurate predictions and rapid retrievals, providing a more efficient approach to signal processing and analysis.

Shan Zeng↗
DOE OSTI2024

Power generation forecasting for solar plants based on Dynamic Bayesian networks by fusing multi-source information

A Dynamic Bayesian network (DBN) model for solar power generation forecasting in solar plants is proposed in this paper. The key idea is to fuse sensor data, operational indicators, meteorological data, lagged output power information, and model errors for more accurate short-term (e.g., hours) and mid-term (e.g., days to weeks) power generation forecasting. The proposed DBN augments automated data-driven structure learning with expert knowledge encoding using continuous and categorical data given constraints to represent causal relationships within a solar inverter system. Additionally, an error compensation mechanism is proposed to capture temporal fluctuation. The effectiveness of the DBN on solar power generation forecasting was evaluated by rolling window analysis with one-year testing data collected from a local solar plant. The proposed DBN is compared with four state-of-art methods including support-vector regression (SVR), k-nearest neighbors (kNN), artificial neural network (ANN), and long short-term memory (LSTM) models. The result show that the proposed DBN achieves better accuracy in general, and it is not as data-hungry as some neural network-based models. The proposed DBN is also shown to have robust and consistent forecasting power with different forecasting horizons. The accuracy is 92% - 95% from one hour to one week ahead forecasting.

14 SOLAR ENERGY↗
DOE OSTI2025

Investigating lab-scaled offshore wind aerodynamic testing failure and developing solutions for early anomaly detections

As offshore wind systems become more complex, the risk of human error or equipment malfunction increases during experimental testing. This study investigates a lab-scale incident involving a 1 : 50 scale 5 MW wind turbine, where a generator failure led to rotor overspeed and a blade–tower strike. To improve early fault detection, we propose a data-driven method based on multivariate long short-term memory (LSTM) models. High-frequency measurements are projected onto principal components, and anomalies are identified using reconstruction error and its time derivative. Two models are trained on different healthy datasets and tested using single- and multi-principal component (1PC and MPC) variations. Results show that combining both error and error derivative improves detection accuracy. The 1PC model detects faults faster, has a higher recall rate, and achieves a 43 % improvement in anomaly detection accuracy, while the MPC model yields higher precision. This approach provides a simple and effective tool for early anomaly detection in lab-scale experiments, helping to reduce the risk of future failures during the testing of new technologies.

17 WIND ENERGY↗
DOE OSTI2024

Crustal permeability generated through microearthquakes is constrained by seismic moment

We link changes in crustal permeability to informative features of microearthquakes (MEQs) using two field hydraulic stimulation experiments where both MEQs and permeability evolution are recorded simultaneously. The Bidirectional Long Short-Term Memory (Bi-LSTM) model effectively predicts permeability evolution and ultimate permeability increase. Our findings confirm the form of key features linking the MEQs to permeability, offering mechanistically consistent interpretations of this association. Transfer learning correctly predicts permeability evolution of one experiment from a model trained on an alternate dataset and locale, which further reinforces the innate interdependency of permeability-to-seismicity. Models representing permeability evolution on reactivated fractures in both shear and tension suggest scaling relationships in which changes in permeability (Δk) are linearly related to the seismic moment (M) of individual MEQs as Δk ∝ M. This scaling relation rationalizes our observation of the permeability-to-seismicity linkage, contributes to its predictive robustness and accentuates its potential in characterizing crustal permeability evolution using MEQs.

15 GEOTHERMAL ENERGY↗
DOE OSTI2021

Sentiment Analysis based Error Detection for Large-Scale Systems

Today's large-scale systems such as High Performance Computing (HPC) Systems are designed/utilized towards exascale computing, inevitably decreasing its reliability due to the increasing design complexity. HPC systems conduct extensive logging of their execution behaviour. In this paper, we leverage the inherent meaning behind the log messages and propose a novel sentiment analysis-based approach for the error detection in large-scale systems, by automatically mining the sentiments in the log messages. Our contributions are four-fold. (1) We develop a machine learning (ML) based approach to automatically build a sentiment lexicon, based on the system log message templates. (2) Using the sentiment lexicon, we develop an algorithm to detect system errors. (3) We develop an algorithm to identify the nodes and components with erroneous behaviors, based on sentiment polarity scores. (4) We evaluate our solution vs. other state-of-the-art machine/deep learning algorithms based on three representative supercomputers' system logs. Experiments show that our error detection algorithm can identify error messages with an average MCC score and f -score of 91% and 96% respectively, while state of the art ML/deep learning model (LSTM) obtains only 67% and 84%. To the best of our knowledge, this is the first work leveraging the sentiments embedded in log entries of large-scale systems for system health analysis.

error detection↗
DOE OSTI2022

Identifying Hydrometeorological Factors Influencing Reservoir Releases Using Machine Learning Methods

Simulation of reservoir releases plays a critical role in social-economic functioning and our nation's security. How-ever, it is challenging to predict the reservoir release accurately because of many influential factors from natural environments and engineering controls such as the reservoir inflow and storage. Moreover, climate change and hydrological intensification causing the extreme precipitation and temperature make the accurate prediction of reservoir releases even more challenging. Machine learning (ML) methods have shown some successful applications in simulating reservoir releases. However, previous studies mainly used inflow and storage data as inputs and only considered their short-term influences (e.g, previous one or two days). In this work, we use long short-term memory (LSTM) networks for reservoir release prediction based on four input variables including inflow, storage, precipitation, and temperature and consider their long-term influences. We apply the LSTM model to 30 reservoirs in Upper Colorado River Basin, United States. We analyze the prediction performance using six statistical metrics. More importantly, we investigate the influence of the input hydrometeorological factors, as well as their temporal effects on reservoir release decisions. Results indicate that inflow and storage are the most influential factors but the inclusion of precipitation and temperature can further improve the prediction of release especially in low flows. Additionally, the inflow and storage have a relatively long-term effect on the release. These findings can help optimize the water resources management in the reservoirs.

Fan, Ming↗
DOE OSTI2021

Real-Time and Adaptive Reservoir Computing With Application to Profile Prediction in Fusion Plasma

Nuclear fusion is a promising alternative to address the problem of sustainable energy production. The tokamak is an approach to fusion based on magnetic plasma confinement, constituting a complex physical system with many control challenges. Here we study the characteristics and optimization of reservoir computing (RC) for real-time and adaptive prediction of plasma profiles in the DIII-D tokamak. Our experiments demonstrate that RC achieves comparable results to state-of-the-art (deep) convolutional neural networks (CNNs) and long short-term memory (LSTM) models, with a significantly easier and faster training procedure. This efficient approach allows for fast and frequent adaptation of the model to new situations, such as changing plasma conditions or different fusion devices.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Learning the Temporal Effect in Infrared Thermal Videos With Long Short-Term Memory for Quality Prediction in Resistance Spot Welding

With the advances of sensing technology, in-situ infrared thermal videos can be collected from Resistance Spot Welding (RSW) processes. Each video records the formulation process of a weld nugget. The nugget evolution creates a “temporal effect” across the frames, which can be leveraged for real-time, nondestructive evaluation (NDE) of the weld quality. Currently, quality prediction with imaging data mainly focuses on optical feature extraction with Convolutional Neural Network (CNN) but does not make the most of such temporal effect. In this study, pixels corresponding to critical locations on the weld nugget surface are extracted from a video to form multivariate time series (MTS). Multivariate Adaptive Regression Splines (MARS) is used in MTS processing to remove noisy signals related to uninformative frames. A Stacked Long Short-Term Memory (LSTM) model is developed to learn from the processed MTS and then predicts weld nugget size and thickness in real-time NDE. Results from a case study on RSW of Boron steel demonstrates the improvement in prediction accuracy and computational time with the proposed method, as compared to CNN-based weld quality prediction.

Guo, Shenghan↗
DOE OSTI2020

Near-Real-Time Forecast of Satellite-Based Soil Moisture Using Long Short-Term Memory with an Adaptive Data Integration Kernel

Nowcasts, or near-real-time (NRT) forecasts, of soil moisture based on the Soil Moisture Active and Passive (SMAP) mission could provide substantial value for a range of applications including hazards monitoring and agricultural planning. To provide such a NRT forecast with high fidelity, we enhanced a time series deep learning architecture, long short-term memory (LSTM), with a novel data integration (DI) kernel to assimilate the most recent SMAP observations as soon as they become available. The kernel is adaptive in that it can accommodate irregular observational schedules. Testing over the CONUS, this NRT forecast product showcases predictions with unprecedented accuracy when evaluated against subsequent SMAP retrievals. It showed smaller error than NRT forecasts reported in the literature, especially at longer forecast latency. The comparative advantage was due to LSTM’s structural improvements, as well as its ability to utilize more input variables and more training data. The DI-LSTM was compared to the original LSTM model that runs without data integration, referred to as the projection model here. We found that the DI procedure removed the autocorrelated effects of forcing errors and errors due to processes not represented in the inputs, for example, irrigation and floodplain/lake inundation, as well as mismatches due to unseen forcing conditions. The effects of this purely data-driven DI kernel are discussed for the first time in the geosciences. Furthermore, this work presents an upper-bound estimate for the random component of the SMAP retrieval error.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Data-Driven Day-Ahead PV Estimation Using Autoencoder-LSTM and Persistence Model

Inherent variability in photovoltaic (PV) and associated impacts on power systems is a challenging problem for both the PV owners and the grid operators. Existing statistical and machine learning algorithms typically work well for weather conditions similar to historical data. Furthermore, uncertain weather conditions pose a great challenge to the estimation accuracy of the estimation models. With the enhanced integration of intelligent electronic devices and the realization of associated automation in the power grid, renewable energy data is becoming more accessible, which can be utilized by deep learning models and improve the PV power generation estimation accuracy. In this paper, a hybrid deep learning model driven by external weather data is proposed to do day-ahead PV output forecasting at 15-minute-interval. The proposed model is motivated by the recent advancement of Long-Short-Term-Memory (LSTM) networks and AutoEncoder (AE), which estimates uncertainties in sequence while making the prediction for complex weather conditions. Meanwhile, the persistence model (PM) is used to predict continuous sunny weather conditions. The forecasting result is validated with data from multiple locations

42 ENGINEERING↗
DOE OSTI2022

Use of Physics to Improve Solar Forecast: Part II, Machine Learning and Model Interpretability

Machine learning (ML) models have been applied to forecast solar energy; however, they often lack clarity of interpretability and underlying physics. This work addresses such challenges by developing a hierarchy of ML models that gradually introduce predictors to improve the forecast accuracy based on a physics-based framework. Three ML models (ARIMA, LSTM, and XGBoost) are examined and compared with four physics-informed persistence models reported in Part I and the simple persistence model to assess the improvement of different models. The 7-year measurements at the U.S. Department of Energy's Atmospheric Radiation Measurement's Southern Great Plains Central Facility site are used for forecasts and evaluations. The results reveal that the step-by-step introduction of predictors leads to different improvements for models at different hierarchical levels. Comparison of the ML models with persistence models shows that LSTM and XGBoost outperform all the persistence models, with LSTM having the overall best performance; however, ARIMA underperforms the four physics-informed persistence models. This study demonstrates the importance and utility of incorporating physics into ML models in improving forecast accuracy by introducing a hierarchy of physics-based predictors, distinguishing predictor contributions, and enhancing the ML interpretability. The combined use of Global Horizontal Irradiance (GHI) and Direct Normal Irradiance (DNI) significantly improves the forecast accuracy compared to using individual irradiances alone because the pair contains more information on cloud-radiation interactions.

interpretability↗
DOE OSTI2022

Trajectory prediction dimensionality reduction for low-cost connected automated vehicle systems

Here, to facilitate low-cost connected automated vehicle (CAV) system development, this study proposes two interpretable dimensionality reduction techniques in vehicle trajectory prediction, i.e., the piecewise Taylor series approximation (PTA) and the piecewise Fourier series approximation (PFA), to lower computation complexity, reduce device investment, and decrease computation energy consumption. Two benchmarks are developed, the long short-term memory (LSTM)-based model without dimensionality reduction and the LSTM-based model with encoder-decoder (a widely used dimensionality reduction technique). Results show that the four predictions have similar accuracy, and the training time (proportional to computation energy consumption) of models with dimensionality reduction techniques is greatly reduced. The reduction is even more significant when PTA/PFA is used. Sensitivity analysis advises PFA/PTA parameter selections to reduce computation complexity without significant loss of prediction accuracy. Further, the robustness of the LSTM PTA/PFA is proven by the investigation of data noises.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Enhancing Streamflow Forecast and Extracting Insights Using Long-Short Term Memory Networks With Data Integration at Continental Scales

Recent observations with varied schedules and types (moving average, snapshot, or regularly spaced) can help to improve streamflow forecasts, but it is challenging to integrate them effectively. Based on a long short-term memory (LSTM) streamflow model, we tested multiple versions of a flexible procedure we call data integration (DI) to leverage recent discharge measurements to improve forecasts. DI accepts lagged inputs either directly or through a convolutional neural network unit. DI ubiquitously elevated streamflow forecast performance to unseen levels, reaching a record continental-scale median Nash-Sutcliffe Efficiency coefficient value of 0.86. Integrating moving-average discharge, discharge from the last few days, or even average discharge from the previous calendar month could all improve daily forecasts. Directly using lagged observations as inputs was comparable in performance to using the convolutional neural network unit. Importantly, we obtained valuable insights regarding hydrologic processes impacting LSTM and DI performance. Before applying DI, the base LSTM model worked well in mountainous or snow-dominated regions, but less well in regions with low discharge volumes (due to either low precipitation or high precipitation-energy synchronicity) and large interannual storage variability. DI was most beneficial in regions with high flow autocorrelation: it greatly reduced baseflow bias in groundwater-dominated western basins and also improved peak prediction for basins with dynamical surface water storage, such as the Prairie Potholes or Great Lakes regions. However, even DI cannot elevate performance in high-aridity basins with 1-day flash peaks. Despite this limitation, there is much promise for a deep-learning-based forecast paradigm due to its performance, automation, efficiency, and flexibility.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Machine learning modeling and model predictive control of a closed-circuit reverse osmosis system

Closed-circuit reverse osmosis (CCRO) offers a flexible and energy-efficient alternative to conventional reverse osmosis by operating in a semi-batch mode that recycles brine, enabling higher recovery rates and reduced specific energy consumption (SEC). However, developing accurate, system-level dynamic models for CCRO remains challenging due to its nonlinear, multi-phase operation and sensitivity to variable feed water conditions. Traditional modeling approaches, such as NARMAX (nonlinear autoregressive moving average with exogenous inputs), often struggle to generalize across varying inlet feed concentrations, necessitating frequent parameter re-estimation and limiting their utility for real-time control applications. To address these limitations, we developed a long short-term memory (LSTM) neural network model trained on an extensive experimental data set from a CCRO pilot plant. The model accepts three inputs, feed flow rate, recirculation flow rate, and initial feed conductivity, and predicts three key outputs: reject conductivity, feed pump power draw, and recirculation pump power draw. We validated the LSTM model against experimental data, demonstrating its ability to distinguish between different feed conductivities and adapt to variable flow rates. Subsequently, we incorporated the LSTM model within a nonlinear model predictive control (MPC) scheme and conducted closed-loop simulations to optimize the integrated SEC (iSEC). In conclusion, the results project up to a 6% reduction in iSEC by using MPC to optimize performance over the entire experiment duration, without requiring any random excitation for data collection or parameter re-estimation.

Desalination↗
DOE OSTI2024

DeepPhenoMem V1.0: deep learning modelling of canopy greenness dynamics accounting for multi-variate meteorological memory effects on vegetation phenology

Abstract. Vegetation phenology plays a key role in controlling the seasonality of ecosystem processes that modulate carbon, water and energy fluxes between the biosphere and atmosphere. Accurate modelling of vegetation phenology in the interplay of Earth's surface and the atmosphere is thus crucial to understand how the coupled system will respond to and shape climatic changes. Phenology is controlled by meteorological conditions at different timescales: on the one hand, changes in key meteorological variables (temperature, water, radiation) can have immediate effects on the vegetation development; on the other hand, phenological changes can be driven by past environmental conditions, known as memory effects. However, the processes governing meteorological memory effects on phenology are not completely understood, resulting in their limited performance of vegetation phenology represented in land surface models. A deep learning model, specifically a long short-term memory network (LSTM), has the potential to capture and model the meteorological memory effects on vegetation phenology. Here, we apply the LSTM to model the vegetation phenology using meteorological drivers and high-temporal-resolution canopy greenness observations through digital repeat photography by the PhenoCam network. We compare a multiple linear regression model, a no-memory-effect LSTM model and a full-memory-effect LSTM model to predict the whole seasonal greenness trajectory and the corresponding phenological transition dates across 50 sites and 317 site years during 2009–2018, covering deciduous broadleaf forests, evergreen needleleaf forests and grasslands. Results show that the deep learning model outperforms the multiple linear regression model, and the full-memory-effect LSTM model performs better than the no-memory-effect model for all three plant function types (median R2 of 0.878, 0.957 and 0.955 for broadleaf forests, evergreen needleleaf forests and grasslands). We also find that the full-memory-effect LSTM model is capable of predicting the seasonal dynamic variations of canopy greenness and reproducing trends in shifting phenological transition dates. We also performed a sensitivity analysis of the full-memory-effect LSTM model to assess its plausibility, revealing its coherence with established knowledge of vegetation phenology sensitivity to meteorological conditions, particularly changes in temperature. Our study highlights that (1) multi-variate meteorological memory effects play a crucial role in vegetation phenology, and (2) deep learning opens up new avenues for improving the representation of vegetation phenological processes in land surface models via a hybrid modelling approach.

Geology↗