Forecasting Nodal Price Difference Between Day-Ahead and Real-Time Electricity Markets Using Long-Short Term Memory and Sequence-to-Sequence Networks
Not Available
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
SEARCH · Search NASA
Results for “Long-Short Term Memory”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Enhancing Streamflow Forecast and Extracting Insights Using Long-Short Term Memory Networks With Data Integration at Continental Scales
Recent observations with varied schedules and types (moving average, snapshot, or regularly spaced) can help to improve streamflow forecasts, but it is challenging to integrate them effectively. Based on a long short-term memory (LSTM) streamflow model, we tested multiple versions of a flexible procedure we call data integration (DI) to leverage recent discharge measurements to improve forecasts. DI accepts lagged inputs either directly or through a convolutional neural network unit. DI ubiquitously elevated streamflow forecast performance to unseen levels, reaching a record continental-scale median Nash-Sutcliffe Efficiency coefficient value of 0.86. Integrating moving-average discharge, discharge from the last few days, or even average discharge from the previous calendar month could all improve daily forecasts. Directly using lagged observations as inputs was comparable in performance to using the convolutional neural network unit. Importantly, we obtained valuable insights regarding hydrologic processes impacting LSTM and DI performance. Before applying DI, the base LSTM model worked well in mountainous or snow-dominated regions, but less well in regions with low discharge volumes (due to either low precipitation or high precipitation-energy synchronicity) and large interannual storage variability. DI was most beneficial in regions with high flow autocorrelation: it greatly reduced baseflow bias in groundwater-dominated western basins and also improved peak prediction for basins with dynamical surface water storage, such as the Prairie Potholes or Great Lakes regions. However, even DI cannot elevate performance in high-aridity basins with 1-day flash peaks. Despite this limitation, there is much promise for a deep-learning-based forecast paradigm due to its performance, automation, efficiency, and flexibility.
54 ENVIRONMENTAL SCIENCES↗
Cross-Market Price Difference Forecast Using Deep Learning for Electricity Markets
Price forecasting is in the center of decision making in electricity markets. Many researches have been done in forecasting energy prices while little research has been reported on forecasting price difference between day-ahead and realtime markets due to its high volatility, which however plays a critical role in virtual trading. To this end, this paper takes the first attempt to employ novel deep learning architecture with Bidirectional Long-Short Term Memory (LSTM) units to forecast the price difference between day-ahead and real-time markets for the same node. The raw data is collected from PJM market, processed and fed into the proposed network. The Root Mean Squared Error (RMSE) and customized performance metric are used to evaluate the performance of the proposed method. Case studies show that it outperforms the traditional statistical models like ARIMA, and machine learning models like XGBoost and SVR methods in both RMSE and the capability of forecasting the sign of price difference. Additionally, to cross-market price difference forecast, the proposed approach has the potential to be applied to solve other forecasting problems such as price spread forecast in DA market for Financial Transmission Right (FTR) trading purpose.
DA/RT price difference↗
Time series anomaly detection in power electronics signals with recurrent and ConvLSTM autoencoders
The anomalies in the high voltage converter modulator (HVCM) remain a major down time for the spallation neutron source facility, that delivers the most intense neutron beam in the world for scientific materials research. In this work, we propose neural network architectures based on Recurrent AutoEncoders (RAE) to detect anomalies ahead of time in the power signals coming from the HVCM. Bi-directional gated recurrent unit, bi-directional long-short term memory (LSTM), and convolutional LSTM (ConvLSTM) are developed, trained, and tested using real experimental signals from the HVCM module. The results show a good performance of the proposed RAE models, achieving precision up to 91%, recall up to 88%, false omission rate as low as 20% (i.e. 80% of the anomalies were detected), and area under the ROC curve up to 0.9. The three RAE models provide very comparable performance, with LSTM showing slightly better performance than GRU and ConvLSTM. The RAE models are benchmarked against other anomaly detection methods, including isolation forest, support vector machine, local outlier factor, feedforward and convolutional autoencoders, and others; showing a better performance. Here, the results of this study demonstrate the promising potential of RAE in anomaly detection for real-world power systems, and for increasing the reliability of the HVCM modules in the spallation neutron source.
42 ENGINEERING↗
Temporal Forecasting of Distributed Temperature Sensing in a Thermal Hydraulic System With Machine Learning and Statistical Models
We benchmark performance of long-short term memory (LSTM) network machine learning model and autoregressive integrated moving average (ARIMA) statistical model in temporal forecasting of distributed temperature sensing (DTS). Data in this study consists of fluid temperature transient measured with two co-located Rayleigh scattering fiber optic sensors (FOS) in a forced convection mixing zone of a thermal tee. We treat each gauge of a FOS as an independent temperature sensor. We first study prediction of DTS time series using Vanilla LSTM and ARIMA models trained on prior history of the same FOS that is used for testing. The results yield maximum absolute percentage error (MaxAPE) and root mean squared percentage error (RMSPE) of 1.58% and 0.06% for ARIMA, and 3.14% and 0.44% for LSTM, respectively. Next, we investigate zero-shot forecasting (ZSF) with LSTM and ARIMA trained on history of the co-located FOS only, which is advantageous when limited training data is available. The ZSF MaxAPE and RMSPE values for ARIMA are comparable to those of the Vanilla use case, while the error values for LSTM increase. We show that in ZSF, performance of LSTM network can be improved by training on most correlated gauges between the two FOS, which are identified by calculating the Pearson correlation coefficient. The improved ZSF MaxAPE and RMSPE for LSTM are 4.4% and 0.33%, respectively. Performance of ZSF LSTM can be further enhanced through transfer learning (TL), where LSTM is re-trained on a subset of the FOS that is the target of forecasting. We show that LSTM pre-trained on correlated dataset and re-trained on 30% of testing target dataset achieves MaxAPE and RMSPE values of 2.32% and 0.28%, respectively.
ARIMA↗
Security Enhancement of Network Constraint Grid-Edge Energy Management System
Network constrained grid edge energy management system (EMS) provides economic solution for active and reactive power dispatch of distributed energy resources (DERs) at the grid edge level. Grid edge EMS ensures secure interconnection of a circuit segment to the distribution system by maintaining grid code requirements (e.g. IEEE 1547–2018). Grid edge EMS is dependent on communication to receive load measurement, which brings a risk of unobservable false data injection attacks (FDIAs). To mitigate the risk, this paper proposes a framework to enhance resilient operation of grid edge EMS by detecting the unobservable FDIAs on loads and replacing them with forecasted values. In this work, a two-step detection algorithm is proposed. In first step, conventional residual based algorithm is deployed. Autoencoder (AE) based data driven mechanism is included in second step to detect the presence of unobservable FDIAs. After ensuring the presence of FDIA, its specific location is detected by checking the maximum residue values till the predefined threshold value is reached. Detected false data injected loads are then replaced with forecasted load values following long-short term memory (LSTM) based forecast to ensure resilient performance of grid edge EMS in the presence of attacks. This proposed security enhancement framework for grid edge EMS is evaluated in IEEE 13 bus system with three integrated DERs. Numerical simulation shows the validation of the proposed framework by reducing voltage violation in real operation of grid edge EMS.
cyber attack detection↗
Machine Learning Assisted Reservoir Operation Model for Long–Term Water Management Simulation
This study explores strategies for long-term reservoir simulations by combining generic rule-based reservoir management model (RMM) and machine learning (ML) models for two major multipurpose reservoirs — Allatoona Lake and Lake Sidney Lanier in the southeastern United States. First, a standalone RMM is developed to simulate daily release and storage during Water Year 1981–2015. Next, using Long-Short Term Memory (LSTM) as the ML technique, a standalone LSTM model is trained based on reservoir inflow and meteorological observations to simulate reservoir release and estimate reservoir storage through water balance calculation. Three hybrid modeling strategies are developed, one using RMM output as an additional LSTM input (H1), another using LSTM as the initial release estimate in RMM (H2), and the third combining the first two strategies (H3). The Nash–Sutcliffe efficiency (NSE) for release (NSE-r), storage (NSE-s), and their mean (NSE-avg) are used for model evaluation. Overall, H1 improves NSE-r to 0.65 and 0.54 for Allatoona and Lanier, respectively, compared to standalone RMM (0.44 and 0.21); however, its storage trajectory did not produce a physically feasible solution, similar to LSTM. H2 and especially H3 show that they can retain the best features from RMM and LSTM, with H3 NSE-avg being 0.695 and 0.55 for Allatoona and Lanier outperforming RMM (0.615 and 0.29). In conclusion, the findings suggest a robust simulation capacity for large-scale water management in future studies.
54 ENVIRONMENTAL SCIENCES↗
An LSTM Deep Learning Network for ¿Background Radiation Prediction
Determination of appropriate background radiation is important in any measurement application. Environmental radiation monitors and monitors used to assess dose to individuals outside of controlled areas are particularly susceptible to changes in readings due to fluctuations in the environmental conditions. These fluctuations (e.g. radon progeny concentrations) lead to changes in the detector response in the actual radiation environment, and they need to be taken into account when extracting the net operational doses. Work has been ongoing to apply advances in Deep Learning and Artificial Intelligence to account for changes in detector responses based on environmental parameters; in particular, a Long-Short Term Memory (LSTM) Deep Learning architecture has been utilized to incorporate time-series data into a prediction model. In this work, we present the current status of the project to predict radiation measurements based on meteorological conditions and air packet trajectories extracted using the National Oceanic and Atmospheric Administration's (NOAA) Hybrid Single-Particle Lagrangian Integrated Trajectory (HYSPLIT4) model.
Degtiarenko, Pavel↗
Long Short-Term Memory Networks for Pattern Recognition of Synthetical Complete Earthquake Catalog
Exploring the spatiotemporal distribution of earthquake activity, especially earthquake migration of fault systems, can greatly to understand the basic mechanics of earthquakes and the assessment of earthquake risk. By establishing a three-dimensional strike-slip fault model, to derive the stress response and fault slip along the fault under regional stress conditions. Our study helps to create a long-term, complete earthquake catalog. We modelled Long-Short Term Memory (LSTM) networks for pattern recognition of the synthetical earthquake catalog. The performance of the models was compared using the mean-square error (MSE). Our results showed clearly the application of LSTM showed a meaningful result of 0.08% in the MSE values. Our best model can predict the time and magnitude of the earthquakes with a magnitude greater than Mw = 6.5 with a similar clustering period. These results showed conclusively that applying LSTM in a spatiotemporal series prediction provides a potential application in the study of earthquake mechanics and forecasting of major earthquake events.
58 GEOSCIENCES↗
On Neural Architectures for Astronomical Time-series Classification with Application to Variable Stars
Despite the utility of neural networks (NNs) for astronomical time-series classification, the proliferation of learning architectures applied to diverse data sets has thus far hampered a direct intercomparison of different approaches. Here we perform the first comprehensive study of variants of NN-based learning and inference for astronomical time series, aiming to provide the community with an overview on relative performance and, hopefully, a set of best-in-class choices for practical implementations. In both supervised and self-supervised contexts, we study the effects of different time-series-compatible layer choices, namely the dilated temporal convolutional neural network (dTCNs), long-short term memory NNs, gated recurrent units and temporal convolutional NNs (tCNNs). Additionally, we also study the efficacy and performance of encoder-decoder (i.e., autoencoder) networks compared to direct classification networks, different pathways to include auxiliary (non-time-series) metadata, and different approaches to incorporate multi-passband data (i.e., multiple time series per source). Performance—applied to a sample of 17,604 variable stars (VSs) from the MAssive Compact Halo Objects (MACHO) survey across 10 imbalanced classes—is measured in training convergence time, classification accuracy, reconstruction error, and generated latent variables. We find that networks with recurrent NNs generally outperform dTCNs and, in many scenarios, yield to similar accuracy as tCNNs. In learning time and memory requirements, convolution-based layers perform better. We conclude by discussing the advantages and limitations of deep architectures for VS classification, with a particular eye toward next-generation surveys such as the Legacy Survey of Space and Time, the Roman Space Telescope, and Zwicky Transient Facility.
79 ASTRONOMY AND ASTROPHYSICS↗
Models and Processes to Extract Drug-like Molecules From Natural Language Text
Researchers worldwide are seeking to repurpose existing drugs or discover new drugs to counter the disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A promising source of candidates for such studies is molecules that have been reported in the scientific literature to be drug-like in the context of viral research. However, this literature is too large for human review and features unusual vocabularies for which existing named entity recognition (NER) models are ineffective. We report here on a project that leverages both human and artificial intelligence to detect references to such molecules in free text. We present 1) a iterative model-in-the-loop method that makes judicious use of scarce human expertise in generating training data for a NER model, and 2) the application and evaluation of this method to the problem of identifying drug-like molecules in the COVID-19 Open Research Dataset Challenge (CORD-19) corpus of 198,875 papers. We show that by repeatedly presenting human labelers only with samples for which an evolving NER model is uncertain, our human-machine hybrid pipeline requires only modest amounts of non-expert human labeling time (tens of hours to label 1778 samples) to generate an NER model with an F-1 score of 80.5%—on par with that of non-expert humans—and when applied to CORD’19, identifies 10,912 putative drug-like molecules. This enriched the computational screening team’s targets by 3,591 molecules, of which 18 ranked in the top 0.1% of all 6.6 million molecules screened for docking against the 3CLPro protein.
60 APPLIED LIFE SCIENCES↗