Search NASA⌕ Search

SEARCH · Search NASA

Results for “Recurrent Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

A Mass‐Conserving‐Perceptron for Machine‐Learning‐Based Modeling of Geoscientific Systems

Although decades of effort have been devoted to building Physical-Conceptual (PC) models for predicting the time-series evolution of geoscientific systems, recent work shows that Machine Learning (ML) based Gated Recurrent Neural Network technology can be used to develop models that are much more accurate. However, the difficulty of extracting physical understanding from ML-based models complicates their utility for enhancing scientific knowledge regarding system structure and function. Here, we propose a physically interpretable Mass-Conserving-Perceptron (MCP) as a way to bridge the gap between PC-based and ML-based modeling approaches. The MCP exploits the inherent isomorphism between the directed graph structures underlying both PC models and GRNNs to explicitly represent the mass-conserving nature of physical processes while enabling the functional nature of such processes to be directly learned (in an interpretable manner) from available data using off-the-shelf ML technology. As a proof of concept, we investigate the functional expressivity (capacity) of the MCP, explore its ability to parsimoniously represent the rainfall-runoff (RR) dynamics of the Leaf River Basin, and demonstrate its utility for scientific hypothesis testing. To conclude, we discuss extensions of the concept to enable ML-based physical-conceptual representation of the coupled nature of mass-energy-information flows through geoscientific systems.

58 GEOSCIENCES↗

Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows

Reduced rank nonlinear filters are increasingly utilized in data assimilation of geophysical flows, but often require a set of ensemble forward simulations to estimate forecast covariance. On the other hand, predictor-corrector type nudging approaches are still attractive due to their simplicity of implementation when more complex methods need to be avoided. However, optimal estimate of nudging gain matrix might be cumbersome. In this paper, we put forth a fully nonintrusive recurrent neural network approach based on a long short-term memory (LSTM) embedding architecture to estimate the nudging term, which plays a role not only to force the state trajectories to the observations but also acts as a stabilizer. Furthermore, our approach relies on the power of archival data and the trained model can be retrained effectively due to power of transfer learning in any neural network applications. In order to verify the feasibility of the proposed approach, we perform twin experiments using Lorenz 96 system. Our results demonstrate that the proposed LSTM nudging approach yields more accurate estimates than both extended Kalman filter (EKF) and ensemble Kalman filter (EnKF) when only sparse observations are available. With the availability of emerging AI-friendly and modular hardware technologies and heterogeneous computing platforms, we articulate that our simplistic nudging framework turns out to be computationally more efficient than either the EKF or EnKF approaches.

42 ENGINEERING↗

Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers

Drawing on ergodic theory, we introduce a novel training method for machine learning based forecasting methods for chaotic dynamical systems. The training enforces dynamical invariants—such as the Lyapunov exponent spectrum and the fractal dimension—in the systems of interest, enabling longer and more stable forecasts when operating with limited data. The technique is demonstrated in detail using reservoir computing, a specific kind of recurrent neural network. Finally, results are given for the Lorenz 1996 chaotic dynamical system and a spectral quasi-geostrophic model of the atmosphere, both typical test cases for numerical weather prediction.

97 MATHEMATICS AND COMPUTING↗

A generative artificial intelligence framework for long-time plasma turbulence simulations

Generative deep learning techniques are employed in a novel framework for the construction of surrogate models capturing the spatiotemporal dynamics of 2D plasma turbulence. The proposed Generative Artificial Intelligence Turbulence (GAIT) framework enables the acceleration of turbulence simulations for long-time transport studies. GAIT leverages a convolutional variational auto-encoder and a recurrent neural network to generate new turbulence data from existing simulations, extending the time horizon of transport studies with minimal computational cost. The application of the GAIT framework to plasma turbulence using the Hasegawa–Wakatani (HW) model is presented, evaluating its performance via various analyses. Very good agreement is found between the GAIT and the HW models in the spatiotemporal Fourier and Proper Orthogonal Decomposition spectra, the flow topology characterized by the Okubo–Weiss parameter, and the time autocorrelation function of turbulent fluctuations. Excellent agreement has also been obtained in the probability distribution function of particle displacements and the effective turbulent diffusivity. In-depth analyses of the latent space of turbulent states, choice of hyperparameters and alternative deep learning models for the time prediction are presented. Our results highlight the potential of Artificial Intelligence-based surrogate models to overcome the computational challenges in turbulence simulation, which can be extended to other situations such as geophysical fluid dynamics.

Artificial intelligence↗

Comparison of machine learning systems trained to detect Alfvén eigenmodes using the CO 2 interferometer on DIII-D

Abstract A Machine-Learning (ML) based detection scheme that automatically detects Alfvén Eigenmodes (AE) in a labelled DIII-D database is presented here. Controlling AEs is important for the success of planned burning plasma devices such as ITER, since resonant fast ions can drive AEs unstable and degrade the performance of the plasma or damage the first walls of the machine vessel. Artificial Intelligence could be useful for real-time detection and control of AEs in steady-state plasma scenarios by implementing ML-based models into control algorithms that drive actuators for mitigation of AE impacts. Thus, the objective is to compare differences in performance between using two different recurrent neural network systems (Reservoir Computing Network and Long Short Term Memory Network) and two different representations of the C O 2 phase data (simple and crosspower spectrograms). All C O 2 interferometer chords are used to train both models, but only one is processed during each training step. The results from the model and data comparison show higher performance for the RCN model (True Positive Rate = 90% and False Positive Rate = 14%), and that using simple magnitude spectrograms is sufficient to detect AEs. Also, the vertical C O 2 interferometer chord passing near the center is better for ML-based detection of AEs.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Enhancement of Unsteady and 3D Aerodynamics Models using Machine Learning

Unsteady aerodynamics will be an important part of the floating wind turbines of the future operating under high shear across the rotor disk coupled with platform motion and atmospheric turbulence. We develop an unsteady aerodynamics and dynamic stall model using a long short-term memory variant of recurrent neural networks. The neural network model is trained using the oscillating airfoil data set from Ohio State University. The predictions from our machine learning (ML)-based model show good agreement with the experimental data and other state-of-the-art dynamic stall models for a wide range of airfoils, Reynolds numbers and reduced frequencies. In some cases the predictions are better than the Beddoes-Leishman model implementation in OpenFAST, when using the default coefficients. The ML-based model is also able to capture the key physics associated with dynamic stall, such as the precedence of moment stall before lift stall and cycle-to-cycle variations in the aerodynamic response. The new unsteady aerodynamics model is expected to improve prediction of fatigue loads for yaw-based wake-steering control scenarios in actuator-line and actuator-disc simulations of wind farms. Our methodology for training the ML-model provides a pathway for improving design level tools using high-fidelity computational fluid dynamics (CFD) simulations in the future.

17 WIND ENERGY↗

On Building Predictive Digital Twin Incorporating Wave Predicting Capabilities: Case Study on UMaine Experimental Campaign - FOCAL

The response of floating wind turbines (FWT) are susceptible to stochastic wave variations. For the optimal operation of FWT, a comprehensive understanding of the phaseresolved wave dynamics and the consequential system response is crucial for real-time monitoring and control. A multi-variate, multi-step, long short term memory (MLSTM), a type of recurrent neural network (RNN) is used to capture complex system dynamics for real-time application. Results indicate that the integration of a wave prediction-reconstruction (WRP) model substantially enhances prediction accuracy by 50% on average relative to the baseline model. The improvement is consistent across various wave extremity and prediction horizons, thereby significantly broadening the scope for timely and precise predictive capabilities.

17 WIND ENERGY↗

Investigating permafrost carbon dynamics in Alaska with artificial intelligence

Abstract Positive feedbacks between permafrost degradation and the release of soil carbon into the atmosphere impact land–atmosphere interactions, disrupt the global carbon cycle, and accelerate climate change. The widespread distribution of thawing permafrost is causing a cascade of geophysical and biochemical disturbances with global impacts. Currently, few earth system models account for permafrost carbon feedback (PCF) mechanisms. This research study integrates artificial intelligence (AI) tools and information derived from field-scale surveys across the tundra and boreal landscapes in Alaska. We identify and interpret the permafrost carbon cycling links and feedback sensitivities with GeoCryoAI, a hybridized multimodal deep learning (DL) architecture of stacked convolutionally layered, memory-encoded recurrent neural networks (NN). This framework integratesin-situmeasurements and flux tower observations for teacher forcing and model training. Preliminary experiments to quantify, validate, and forecast permafrost degradation and carbon efflux across Alaska demonstrate the fidelity of this data-driven architecture. More specifically, GeoCryoAI logs the ecological memory and effectively learns covariate dynamics while demonstrating an aptitude to simulate and forecast PCF dynamics—active layer thickness (ALT), carbon dioxide flux (CO 2 ), and methane flux (CH 4 )—with high precision and minimal loss (i.e. ALT RMSE : 1.327 cm [1969–2022]; CO 2 RMSE : 0.697µmolCO 2 m −2 s −1 [2003–2021]; CH 4 RMSE : 0.715 nmolCH 4 m −2 s −1 [2011–2022]). ALT variability is a sensitive harbinger of change, a unique signal characterizing the PCF, and our model is the first characterization of these dynamics across space and time.

Environmental Sciences & Ecology↗

Predicting nonequilibrium Green’s function dynamics and photoemission spectra via nonlinear integral operator learning

Understanding the dynamics of nonequilibrium quantum many-body systems is an important research topic in a wide range of fields across condensed matter physics, quantum optics, and high-energy physics. However, numerical studies of large-scale nonequilibrium phenomena in realistic materials face serious challenges due to intrinsic high-dimensionality of quantum many-body problems and the absence of time-invariance. The nonequilibrium properties of many-body systems can be described by the dynamics of the correlator, or the Green's function of the system, whose time evolution is given by a high-dimensional system of integro-differential equations, known as the Kadanoff–Baym equations (KBEs). The time-convolution term in KBEs, which needs to be recalculated at each time step, makes it difficult to perform long-time numerical simulation. In this paper, we develop an operator-learning framework based on recurrent neural networks (RNNs) to address this challenge. We utilize RNNs to learn the nonlinear mapping between Green's functions and convolution integrals in KBEs. By using the learned operators as a surrogate model in the KBE solver, we obtain a general machine-learning scheme for predicting the dynamics of nonequilibrium Green's functions. Besides significant savings per each time step, the new methodology reduces the temporal computational complexity from $O(N_t^3)$ to $O(N_t)$ where N t is the number of steps taken in a simulation, thereby making it possible to study large many-body problems which are currently infeasible with conventional KBE solvers. Through various numerical examples, we demonstrate the effectiveness of the operator-learning based approach in providing accurate predictions of physical observables such as the reduced density matrix and time-resolved photoemission spectra. Moreover, our framework exhibits clear numerical convergence and can be easily parallelized, thereby facilitating many possible further developments and applications.

97 MATHEMATICS AND COMPUTING↗

Understanding the extreme luminosity of DES14X2fna

We present DES14X2fna, a high-luminosity, fast-declining Type IIb supernova (SN IIb) at redshift z = 0.0453, detected by the Dark Energy Survey (DES). DES14X2fna is an unusual member of its class, with a light curve showing a broad, luminous peak reaching M r ≃ –19.3 mag 20 d after explosion. This object does not show a linear decline tail in the light curve until ≃60 d after explosion, after which it declines very rapidly (4.30 ± 0.10 mag 100 d –1 in the r band). By fitting semi-analytic models to the photometry of DES14X2fna, we find that its light curve cannot be explained by a standard 56 Ni decay model as this is unable to fit the peak and fast tail decline observed. Inclusion of either interaction with surrounding circumstellar material or a rapidly-rotating neutron star (magnetar) significantly increases the quality of the model fit. Furthermore, we also investigate the possibility for an object similar to DES14X2fna to act as a contaminant in photometric samples of SNe Ia for cosmology, finding that a similar simulated object is misclassified by a recurrent neural network (RNN)-based photometric classifier as an SN Ia in ~1.1–2.4 per cent of cases in DES, depending on the probability threshold used for a positive classification.

79 ASTRONOMY AND ASTROPHYSICS↗

Hamiltonian learning using machine-learning models trained with continuous measurements

Here, we build upon recent work on the use of machine-learning models to estimate Hamiltonian parameters using continuous weak measurement of qubits as input. We consider two settings for the training of our model: (1) supervised learning, where the weak-measurement training record can be labeled with known Hamiltonian parameters, and (2) unsupervised learning, where no labels are available. The first has the advantage of not requiring an explicit representation of the quantum state, thus potentially scaling very favorably to a larger number of qubits. The second requires the implementation of a physical model to map the Hamiltonian parameters to a measurement record, which we implement using an integrator of the physical model with a recurrent neural network to provide a model-free correction at every time step to account for small effects not captured by the physical model. We test our construction on a system of two qubits and demonstrate accurate prediction of multiple physical parameters in both the supervised context and the unsupervised context. We demonstrate that the model benefits from larger training sets, establishing that it is “learning,” and we show robustness regarding errors in the assumed physical model by achieving accurate parameter estimation in the presence of unanticipated single-particle relaxation.

97 MATHEMATICS AND COMPUTING↗

Nonintrusive reduced order modeling framework for quasigeostrophic turbulence

In this study, we present a nonintrusive reduced order modeling (ROM) framework for large-scale quasistationary systems. The framework proposed in this work exploits the time series prediction capability of long short-term memory (LSTM) recurrent neural network architecture such that (1) in the training phase, the LSTM model is trained on the modal coefficients extracted from the high-resolution data snapshots using proper orthogonal decomposition (POD) transform, and (2) in the testing phase, the trained model predicts the modal coefficients for the total time recursively based on the initial time history. Hence, no prior information about the underlying governing equations is required to generate the ROM. To illustrate the predictive performance of the proposed framework, the mean flow fields and time series response of the field values are reconstructed from the predicted modal coefficients by using an inverse POD transform. As a representative benchmark test case, we consider a two-dimensional quasigeostrophic ocean circulation model which, in general, displays an enormous range of fluctuating spatial and temporal scales. We first demonstrate that the conventional Galerkin projection-based reduced order modeling of such systems requires a high number of POD modes to obtain a stable flow physics. In addition, ROM-Galerkin projection (ROM-GP) does not seem to capture the intermittent bursts appearing in the dynamics of the first few most energetic modes. However, the proposed nonintrusive ROM framework based on LSTM (ROM-LSTM) yields a stable solution even for a small number of POD modes. We also observe that the ROM-LSTM model is able to capture quasiperiodic intermittent bursts accurately, and yields a stable and accurate mean flow dynamics using the time history of a few previous time states, denoted as the lookback time window in this paper. We show several features of ROM-LSTM framework such as significantly higher accuracy than ROM-GP, and faster performance using larger time step size. Throughout the paper, we demonstrate our findings in terms of time series evolution of the field values and mean flow patterns, which suggest that the proposed fully nonintrusive ROM framework is robust and capable of predicting chaotic nonlinear fluid flows in an extremely efficient way compared to the conventional projection-based ROM framework.

42 ENGINEERING↗

Autoregressive neural quantum states of Fermi Hubbard models

Neural quantum states (NQSs) have emerged as a powerful ansatz for variational quantum Monte Carlo studies of strongly correlated systems. Here, we apply recurrent neural networks (RNNs) and autoregressive transformer neural networks to the Fermi-Hubbard and the (non-Hermitian) Hatano-Nelson-Hubbard models in one and two dimensions. In both cases, we observe that the convergence of the RNN ansatz is challenged when increasing the interaction strength. We present a physically motivated and easy-to-implement strategy for improving the optimization, namely, by ramping of the model parameters. Furthermore, we investigate the advantages and disadvantages of the autoregressive sampling property of both network architectures. For the Hatano-Nelson-Hubbard model, we identify convergence issues that stem from the autoregressive sampling scheme in combination with the non-Hermitian nature of the model. Our findings provide insights into the challenges of the NQS approach and make the first step towards exploring strongly correlated electrons using this ansatz. Published by the American Physical Society 2025

Ibarra-García-Padilla, Eduardo (ORCID:000000019165↗

The Challenge of Disproportionate Importance of Temporal Features in Predicting HPC Power Consumption

In this work, we demonstrate the challenges in predicting HPC cluster power consumption in the face of significant temporal skew in power consumption behavioral patterns. Predicting large power swings that extend several megawatts has significant operational value for HPC centers, however, prediction is challenging due to the relative rarity of such events and also due to the abrupt or disjoint deviation from the average power consumption levels. To study the impact of this challenge, we have trained a recurrent neural network (RNN) as a reasonably sophisticated model to predict power consumption of the one-year worth of node power consumption data from the Summit supercomputer located in the Oak Ridge Leadership Computing Facility. By studying the prediction results, we have found that although simple usage of RNN models can provide good results on average power consumption levels, it would fail at predicting the power swings that have more operational value. With such results, we discuss potential next steps in addressing such issues aiming towards a robust usage of power prediction techniques in HPC operations.

Li, Chengcheng↗

Use of Event-Time Embeddings via RNN to Discern Novel Event Sequences in EHRs

In highly configurable health information technology (HIT) systems, such as VistA of the Veterans Health Administration, the variations in how the system is used among different healthcare facilities and how the data are recorded can be significant. Despite the successful standardization of care efforts, some of these variations can be indicative of HIT hazards and demand further investigation. In this work, we implemented a recurrent neural network (RNN) architecture to learn clinical provider order sequences and their temporal dynamics while predicting the orders' terminal state. We demonstrate model performance and provide a use case for the model discerning novel event sequences. This model is proposed to find novel event sequences in an operational environment.

Ozmen, Ozgur↗

A Learning Framework for Control-Oriented Modeling of Buildings

Buildings consume a significant amount of energy worldwide. Several building optimization and control use cases require models of energy consumption which are control oriented, have high predictive capability, imposes minimal data pre-processing requirements, and have the ability to be adapted continuously to account for changing conditions as new data becomes available. Data driven modeling techniques, that have been investigated so far, while promising in the context of buildings, have been unable to simultaneously satisfy all the requirements mentioned above. In this context, deep learning techniques such as Recurrent Neural Networks (RNNs) hold promise, empowered by advanced computational capabilities and big data opportunities. In this paper, we propose a deep learning based methodology for the development of control oriented models for building energy management and test in on data from a real building. Results show that the proposed methodology outperforms other data driven modeling techniques significantly. We perform a detailed analysis of the proposed methodology along dimensions such as topology, sensitivity, and downsampling. Lastly, we conclude by envisioning a building analytics suite empowered by the proposed deep framework, that can drive several use cases related to building energy management.

buildings↗

Fingerprinting Anomalous Computation with RNN for GPU-accelerated HPC Machines

This paper presents a workload classification framework that discriminates illicit computation from authorized workloads on GPU-accelerated HPC systems. As such systems become more and more powerful, they are exploited by attackers to run malicious and for-profit programs that typically require extremely high computing ability to be successful. Our classification framework leverages the distinctive signatures between illicit and authorized workloads, and explore machine learning methods to learn the workloads and classify them. The framework uses lightweight, non-intrusive workload profiling to collect model input data, and explores multiple machine learning methods, particularly recurrent neural network (RNN) that is suitable for online anomalous workload detection. Evaluation results on four generations of GPU machines demonstrate that the workload classification framework can tell apart the illicit authorized workloads with a high accuracy of over 95%.

Zou, Pengfei↗

Univariate Time Series Prediction of Solar Power Using a Hybrid Wavelet-ARMA-NARX Prediction Method

This paper proposes a new hybrid method for super short-term solar power prediction. Solar output power usually has a complex, nonstationary, and nonlinear characteristic due to intermittent and time varying behavior of solar radiance. In addition, solar power dynamics is fast and is inertia less. An accurate super short-time prediction is required to compensate for the fluctuations and reduce the impact of solar power penetration on the power system. The objective is to predict one step-ahead solar power generation based only on historical solar power time series data. The proposed method incorporates discrete wavelet transform (DWT), Auto-Regressive Moving Average (ARMA) models, and Recurrent Neural Networks (RNN), while the RNN architecture is based on Nonlinear Auto-Regressive models with eXogenous inputs (NARX). The wavelet transform is utilized to decompose the solar power time series into a set of richer-behaved forming series for prediction. ARMA model is employed as a linear predictor while NARX is used as a nonlinear pattern recognition tool to estimate and compensate the error of wavelet-ARMA prediction. The proposed method is applied to the data captured from UCLA solar PV panels and the results are compared with some of the common and most recent solar power prediction methods. The results validate the effectiveness of the proposed approach and show a considerable improvement in the prediction precision.

Nazaripouya, Hamidreza↗