Identifying Recurrent Causal Activity Patterns in Spiking Neural Networks
Explore the source record for details and available documents.
Theilman, Bradley Harold↗
SEARCH · Search NASA
Results for “Recurrent networks”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Dynamic data-driven multiscale modeling for predicting the degradation of a 316L stainless steel nuclear cladding material
Here, we have developed a long short-term memory stacked ensemble (LSTM-SE) surrogate modeling approach that can provide rapid predictions of microstructural evolution and the resultant mechanical properties of American Iron and Steel Institute (AISI) 316L series stainless steel (316LSS) fuel cladding under conditions of varying temperature and radiation dose rate. To acquire training data, we developed and implemented a kinetic Monte Carlo (KMC) model to simulate precipitation kinetics of M 23 C 6 , γ', and G phases within SS316L cladding. Experimentally reported precipitation kinetics of SS316L in literature were linked to the kinetic parameters of the simulated precipitation in our KMC model. The model was then used to simulate microstructure evolution under synthetically generated treatments of varying temperature and radiation dose rate, for periods of up to 3000 hours. Changes in volume fraction, number density, and particle size of precipitates were recorded, and particle area fractions were correlated using statistical methods to develop the surrogate model. Simultaneously, the mechanical properties of the simulated microstructures were evaluated using microstructure-based finite element method (FEM) analysis to determine the elastic modulus, yield stress, ultimate tensile strength, and elongation to failure of the aged microstructures. Using this approach, our surrogate model can predict precipitation behavior within 0.25% volume fraction and mechanical properties within 6% relative error from the values predicted by the KMC and FEM models using 50 training simulations as input. The trained recurrent neural network-based model can return estimations of precipitation kinetics and mechanical properties ~1000 times faster than the physics-based codes. This work demonstrates, as a proof of concept, that reactor material service lifetimes under variable service conditions can be predicted for a statistics-based model from a practicably obtainable dataset.
36 MATERIALS SCIENCE↗
Development of Predictive Models for Advanced Reactor Autonomous Control
Advanced reactor designs including microreactors and small modular reactors will contribute to the clean production of cheap energy, and autonomous control for advanced reactors is an appealing option for reducing cost. However, there is a lack of industry experience applying autonomous control for advanced nuclear reactors. To accelerate the development and industry acceptance of autonomous control software for nuclear reactors, we aim to demonstrate autonomous control of the Purdue University research reactor (PUR-1) using INL-developed model predictive control (MPC) methods. To prepare for this demonstration, data-driven predictive models based on process data collected from PUR-1 have been developed and integrated with MPC and used to control a physics-based model of PUR-1. A data-driven dynamics model and a gated recurrent unit (GRU) network were both trained on process data from PUR-1. The dynamics model was shown to effectively control the reactor model with MPC when provided reactivity as a control variable but failed to control the model through the control rod positions. The GRU network produced more accurate predictions than the dynamics model when evaluated on operational data, and future work will include the evaluation of the GRU network in the controller.
22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
Stochastic architecture for Hopfield neural nets
An expandable stochastic digital architecture for recurrent (Hopfield like) neural networks is proposed. The main features and basic principles of stochastic processing are presented. The stochastic digital architecture is based on a chip with n full interconnected neurons with a pipeline, bit processing structure. For large applications, a flexible way to interconnect many such chips is provided.
Pavel, Sandy↗
Neuromorphic processing for next generation automotive control and diagnostics
This paper describes intra-layer architecture of a neuroprocessor organized in a SIMD configuration and the two challenging applications, misfire detection and engine idle speed control, that had served as the focus of this effort.
neuroprocessors idle speed control misfire detecti↗
Leveraging Hydropower Multi-Sensor Data for Inference and Age-Informed Modeling
Increased demand of operational flexibility such as faster ramp up/down in generation, and more frequent start/stops are putting hydropower plants and their associated components in unprecedented stress. Consequently, these plants are at the high risk of extended and more frequent outage to accommodate unscheduled, and unexpected maintenance. Therefore, hydropower plants are in critical need of data driven and age-informed analysis for their regular and unscheduled operation. Yet not all hydropower plants are exhaustively equipped with sensors and/or measurement streams for their respective components – demanding solutions on how to detect, identify, and locate the cause of any event from the unobservable. Idaho National Laboratory (INL) analyzed the anonymized measurements and event records from the Hydropower Research Institute (HRI) to address this issue, as part of the Water Power Technologies Office (WPTO) funded one year multi-lab project. First, we investigated how time series of multiple sensor measurements can be leveraged to identify an event “root cause” as well as to develop an inference (i.e., estimate the unobservable) problem. INL also investigated how individual hydropower components’ reaction or response times vary across the pre-event, during event, and post-event conditions – enabling the hydropower dynamic models to be age-informed. Finally, the impact of clustering multi-sensor time series on short-term vibration prediction is analyzed. INL will present key findings from these analyses and recommend next steps for stakeholder adoption.
13 HYDRO ENERGY↗
A Comparative Study of Physics‐Informed and Data‐Driven Neural Networks for Compound Flood Simulation at River‐Ocean Interfaces: A Case Study of Hurricane Irene
Simulating compound flooding (CF) at the river-ocean interface within large-scale Earth System Models (ESMs) presents significant challenges due to complex interactions between river discharge, storm surge, and tides. This study assesses the comparative advantages of physics-informed and data-driven machine learning (ML) approaches for enhancing local ESM performance. We systematically compare data-driven neural network models (i.e., CNNs, U-Net, Long Short-Term Memory (LSTM), Gated Recurrent Unit), and physics-informed neural network (PINN) models, including vanilla PINN and a finite-difference-based PINN (FD-PINN). Specifically, FD-PINN is introduced to enhance computational efficiency, accelerating vanilla PINNs by ∼6.5 times while improving accuracy. To enhance data-driven model training, a new data-generation approach is developed to sample historical fluvial and coastal flood events, which ensures a robust data set for extreme event prediction. The models are evaluated using a realistic one-dimensional river domain extracted from an ESM's river mesh and the Hurricane Irene event as an independent test case. Results show that FD-PINN achieves accurate predictions with significantly reduced computational costs relative to vanilla PINNs. Among data-driven models, the best overall performance is achieved by a CNN-LSTM hybrid, which balances accuracy and efficiency. While a fully connected CNN (CNN-FC) provides the best accuracy, it incurs high computational cost. Architectures lacking strong temporal modeling tend to underperform on unseen events. These findings highlight the importance of sequence-aware designs for robust generalization. This study reveals the trade-offs between physics-informed and data-driven models and proposes an adaptive hybrid framework for integrating ML into ESMs to enhance local flood simulations.
Earth Systems Modeling↗
Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification" Willard et al. (2025).
This data release provides all data and code used in the paper " "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantifications" Willard et al. (2025)" to model stream temperature, evaluate, and assess results. The associated manuscript explores the effect of different ensemble construction techniques across different common machine learning (ML) architectures for predictions in unmonitored basins. Modeling was done using long short-term memory (LSTM), gated recurrent unit (GRU), temporal convolution network (TCN), and extreme gradient boosting (XGBoost) models, and stream site coverage spans 1362 locations across the conterminous United States. The ensemble construction techniques investigated include ensemble by random weight initialization, differing hyperparameters, different random subsets of training data, different subselections of input features, different architectures, and Monte Carlo Dropout. The data is organized into these items items:Code repository and data for the paper " "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantifications" Willard et al. (2025).Code: stream_temp_ml_regionalization.zip contains the code repositoryData to run the code:- data_dir.zip -- contains all files that should be moved to the "DATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- metadata_dir.zip -- contains all files that should be moved to the "METADATA_DIR" variable defined in the "set_env_vars.sh" script in the code repositoryData produced by the code and used in the paper:- outputs_dir.zip - contains model output and results (outputs_dir/results), model weights (outputs_dir/models), and all other outputs used for the paper including feature importances.To cite this code, please use the following BibTeX or MLA entries:bibtex:@misc{willard2025streamensembles,author = {Jared Willard and Charuleka Varadharajan},title = {Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification"},year = {2024},doi = {10.15485/2527393},publisher = {ESS-DIVE Repository},url = {https://data.ess-dive.lbl.gov/datasets/doi:10.15485/2527393}}MLA: Willard, Jared, et al. Dataset for "Machine Learning Ensembles Can Enhance Hydrologic Predictions and Uncertainty Quantification". 2025. ESS-DIVE Repository, doi:10.15485/2448016.
54 ENVIRONMENTAL SCIENCES↗
Transformer Neural Networks with Spatiotemporal Attention for Predictive Control and Optimization of Industrial Processes
In the context of real-time optimization and model predictive control of industrial systems, machine learning, and neural networks represent cutting-edge tools that hold promise for enhancing dynamic modeling. This work presents a novel transformer neural network architecture for real-time optimization and model predictive control. This network design includes a modified attention mechanism inspired by positional embedding attention from vision transformers and task-specific modifications to the input-output structure of the transformer’s decoder stack. Experiments were conducted using data from a 450 MW coal-fired power plant to evaluate this approach's effectiveness. The transformer neural network was compared with conventional recurrent models, including GRU and LSTM. The transformer exhibited a 6% increase in the R-squared (R2) value of predictions and an 83% reduction in mean squared error (MSE). Computation time was also reduced by 84% compared to conventional recurrent models.
Gallup, Ethan R.↗
GRUMDN: A Multi-Task Model for Predicting Human Patterns-of-Life from Stay Transition Data
Understanding human patterns-of-life (PoL) is essential towards ensuring safe and secure indoor facility environment as well as outdoor urban environment. Prediction of human movement in between places of interest is vital in understanding human PoL. Movement between spaces maybe represented and detected in one of the two forms: 1) trajectories: locations measured at regular time intervals by mobile sensors, bluetooth or GPS sensors; or 2) stay transitions: semantic PoI (points of interest) and stay duration data measurable by eventbased sensors that collect data when a check-in or check-out event is detected. Stay transition data provides a more compressed data format compared to trajectories data, especially in situations with longer stay durations, while preserving the information necessary for PoL analysis. Now as introduced briefly in the paper, our deployed end application (Digital Twin of a facility with non-player characters, besides the interactive user in virtual reality) needed a well-performing and validated AI/ML model for simulating high quality stay transitions behavior. In this study we thus primarily present our findings with developing and validating that model, which is a multi-task neural network for stay transition prediction. The neural network consists of two heads, for corresponding two tasks of stay category prediction and stay duration prediction. We evaluated gated recurrent units and multi-layer perceptrons of varying network sizes for stay category prediction; while mixture density networks, noisy generator-only networks, and generative adversarial networks of varying network sizes for stay duration prediction. We have then evaluated four multi-task models, constructed by combining these specialized models, on their ability to predict stay transition data. We tested our models on datasets from two different cases: 1) a simulation-generated dataset of indoor movement within the HFIR (high flux isotope reactor) nuclear reactor facility at Oak Ridge National Laboratory (ORNL); and 2) the GeoLife human mobility dataset of outdoor urban movement available in literature. Our results indicate that GRUMDN, which combines gated recurrent units (GRU) for stay category prediction task, and mixture density networks (MDN) for stay duration prediction task, did overall outperform other multitask models and the current state-of-the-art.
Gunaratne, Chathika [ORNL] (ORCID:0000000225088745↗
Modular, Hierarchical Learning By Artificial Neural Networks
Modular and hierarchical approach to supervised learning by artificial neural networks leads to neural networks more structured than neural networks in which all neurons fully interconnected. These networks utilize general feedforward flow of information and sparse recurrent connections to achieve dynamical effects. The modular organization, sparsity of modular units and connections, and fact that learning is much more circumscribed are all attractive features for designing neural-network hardware. Learning streamlined by imitating some aspects of biological neural networks.
Baldi, Pierre F.↗
NeuroCoreX: An Open-Source FPGA-Based Spiking Neural Network Emulator with On-Chip Learning
Spiking Neural Networks (SNNs) are computational models inspired by the event-driven communication and connectivity patterns of biological neural circuits. They enable high energy efficiency and natural support for diverse architectures ranging from layered networks to small-world and graphstructured topologies. In this work, we introduce NeuroCoreX, an open-source, FPGA-based spiking neural network emulator that provides real-time, on-chip learning and flexible network organization. NeuroCoreX supports both feedforward sensory inputs streamed directly from sensors or PCs via UART and recurrent on-chip connectivity, enabling simultaneous processing and learning from external stimuli and internal network dynamics-capabilities rarely available in existing FPGA SNN platforms. The system implements a Leaky Integrate-and-Fire (LIF) neuron model with current-based synapses and supports pair-based STDP learning on both feedforward and recurrent synapses. A lightweight Python interface enables interactive configuration, live monitoring, weight read-back, and experiment control. Importantly, NeuroCoreX is tightly integrated with the SuperNeuroMAT simulator, allowing SNN models to be transferred seamlessly from software to hardware for hardware-in-the-loop development. By combining real-time plasticity, flexible connectivity, and an open-source VHDL implementation, NeuroCoreX provides an extensible and accessible platform for neuromorphic research, algorithm-hardware co-design, and energy-efficient edge intelligence.
Gautam, Ashish [ORNL]↗
A neural network with modular hierarchical learning
This invention provides a new hierarchical approach for supervised neural learning of time dependent trajectories. The modular hierarchical methodology leads to architectures which are more structured than fully interconnected networks. The networks utilize a general feedforward flow of information and sparse recurrent connections to achieve dynamic effects. The advantages include the sparsity of units and connections, the modular organization. A further advantage is that the learning is much more circumscribed learning than in fully interconnected systems. The present invention is embodied by a neural network including a plurality of neural modules each having a pre-established performance capability wherein each neural module has an output outputting present results of the performance capability and an input for changing the present results of the performance capabilitiy. For pattern recognition applications, the performance capability may be an oscillation capability producing a repeating wave pattern as the present results. In the preferred embodiment, each of the plurality of neural modules includes a pre-established capability portion and a performance adjustment portion connected to control the pre-established capability portion.
Baldi, Pierre F.↗
Controlling basins of attraction in a neural network-based telemetry monitor
The size of the basins of attraction around fixed points in recurrent neural nets (NNs) can be modified by a training process. Controlling these attractive regions by presenting training data with various amount of noise added to the prototype signal vectors is discussed. Application of this technique to signal processing results in a classification system whose sensitivity can be controlled. This new technique is applied to the classification of temporal sequences in telemetry data.
Bell, Benjamin↗
Dense Feature Tracking of Atmospheric Winds with Deep Optical Flow
Atmospheric winds are a key physical phenomenon impacting natural hazards, energy transport, ocean currents, large-scale circulation, and ecosystem fluxes. Observing winds is a complex process and presents a large gap in NASA’s Earth Observation System. Atmospheric motion vectors (AMVs) aim to fill this gap by making numerical estimates of cloud movement between sequences of multi-spectral satellite images, tracking clouds and water vapor. Recent imaging hardware and software advancements have enabled the use of numerical optical flow techniques to produce accurate and dense vector fields outperforming traditional methods. This work presents WindFlow as the first machine learning based system for feature tracking atmospheric motion using optical flow. Due to the lack of large-scale satellite-based observations, we leverage high-resolution numerical simulations from NASA's GEOS-5 Nature Run to perform supervised learning and transfer to satellite images. We demonstrate that our approach using deep learning based optical flow scales to ultra-high-resolution images of size 2881x5760 with less than 1 m/s bias and 2.5 m/s average error. Four network and learning architectures are compared and it is found that recurrent all-pairs field transforms (RAFT) produces the lowest errors on all metrics for wind speed and direction. Results on held out numerical outputs shows RAFT's good performance in each of the spatial, temporal, and physical dimensions. A comparison between WindFlow and an operational AMV product against rawinsonde observations show that RAFT transfers across simulations and thermal infrared satellite observations. This work shows that machine learning based optical flow is an efficient approach to generating robust feature tracking for AMVs consistently over large regions.
Atmospheric winds↗
A hereditary integral, transient network approach to modeling permanent set and viscoelastic response in polymers
An efficient numerical framework is presented for modeling viscoelasticity and permanent set of polymers. It is based on the hereditary integral form of transient network theory, in which polymer chains belong to distinct networks each with different natural equilibrium states. Chains continually detach from previously formed networks and reattach to new networks in a state of zero stress. The free energy of these networks is given in terms of the deformation gradient relative to the configuration at which the network was born . A decomposition of the kernel for various free energies allows for a recurrence relationship to be established, bypassing the need to integrate over all time history. The technique is established for both highly compressible and nearly incompressible materials through the use of neo-Hookean, Blatz–Ko, Yeoh, and Ogden-Hill material models. Multiple examples are presented showing the ability to handle rate-dependent response and residual strains under complex loading histories.
Finite element method↗
African Equatorial and Subtropical Ozone Plumes: Recurrences Timescales of the Brown Cloud Trans-African Plumes and Other Plumes
We have found repeated illustrations in the maps of Total Tropospheric Ozone (TTO) of apparent transport of ozone from the Indian Ocean to the Equatorial Atlantic Ocean. Most interesting are examples that coincide with the INDOEX observations of late northern winter, 1999. Three soundings associated with the SHADOZ (Southern Hemisphere Additional Ozonesondes) network help confirm and quantify degree of influence of pollution, lightning, and stratospheric sources, suggesting that perhaps 40% of increased Atlantic ozone could be Asian pollution during periods of maximum identified in the TTO maps. We outline recurrent periods of apparent ozone transport from Indian to Atlantic Ocean regions both during and outside the late-winter period. These are placed in the context of some general observations about factors controlling recurrence timescales for the expression of both equatorial and subtropical plumes. Low-level subtropical plumes are often controlled by frontal systems approaching the Namib coast; these direct mid-level air into either easterly equatorial plumes or westerly mid- troposphere plumes. Equatorial plumes of ozone cross Africa on an easterly path due to the occasional coincidence of two phenomena: (1) lofting of ozone to mid and upper levels, often in the Western Indian Ocean, and (2) the eastward extension of an Equatorial African easterly jet.
Chatfield, Robert B.↗
Nonlinear functional approximation with networks using adaptive neurons
A novel mathematical framework for the rapid learning of nonlinear mappings and topological transformations is presented. It is based on allowing the neuron's parameters to adapt as a function of learning. This fully recurrent adaptive neuron model (ANM) has been successfully applied to complex nonlinear function approximation problems such as the highly degenerate inverse kinematics problem in robotics.
Tawel, Raoul↗