Search NASA⌕ Search

SEARCH · Search NASA

Results for “learning based systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Deep reinforcement learning of cell movement in the early stage of C.elegans embryogenesis

Motivation: Cell movement in the early phase of Caenorhabditis elegans development is regulated by a highly complex process in which a set of rules and connections are formulated at distinct scales. Previous efforts have demonstrated that agent-based, multi-scale modeling systems can integrate physical and biological rules and provide new avenues to study developmental systems. However, the application of these systems to model cell movement is still challenging and requires a comprehensive understanding of regulatory networks at the right scales. Recent developments in deep learning and reinforcement learning provide an unprecedented opportunity to explore cell movement using 3D time-lapse microscopy images. Results: We present a deep reinforcement learning approach within an agent-based modeling system to characterize cell movement in the embryonic development of C.elegans. Our modeling system captures the complexity of cell movement patterns in the embryo and overcomes the local optimization problem encountered by traditional rule-based, agent-based modeling that uses greedy algorithms. Here, we tested our model with two real developmental processes: the anterior movement of the Cpaaa cell via intercalation and the rearrangement of the superficial left-right asymmetry. In the first case, the model results suggested that Cpaaa’s intercalation is an active directional cell movement caused by the continuous effects from a longer distance (farther than the length of two adjacent cells), as opposed to a passive movement caused by neighbor cell movements. In the second case, a leader-follower mechanism well explained the collective cell movement pattern in the asymmetry rearrangement. These results showed that our approach to introduce deep reinforcement learning into agent-based modeling can test regulatory mechanisms by exploring cell migration paths in a reverse engineering perspective. Lastly, this model opens new doors to explore the large datasets generated by live imaging. Availability and implementation: Source code is available at https://github.com/zwang84/drl4cellmovement.

59 BASIC BIOLOGICAL SCIENCES↗

A comparison of machine learning methods to classify radioactive elements using prompt-gamma-ray neutron activation data

The detection of illicit radiological materials is critical to establishing a robust second line of defence in nuclear security. Neutron-capture prompt-gamma activation analysis (PGAA) can be used to detect multiple radioactive materials across the entire Periodic Table. However, long detection times and a high rate of false positives pose a significant hindrance in the deployment of PGAA-based systems to identify the presence of illicit substances in nuclear forensics. In the present work, six different machine-learning algorithms were developed to classify radioactive elements based on the PGAA energy spectra. The model performance was evaluated using standard classification metrics and trend curves with an emphasis on comparing the effectiveness of algorithms that are best suited for classifying imbalanced datasets. We analyse the classification performance based on Precision, Recall, F1-score, Specificity, Confusion matrix, ROC-AUC curves, and Geometric Mean Score (GMS) measures. The tree-based algorithms (Decision Trees, Random Forest and AdaBoost) have consistently outperformed Support Vector Machine and K-Nearest Neighbours. Based on the results presented, AdaBoost is the preferred classifier to analyse data containing PGAA spectral information due to the high recall and minimal false negatives reported in the minority class.

97 MATHEMATICS AND COMPUTING↗

Prediction of vacancy defect diffusion paths in high entropy alloys via machine learning on molecular dynamics data

Identifying the diffusion path of point defects is a critical step in understanding their evolution and the mechanisms of related phenomena. Defect diffusion occurs at small length and time scales, with impacts on material properties that may continue to evolve over ns to μs, ms, and the continuum scale (s, min, etc., and cm, m, etc.). The time scale accessible to molecular dynamics (MD) simulations is limited by small step sizes, typically in the fs range. Thus, surrogate models of MD simulations through machine learning (ML)-based algorithms are of great interest, especially for complex systems such as high entropy alloys (HEAs). In this work, dynamics governing vacancy migration in HEA were approximated with graph convolutional network (GCN) models as ansatzes for kinetic Monte Carlo (KMC) rate catalogs. Network design considered that diffusion in crystalline solids generally depends on interactions between defects and their immediate neighbor atoms. Graphs represented the vacancy surroundings, MD-generated trajectories provided training and comparison datasets, and unsupervised GCN models approximated interatomic dynamics governing vacancy migration in HEAs as ansatzes for KMC. A proof-of-concept model trained on MD data for the Fe, Ni, Cr, Co, and Cu HEA environment was used with two different neighbor interactions to assess the feasibility of training a GCN to predict vacancy defect transition rates in the HEA environment. The resulting setup rapidly generated MD-formatted synthetic trajectories based on dynamics learned from the MD training set, with a time acceleration of roughly two orders of magnitude and a similar diffusion coefficient to MD observations. Additionally, Nudged Elastic Band (NEB) calculations were performed on randomly generated FeNiCrCoCu HEA structures to determine vacancy migration barriers across nearest-neighbor sites. Transition probabilities for each jump, categorized by atomic type, were extracted from these calculations. NEB-based and GCN-based approaches led to similar outcomes.

Reimer, C↗

Dynamic Parameter Estimation with Physics-based Neural Ordinary Differential Equations

Accurate estimation of dynamic parameters of gen-erators is crucial to building a reliable model for dynamical studies and reliable operation of the power system. This paper develops a physics-based neural ordinary differential equations (ODE) approach to learn the parameters of generator dynamic model using phasor measurement units (PMU) data. We design a physics-based neural network to represent the swing equations of the power system dynamics. A loss function is defined as the difference between dynamic simulation results from the physics-based neural networks and pseudo PMU measurements. The parameters of generator dynamic model are iteratively updated using the neural ODEs and the adjoint method. By exploiting the mini-batch scheme in neural ODE training, the parameter estimation performance is significantly improved. Numerical study results on a 3-machine 9-bus system show that the proposed algorithm outperforms state-of-the-art baseline method in both computation time and dynamic parameter estimation accuracy.

24 POWER TRANSMISSION AND DISTRIBUTION↗

PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control

This paper develops an efficient multi-agent deep reinforcement learning algorithm for cooperative controls in powergrids. Specifically, we consider the decentralized inverter-based secondary voltage control problem in distributed generators (DGs), which is first formulated as a cooperative multi-agent reinforcement learning (MARL) problem. We then propose a novel on-policy MARL algorithm, PowerNet, in which each agent (DG) learns a control policy based on (sub-)global reward but local states and encoded communication messages from its neighbors. Motivated by the fact that a local control from one agent has limited impact on agents distant from it, we exploit a novel spatial discount factor to reduce the effect from remote agents, to expedite the training process and improve scalability. Furthermore, a differentiable, learning-based communication protocol is employed to foster the collaborations among neighboring agents. In addition, to mitigate the effects of system uncertainty and random noise introduced during on-policy learning, we utilize an action smoothing factor to stabilize the policy execution. To facilitate training and evaluation, we develop PGSim, an efficient, high-fidelity powergrid simulation platform. Here, experimental results in two microgrid setups show that the developed PowerNet outperforms the conventional model-based control method, as well as several state-of-the-art MARL algorithms. The decentralized learning scheme and high sample efficiency also make it viable to large-scale power grids.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Using Machine Learning to Predict Future Temperature Outputs in Geothermal Systems

Optimizing the power output, and economic value, of geothermal power plants over decades of operation is a major challenge in renewable energy. Optimizing the output requires the ability to predict the mass flow rates and the output temperatures of production wells based on the inputs of injection wells, as well as the time history of the system. Machine Learning (ML) that incorporates the known physics of geothermal systems is one possible solution to this challenge. In this work, we explore the ability of ML algorithms to predict future temperature outputs based on historical data. Considering the challenges with obtaining an empirical dataset from field data that is large enough to enable reliable ML, we propose an alternate approach: developing a high-fidelity reservoir model and using computational resources to build a dataset that enables ML. As a first step towards achieving this goal, we present preliminary results from applying ML to predict the temperature timeseries of simple modeled geothermal systems. We describe the application of relevant state-of-the-art ML approaches, such as the Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNN), to extract temporal structures in the model data. We assess the accuracy of the forecasts we obtain, compare the selected approaches, and share the lessons learned that would inform the process of training and utilizing ML algorithms for larger and more complex geothermal systems.

GEOTHERMAL ENERGY↗

Implementation Aspects of Smart Grids Cyber-Security Cross-Layered Framework for Critical Infrastructure Operation

Communication networks in power systems are a major part of the smart grid paradigm. It enables and facilitates the automation of power grid operation as well as self-healing in contingencies. Such dependencies on communication networks, though, create a roam for cyber-threats. An adversary can launch an attack on the communication network, which in turn reflects on power grid operation. Attacks could be in the form of false data injection into system measurements, flooding the communication channels with unnecessary data, or intercepting messages. Using machine learning-based processing on data gathered from communication networks and the power grid is a promising solution for detecting cyber threats. In this paper, a co-simulation of cyber-security for cross-layer strategy is presented. The advantage of such a framework is the augmentation of valuable data that enhances the detection as well as identification of anomalies in the operation of the power grid. The framework is implemented on the IEEE 118-bus system. The system is constructed in Mininet to simulate a communication network and obtain data for analysis. A distributed three controller software-defined networking (SDN) framework is proposed that utilizes the Open Network Operating System (ONOS) cluster. According to the findings of our suggested architecture, it outperforms a single SDN controller framework by a factor of more than ten times the throughput. This provides for a higher flow of data throughout the network while decreasing congestion caused by a single controller’s processing restrictions. Furthermore, our CECD-AS approach outperforms state-of-the-art physics and machine learning-based techniques in terms of attack classification. The performance of the framework is investigated under various types of communication attacks.

cross-layered↗

Learning Interactions in Complex Biological Systems

In this project, we developed novel methods for integrating simulation and data-driven methods to accelerate biomedical discovery. In particular, we exploit agent-based models of disease processes and deep reinforcement learning to identify potential novel multi-drug, patient-specific, and adaptive therapeutic strategies. We use the Innate Immune Response agent-based model (IIRABM) to model sepsis as a demonstration of our approach. We first calibrate the agent-based model using novel stochastic optimization algorithms develop in this project, then use the calibrated model to identify optimal cytokine mediation strategies using deep reinforcement learning (DRL). The learned policy achieves a dramatic reduction in mortality over a set of 500 simulated patients relative to standalone antibiotic therapy. Advantages of our approach are three-fold: 1) the use of simulation allows exploring therapeutic strategies beyond clinical practice and available data, 2) advances in DRL accommodate learning complex therapeutic strategies for complex biological systems, and 3) optimized treatments respond to a patient’s individual disease progression over time, therefore capturing both differences across patients and the inherent randomness of disease progression within a single patient. We hope that this work motivates both considering adaptive, personalized multi-cytokine mediation therapy for sepsis and exploiting simulation with DRL for precision medicine more broadly.

59 BASIC BIOLOGICAL SCIENCES↗

Time Dilated Bundt Cake Analysis of PV Output [Poster]

We present a novel method for modeling time-dependent statistics in the power signal generated by a photovoltaic (PV) system. Our white-box machine learning method is interpretable and auditable, based on principles of multiperiodic basis functions and convex optimization. Our proposed method of time dilating the daily signal to remove night time values results in a novel representation of PV power signals, evocative of a ‘Bundt cake’. The proposed model describes the marginal distribution of power output as a function of date and time. The resulting probabilistic model of a PV system can be used to perform a variety of tasks, and here, we demonstrate the application of clear sky detection.

14 SOLAR ENERGY↗

Intelligent Leak Detection System

apability of underground carbon dioxide storage to confine and sustain injected CO2 for a very long time is the main concern for geologic CO2 sequestration. If a leakage from a geological CO2 sequestration site occurs, it is crucial to find the approximate amount and the location of the leak in order to implement proper remediation activity. An overwhelming majority of research and development for storage site monitoring has been concentrated on atmospheric, surface or near surface monitoring of the sequestered CO2. This study aims to monitor the integrity of CO2 storage at the reservoir level. This work proposes developing in-situ CO2 Monitoring and Verification technology based on the implementation of Permanent Down-hole Gauges (PDG) or “Smart Wells” along with Artificial Intelligence and Data Mining (AI&DM). The technology attempts to identify the characteristics of the CO2 leakage by de-convolving the pressure signals collected from Permanent Down-hole Gauges (PDG). Citronelle field, a saline aquifer reservoir, located in the U.S. was considered for this study. A reservoir simulation model for CO2 sequestration in the Citronelle field was developed and history matched. The presence of the PDGs were considered in the reservoir model at the injection well and an observation well. High frequency pressure data from sensors were collected based on different synthetic CO2 leakage scenarios in the model. Due to complexity of the pressure signal behaviors, a Machine Learning-based technology was introduced to build an Intelligent Leakage Detection System (ILDS). The ILDS was able to detect leakage characteristics in a short period of time (less than a day) demonstrating the capability of the system in quantifying leakage characteristics subject to complex rate behaviors. The performance of ILDS was examined under different conditions such as multiple well leakages, cap rock leakage, availability of an additional monitoring well, presence of pressure drift and noise in the pressure sensor and uncertainty in the reservoir model.

Mohaghegh, Shahab D.↗

Leveraging System Dynamics to Predict the Commercialization Success of Emerging Energy Technologies: Lessons from Wind Energy

The United States urgently needs to tackle the climate crisis while enhancing energy security and resiliency. The complexity of the U.S. energy system, with its interconnected elements, makes predicting future states challenging, especially with the introduction of novel energy systems like wind, solar, clean hydrogen, and advanced nuclear technologies. Modern systems engineering methods and tools can provide deeper insights into these dynamics and future behaviors. This research aims to develop a comprehensive model that captures the main elements and behaviors of new energy technologies within the existing energy system. We hypothesized that the market uptake of novel energy systems is influenced by multiple diverse factors, such as technological learning, availability of resources, and economic incentives; examined the history of electricity generation using land-based wind technologies; and developed a system dynamics model to investigate the relationships between capacity growth and influencing factors, both internal and external. The developed model yielded outcomes that confirmed the hypothesized dynamics of wind energy system diffusion through a quantitative comparison of installed capacity and highlighted the significant influence of resource availability, federal incentives (production tax credits), and technological learning on capacity growth and cost reduction. This research aims to support informed decision-making for investments in novel energy systems and aid in developing effective policies for technology deployment.

17 WIND ENERGY↗

Deep Reinforcement Learning Based Control of Wind Turbines for Fast Frequency Response

In order to fulfill vital auxiliary grid services, such as load regulation, spin and non-spin reserve provision, and frequency support during emergencies, there is often a requirement for certain wind farms to operate in de-loaded modes. Leveraging the swift response capabilities of wind farms, this study demonstrates that reserving power in de-loaded modes can significantly enhance power grid stability and reliability during system contingencies. Controlling wind farms optimally for frequency support is intricate due to the nonlinearity of models and controllers and the complexity of wind farm interactions with power systems. Here, to address this challenge, this paper introduces a novel approach that integrates wind turbines into reinforcement learning-based solutions for frequency response. This innovative methodology utilizes the state-of-the-art reinforcement learning algorithm known as the surrogate-gradient-based evolutionary strategy. The proposed learning-based algorithm provides continuous control of wind farm output to rapidly stabilize system frequency and prevent unnecessary trips of under-frequency load shedding relays. To facilitate efficient training, parallel computing techniques are employed. The proposed methodology is evaluated on a modified IEEE-39 bus system, and simulation results reveal its efficacy in reliably supporting power system frequency and preventing the need for unnecessary load shedding.

Gao, Wei [Argonne National Laboratory (ANL), Argon↗

Light Water Reactor Sustainability Program Support and Modeling for the Boiling Water Reactor Station Black Out Case Study Using RELAP and RAVEN

The existing fleet of nuclear power plants is in the process of extending its lifetime and increasing the power generated. In order to evaluate the impact of these two factors on the safety of the plant, the Risk Informed Safety Margin Characterization (RISMC) project aims to provide insight to decision makers through a series of simulations of the plant dynamics for different initial conditions (e.g., probabilistic analysis and uncertainty quantification). This report focuses, in particular, on the impact of power uprate on the safety of a boiled water reactor system. The case study considered is a loss of off-site power followed by the loss of diesel generators, i.e., a station black out (SBO) event. Analysis is performed by using a thermo-hydraulic code, i.e. RELAP-5, and a stochastic analysis tool currently under development at INL, i.e. RAVEN. Starting from the event tree models contained in SAPHIRE, we built the input file for RELAP-5 that models in great detail system dynamics under SBO conditions. We also interfaced RAVEN with RELAP-5 so that it would be possible to run multiple RELAP-5 simulation runs by changing specific keywords of the input file. We both employed classical statistical tools, i.e. Monte-Carlo, and more advanced machine learning based algorithms to perform uncertainty quantification in order to quantify changes in system performance and limitations as a consequence of power uprate. We also employed advanced data analysis and visualization tools that helped us to correlate simulation outcome such as maximum core temperature with a set of input uncertain parameters. Results obtained gave a detailed overview of the issues associated to power uprate for a SBO accident scenario. We were able to quantify how timing of safety related events were impacted by a higher reactor core power. Such insights can provide useful material to the decision makers to perform risk-infomed safety margins management.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Data Driven Temperature Control of Heat Exchangers

The following poster presents a couple of approaches for temperature control at heat exchanger’s exit. One of the approaches is based on using n-step ahead neural network while the other approach utilizes a combination of control + learning module for temperature control. Designing robust controllers for typical heat exchangers is very challenging primarily because of complexities associated with the synthesis of dynamics of industrial heat exchangers. Even if a dynamics model is synthesized, the accuracy of it would be called into question as there can be lot of unmodeled system dynamics and random noise in input parameters. In many situations, the degree of uncertainty in the model of system being controlled limits the utility of optimal control design. Controller can be manually tuned on the field but it is very difficult to determine manual adjustments that result in overall improvement. These practical challenges call for learning based approaches for temperature control that would result in optimal heat exchanger performance. We utilize the existing mathematical model for air-water system synthesized by Underwood and Crawford for testing our algorithms. Underwood and Crawford developed a model of heating coil by fitting a second-order non-linear equations to measurements of air and water temperature and flow rates obtained from the actual coil. The uncertainties in the model include air and water inlet temperatures and air-flow rate. These uncertainties are modeled as random variables that are sampled from a uniform distribution. The control variable is the water flow rate which is controlled for regulating the air-outlet temperature. Neural networks are frequently used in various literatures for learning time-series. We propose a modified version of neural networks known as n-step ahead neural network that take the state variables and output at time t as input to successfully predict output at time t+n. The variable n is a hyper-parameter which can be optimized based on performance constraints. The n-step ahead neural network was then trained on training data generated from experimental model to high training accuracies. This trained model performed appreciably in its objective to regulate outlet temperature of a heat exchanger. The second approach uses a combination of control and learning module for temperature regulation. The control module used is a proportional controller while the learning module used is steady state neural network. The steady state neural network is used to learn the controller output as a function of random variables and temperature set-point. The proportional controller is then used to compensate for transient behavior of linearized system around various operating points. The algorithm works effectively for temperature regulation as neural network captures the non-linear input-output behavior of closed loop system leaving proportional controller to compensate for steady state errors of a linearized plant. The proposed algorithm’s performances were evaluated against the performance of a PI controller. Further, the robustness of the algorithms were also studied in great detail under small perturbations around tuned parameters of the proposed algorithms.

Sundar, Sreenath↗

Knowledge-Informed Uncertainty-Aware Machine Learning for Time Series Forecasting of Dynamical Engineered Systems

The high complexity and multiscale nature of many engineered systems—such as those in nuclear power plants—make representing and forecasting their dynamic behavior challenging. Physics-based models can be overly complex and computationally intractable, whereas machine learning (ML) tools are often data-hungry and prone to unphysical solutions. This study proposes a knowledge-informed ML-aided hybrid residual modeling approach that offers accurate and efficient time series forecasting for the operation of dynamical engineered systems. Hybrid residual modeling entails a baseline solution from domain knowledge and known physics expressions about the system dynamics integrated with an ML model to capture undiscovered information from the mismatch (i.e., residuals) between true states from measurements and baseline-predicted outputs. This study further quantifies the ML model uncertainty to provide trustworthy solutions. Real-time operational data from thermal-hydraulic flow loops of the cryogenic moderator system in Oak Ridge National Laboratory’s Spallation Neutron Source facility were used to demonstrate the potential of knowledge-informed uncertainty-aware ML in real-world applications. The state variables of the cryogenic helium loop were modeled with (1) first principles–based system identification (sysID), (2) long short-term memory (LSTM) neural network, and (3) hybrid sysID (baseline) + LSTM (residual). The superior predictive capability of the sysID+LSTM model versus stand-alone sysID and LSTM is confirmed by average performance metrics and individual data points across different prediction horizons. By creating a robust representation of the underlying physical system, the widely applicable hybrid residual modeling approach will enable the future development of digital twins for performance prediction, prognostics, and operation control.

Zhao, Xingang↗

Potential of deep learning methods to enhance satellite-based monitoring of nuclear power plants focusing on remote operation evaluations

The anticipated expansion of the nuclear industry and the deployment of new nuclear reactors (200 + GW of new nuclear capacity by 2050) require the development of monitoring systems that align with safety and security concerns, providing enhanced evaluation capabilities. A remote monitoring system using satellites and deep learning techniques was evaluated for its ability to detect anomalies and capture various features of nuclear reactors independently of the conditions on the ground. Satellite images of current operational and under-construction nuclear power plants were collected from Google Earth Pro as a surrogate database. Subsequently, five datasets were created from the collected images. Transfer learning technique was used for several classification tasks utilizing VGG16, ResNet50V2, Xception, DenseNet121, and MobileNetV2 pre-trained models. In the first task, the capability of the monitoring system to detect abnormal conditions or processes in a nuclear power plant was investigated. In the second task, the ability to capture operational features remotely was examined. As an example, for the purposes of this study, these features included classifying reactors based on type, power range, or onsite condition. Several evaluation metrics were used to compare the performance of the pre-trained models and the overall monitoring system. Here, the evaluation results demonstrated that deep learning techniques and pre-trained models applied to satellite images have the potential to facilitate further and expand capabilities in monitoring systems to assess plant operation details.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

42 ENGINEERING↗