Factors influencing convergence of a learning system
Measurement rays affecting algorithm for learning control system
Zajac, G. E.↗
SEARCH · Search NASA
Results for “learning control”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
An improved algorithm for learning systems
Algorithm for implementing learning controller based on subgoal concept applicable to linear stationary system
Lindorff, D. P.↗
System parameter adaptation via a learning procedure
The paper considers the problem of designing a learning control system (LCS) that is capable of meeting design requirements over many possible operating conditions of the plant by adjusting, in a prescribed manner, the feedforward and feedback gains of the plant. The approach utilizes the best features of two methods, gain scheduling and adaptive control. The LCS was implemented using two models, one representing the longitudinal dynamics and the other the lateral dynamics of the simulated plant. A block diagram illustrating the functional organization of an LCS is presented.
Mekel, R.↗
System identification in the repetition domain
Procedures for system identification using realization theory in conjunction with learning control ideas are developed. The Markov parameters of the system are identified by combining data from repeated experiments. Three approaches are discussed for identification of as many Markov parameters as sample points in the experiment. Making use of all the parameters, realization theory is then employed to determine the system order and to obtain a minimal order representation. The first two approaches are non-recursive, which in the case of noise-free data yields a one step solution. The third approach uses a recursive formulation rendered from adaptive control but modified for successive experiments. A simple example shows the numerical convergence of the identified parameters as a function of the number of experiments. The procedure presented herein is an extension of the existing Eigensystem Realization Algorithm (ERA), which has been successfully applied for system identification of large structures.
Juang, Jer-Nan↗
Search-based model identification of smart-structure damage
This paper describes the use of a combined model and parameter identification approach, based on modal analysis and artificial intelligence (AI) techniques, for identifying damage or flaws in a rotating truss structure incorporating embedded piezoceramic sensors. This smart structure example is representative of a class of structures commonly found in aerospace systems and next generation space structures. Artificial intelligence techniques of classification, heuristic search, and an object-oriented knowledge base are used in an AI-based model identification approach. A finite model space is classified into a search tree, over which a variant of best-first search is used to identify the model whose stored response most closely matches that of the input. Newly-encountered models can be incorporated into the model space. This adaptativeness demonstrates the potential for learning control. Following this output-error model identification, numerical parameter identification is used to further refine the identified model. Given the rotating truss example in this paper, noisy data corresponding to various damage configurations are input to both this approach and a conventional parameter identification method. The combination of the AI-based model identification with parameter identification is shown to lead to smaller parameter corrections than required by the use of parameter identification alone.
Glass, B. J.↗
A Rideshare Tensegrity Rover Concept to Explore Titan’s Lands and Oceans
The Tension Adjustable Network for Deploying Entry Membrane (TANDEM) concept is a phase II NASA Innovative Advanced Concepts (NIAC) project. One of the goals for this project was to extend the application of tensegrity-based rovers to a rideshare concept for a Titan mission. This report provides preliminary details of the TANDEM concept for Titan. System analyses of the entry vehicle were performed and compared to data from the Huygens mission to Titan. The entry aeroshell was based on Huygens and Stardust aeroshells. Modeling, simulation and design of the tensegrity deployment were performed using a nonlinear optimization form-finding algorithm. Lastly, a neuroevolution-based machine learning control strategy was applied, which produced efficient rolling locomotion gaits.
tensegrity↗
SoMoGym: A Toolkit for Developing and Evaluating Controllers and Reinforcement Learning Algorithms for Soft Robots
Soft robotsoffer a host of benefits over traditional rigid robots, including inherent compliance that lets them passively adapt to variable environments and operate safely around humans and fragile objects. However, that same compliance makes it hard to use model-based methods in planning tasks requiring high precision or complex actuation sequences. Reinforcement learning (RL) can potentially find effective control policies, but training RL using physical soft robots is often infeasible, and training using simulations has had a high barrier to adoption. To accelerate research in control and RL for soft robotic systems, we introduce SoMoGym ( So ft Mo tion Gym ), a software toolkit that facilitates training and evaluating controllers for continuum robots. SoMoGym provides a set of benchmark tasks in which soft robots interact with various objects and environments. It allows evaluation of performance on these tasks for controllers of interest, and enables the use of RL to generate new controllers. Custom environments and robots can likewise be added easily. We provide and evaluate baseline RL policies for each of the benchmark tasks. These results show that SoMoGym enables the use of RL for continuum robots, a class of robots not covered by existing benchmarks, giving them the capability to autonomously solve tasks that were previously unattainable.
Moritz A. Graule↗
Discovery of False Data Injection Attacks on Power Grid Frequency Controllers with Reinforcement Learning [Poster]
While inverter-based DER (distributed energy resources) are instrumental to integrating renewable energy into the power grid, they reduce the grid's mechanical inertia, thereby increasing the risk of frequency instabilities. To compensate for frequency instability risks, the grid must also undergo a transformation to include digital technologies that allow for two-way communication between the utility and customers. The current and future state of the power grid allows for building a cleaner energy landscape. However, the grid may also become vulnerable to novel cyber threats. To preemptively protect the power grid against elaborate cyber-attacks, we propose to discover potential threats via reinforcement learning. In this work, the focus is on studying false data injection attacks that target the control logic of frequency controllers. We show that a reinforcement learning agent can successfully discover how to best inject false data into linear droop controllers.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Constraints on OPF Surrogates for Learning Stable Local Volt/Var Controllers
We consider the problem of learning local Volt/Var controllers in distribution grids (DGs). Our approach starts from learning separable surrogates that take both local voltages and reactive powers as arguments and predict the reactive power setpoints that approximate optimal power flow (OPF) solutions. We propose an incremental control algorithm and identify two different sets of slope conditions on the local surrogates such that the network is collectively steered toward desired configurations asymptotically. Our results reveal the trade-offs between each set of conditions, with coupled voltage-power slope constraints allowing an arbitrary shape of surrogate functions but risking limitations on exploiting generation capabilities, and reactive power slope constraints taking full advantage of generation capabilities but constraining the shape of surrogate functions. AC power flow simulations on the IEEE 37-bus feeder illustrate their guaranteed stability properties and respective advantages in two DG scenarios.
asymptotic stability↗
Hybrid-RL-MPC4CLR (Hybird-Reinforcement-Learning-Model-Predictive-Control-for-Reserve-Policy-Assisted-Critical-Load-Restoration-in-Distribution-Grids)
Hybrid-RL-MPC4CLR was developed as a hybrid controller for active distribution grid critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. The RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while the MPC models grid operations incorporating the RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. The developers formulated the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The software is developed using various software packages in Python. The MPC's optimal power flow (OPF) model is implemented using the Pyomo package, the RL simulation environment is implemented using the MPC simulation with various scenarios of renewable energy and load demand profiles and power outage beginning times, based on the OpenAI Gym framework. The RL agent training is performed using the RLlib Ray package. The RL algorithm is trained offline using historical forecasts of renewable generation and load demand profiles. Simulation analysis and performance tests are conducted using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery.
Eseye, Abinet Tesfaye↗
Methodology for interpretable reinforcement learning model for HVAC energy control
Deep reinforcement learning (DRL) approaches have been used in various application areas to improve efficiency, optimization, or automation. However, very little is known about how the DRL algorithms make decisions and what features affect their performance. Using a case study of a DRL based Heating, Ventilation and Air Conditioning (HVAC) optimization methodology, we demonstrate how we can address these challenges by applying interpretability tools and systematically exploring the model inputs for better understanding the DRL behaviour and decision making process. We developed a methodology for interpretable reinforcement learning and evaluated our approach in real-world house located in Knoxville, TN. Our findings explain the reasoning behind DRL-based optimization decisions under different circumstances which has been discussed and confirmed by the experts in the field.
Kotevska, Olivera↗
Deep reinforcement learning for optimal control of induction welding process
Optimizing induction welding (IW) process parameters for the application of joining thermoplastic composites is challenging as it requires achieving complex spatiotemporal thermal characteristics along the weld-line to obtain desired weld quality. We formulate an optimal control problem which captures these requirements and seeks to optimize the IW coil speed using a fast-acting dynamic IW process model. We develop a novel Deep Reinforcement Learning (DRL) framework to solve this computationally challenging control problem and demonstrate via simulation study that the learned DRL feedback control policy results in better spatiotemporal thermal characteristics as compared to the current state-of-the-art.
36 MATERIALS SCIENCE↗
Machine Learning-Based Process Control for Injection Molding of Recycled Polypropylene
The increased interest in artificial intelligence in manufacturing has driven the adoption of machine learning to optimize processes and improve efficiency. A key challenge in injection molding is the variability of recycled materials, which affects part quality and processing stability. This study presents a novel closed-loop process control approach for injection molding, leveraging machine learning to adaptively predict processing inputs and quality outcomes. The methodology was tested on five blends of recycled polypropylene (rPP), using artificial neural networks (ANNs), linear regression, and polynomial regression to model the relationships between material properties and process parameters. The dataset was split 80/20 into training and testing sets. The ANN model was implemented using TensorFlow and Keras, with six hidden layers of 32 neurons per layer, ReLU activation, and an Adam optimizer. Empirical tuning and early stopping were used to optimize performance and prevent overfitting. Predictions were evaluated based on mean absolute error (MAE), mean squared error (MSE), and percentage error. The results showed that yield stress, ultimate elongation, and part weight were accurately predicted within a 5% error for linear and polynomial regression models and within a 10% error for the ANN. However, modulus predictions were less reliable, with errors of ~11% for ANN and linear regression and ~40% for polynomial regression, reflecting the inherent variability of this property in rPP blends. Predictions of processing inputs had errors ranging from 3% to 25%, depending on the model and response variable. No single modeling approach was consistently superior across all responses, highlighting the complexity of the relationship between material properties, process parameters, and quality metrics. Overall, the work demonstrates that closed-loop process control, powered by machine learning, can effectively predict key quality parameters in injection molding of recycled materials. The proposed approach can improve process stability and material utilization, facilitating increased adoption of sustainable materials.
Krantz, Joshua↗
Reducing Urban Traffic Congestion Using Deep Learning and Model Predictive Control
This article proposes a deep learning (DL)-based control algorithm—DL velocity-based model predictive control (VMPC)—for reducing traffic congestion with slowly time-varying traffic signal controls. This control algorithm consists of system identification using DL and traffic signal control using VMPC. For the training process of DL, we established a modeling error entropy loss as the criteria inspired by the theory of stochastic distribution control (SDC) originated by the fourth author. Herein, simulation results show that the proposed algorithm can reduce traffic congestion with a slowly varying traffic signal control input. Results of an ablation study demonstrate that this algorithm compares favorably to other model-based controllers in terms of prediction error, signal varying speed, and control effectiveness.
97 MATHEMATICS AND COMPUTING↗
Encrypted Control Using Modified Learning With Errors-based Schemes
Cyber-physical systems (CPSs) require reliable, safe, and secure control of critical infrastructure, combining computational and networking capabilities, which heighten the risk of cyber attacks. These attacks can disrupt the physical process, causing unforeseen consequences. One solution is the use of fully homomorphic encryption (FHE) to protect the control loop, allowing for secure computations and communications without compromising signal and control system privacy. The challenge with FHE, however, is its requirement for inputs to be integers. This paper introduces a modified Learning With Errors (LWE) FHE approach that encodes control system dynamics and signals into integers. Our proposed scheme leverages a generalized LWE encoding function and modifies the Gentry-Sahai-Waters (GSW) gadget decomposition tool to encrypt the control system. Using the modified LWE scheme, we formalize a fully encrypted control system, supported by simulated results.
42 - ENGINEERING↗
Development of algorithms for augmenting and replacing conventional process control using reinforcement learning
Here, this work seeks to allow for the online operation and training of model-free reinforcement learning (RL) agents but limit the risk to system equipment and personnel. The parallel implementation of RL alongside more conventional process control (CPC) allows for the RL algorithm to learn from CPC. The past performance of both methods are assessed on a continuous basis allowing for a transition from CPC to RL and, if needed, transitioning back to CPC from RL. This allows for the RL algorithm to slowly and safely assume control of the process without significant degradation in control performance. It is shown that the RL can derive a near optimal policy even when coupled with a suboptimal CPC. It is also demonstrated that the coupled RL-CPC algorithm learns at a faster rate than traditional RL methods of exploration while the algorithm’s performance does not deteriorate below CPC, even when exposed to an unknown operating condition.
30 DIRECT ENERGY CONVERSION↗
Various Embodiments of the Non-Invasive Endoscopic Feedback for Learning of Voluntary Control of Physiological Functioning
The research performed was a small portion of the patent to be submitted by Dr. Alan T. Pope entitled 'A Method of Providing Veridical Non-Invasive Endoscopic Feedback for Learning of Voluntary Control of Physiological Functioning'. The focus of this study is to incorporate the emerging technology of virtual reality with the forms of biofeedback already in existance producing a life-like, real-time model of the body's functioning without using invasive procedures, yet still producing the equivalent of a picture from an invasive endoscopic procedure in the region of interest. The portion of the project designated to me was to research and report as many possible uses for such technology as possible.
Henriksen, Mina D.↗
Anomaly Detection and Mitigation for Wide-Area Damping Control using Machine Learning
In an interconnected multi-area power system, wide-area measurement based damping controllers are used to damp out inter-area oscillations, which jeopardize grid stability and constrain the power flows below to their transmission capacity. The effect of wide-area damping control (WADC) significantly depends on both power and cyber systems. At the cyber system layer, an adversary can inflict the WADC process by compromising either measurement signals, control signals or both. Stealthy and coordinated cyber-attacks may bypass the conventional cybersecurity measures to disrupt the seamless operation of WADC. This paper proposes an anomaly detection (AD) algorithm using supervised Machine Learning and a model-based logic for mitigation. The proposed AD algorithm considers measurement signals (input of WADC) and control signals (output of WADC) as input to evaluate the type of activity such as normal, perturbation (small or large signal faults), attack and perturbation-and-attack. Upon anomaly detection, the mitigation module tunes the WADC signal and sets the control status mode as either wide-area mode or local mode. The proposed anomaly detection and mitigation (ADM) module works inline with the WADC at the control center for attack detection on both measurement and control signals and eliminates the need for ADMs at the geographically distributed actuators. Here, we consider coordinated and primitive data-integrity attack vectors such as pulse, ramp, relay-trip and replay attacks. The performance of the proposed ADM algorithms was evaluated under these attack vector scenarios on a testbed environment for 2-area 4-machine power system. The ADM module shows effective performance with 96:5% accuracy to detect anomalies.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗