A theory and model of human learning behavior in a manual control task
Human learning behavior theory in manual control task modeled as computer program
Preyss, A. E.↗
SEARCH · Search NASA
Results for “learning control”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Data-driven predictive probability density function control of fiber length stochastic distribution shaping in refining process
Pulp is the most important raw material in paper in paper industries, whose Fiber Length Stochastic Distribution (FLSD) shaping directly determines the energy consumption and paper quality of the subsequent papermaking processes. However, the mean and variance are insufficient to describe the output FLSD shaping, which display non-Gaussian distribution properties. Therefore, the traditional control, method based on the mean and variance of fiber length is difficult to control the FLSD shaping effectively. In this paper, a novel data-driven predictive probability density function (PDF) control method is proposed for output FLSD shaping in refining process. Primarily, in order to improve the approximation accuracy of the PDF employing RBF neural network, the parameters (center value and width) of each RBF basis functions are tuned via utilizing iterative learning control (ILC) law, and the corresponding estimations of weights law can be obtained. Secondly, considering that the conventional linear model of weights vector has drawbacks of low accuracy and weak generalization ability, random vector functional link networks (RVFLNs) based data driven nonlinear modelling method is employed to characterize the prediction model between the input variables and weights vector. Finally, in order to reduce the randomness of the output FLSD, a minimum entropy control method under mean constraint is employed to design the predictive PDF controller for the output FLSD shaping with the help of the established nonlinear stochastic distribution model. Both simulations and experiments are given to illustrate the effectiveness and practicability of the proposed method.
Probability debsity function, Fiber length distrib↗
Grid-Interactive Multi-Zone Building Control Using Reinforcement Learning with Global-Local Policy Search
In this paper, we develop a grid-interactive multi-zone building controller based on a deep reinforcement learning (RL) approach. The controller is designed to facilitate building operation during normal conditions and demand response events, while ensuring occupants comfort and energy efficiency. We leverage a continuous action space RL formulation, and devise a two-stage global-local RL training framework. In the first stage, a global fast policy search is performed using a gradient-free RL algorithm. In the second stage, a local fine-tuning is conducted using a policy gradient method. In contrast to the state-of-the-art model predictive control (MPC) approach, the proposed RL controller does not require complex computation during real-time operation and can adapt to nonlinear building models. We illustrate the controller performance numerically using a five-zone commercial building.
30 DIRECT ENERGY CONVERSION↗
Data-Driven Voltage Regulation of Distribution Grid Using Nonlinear Autoregressive Model with Exogenous Inputs (NARX)
This article proposes data-driven control via a nonlinear autoregressive model with exogenous inputs (NARX) for real-time voltage regulation of a modified feeder using reactive power sources. Traditional voltage control strategies rely on rule-based heuristics or optimization techniques, which often require detailed system models and extensive computational resources. The NARX-based controller learns system dynamics from historical data and predicts optimal reactive power dispatch in real-time for voltage correction. The proposed approach is evaluated on a power system feeder model under varying load and network conditions. Simulation results demonstrate that the NARX-based controller achieves improved voltage regulation, offering higher adaptability to system fluctuations. This study highlights the potential of data-driven control for enhancing the reliability of power distribution networks.
Donge, Vrushabh [ORNL] (ORCID:0000000306062803)↗
Multiresolutional schemata for unsupervised learning of autonomous robots for 3D space operation
This paper describes a novel approach to the development of a learning control system for autonomous space robot (ASR) which presents the ASR as a 'baby' -- that is, a system with no a priori knowledge of the world in which it operates, but with behavior acquisition techniques that allows it to build this knowledge from the experiences of actions within a particular environment (we will call it an Astro-baby). The learning techniques are rooted in the recursive algorithm for inductive generation of nested schemata molded from processes of early cognitive development in humans. The algorithm extracts data from the environment and by means of correlation and abduction, it creates schemata that are used for control. This system is robust enough to deal with a constantly changing environment because such changes provoke the creation of new schemata by generalizing from experiences, while still maintaining minimal computational complexity, thanks to the system's multiresolutional nature.
Lacaze, Alberto↗
CTRL-STEER: Closed-Loop Neuron Activation Control in Vision-Language-Action Models
Vision-Language-Action (VLA) models enable test-time behavioral steering via neuron-level interventions, but existing methods use fixed strengths and operate in open loop. This static modulation fails under evolving task dynamics, leading to overcorrection, oscillations, and reduced task success—especially for temporal attributes like speed. We propose CTRL-STEER, a control-theoretic framework that casts activation steering as closed-loop feedback with adaptive, time-varying interventions. Instead of assuming neurons encode temporal concepts, we steer along motion-aligned residual directions and regulate intervention magnitude via feedback. We instantiate this with both PID and reinforcement learning controllers that jointly optimize concept adherence and task success. Experiments on fine-tuned OpenVLA policies across four LIBERO suites show improved stability and a better steering–success trade-off over fixed-coefficient baselines, without retraining the base model.
Babu, Abhijith [Florida International University, ↗
Grid-Interactive Multi-Zone Building Control Using Reinforcement Learning with Global-Local Policy Search: Preprint
In this paper, we develop a grid-interactive multi-zone building controller based on a deep reinforcement learning (RL) approach. The controller is designed to facilitate building operation during normal conditions and demand response events, while ensuring occupants comfort and energy efficiency. We leverage a continuous action space RL formulation, and devise a two-stage global-local RL training framework. In the first stage, a global fast policy search is performed using a gradient-free RL algorithm. In the second stage, a local fine-tuning is conducted using a policy gradient method. In contrast to the state-of-the-art model predictive control (MPC) approach, the proposed RL controller does not require complex computation during real-time operation and can adapt to non-linear building models. We illustrate the controller performance numerically using a five-zone commercial building.
30 DIRECT ENERGY CONVERSION↗
CityLearn v2: energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communities
As more distributed energy resources become part of the demand-side infrastructure, quantifying their energy flexibility on a community scale is crucial. CityLearn v1 provided an environment for benchmarking control algorithms. However, there is no standardized environment utilizing realistic building-stock datasets for distributed energy resource control benchmarking without co-simulation or third-party frameworks. CityLearn v2 extends CityLearn v1 by providing a stand-alone simulation environment that leverages the End-Use Load Profiles for the U.S. Building Stock dataset to create grid-interactive communities for resilient, multi-agent, and objective control of distributed energy resources with dynamic occupant feedback. While the v1 environment used pre-simulated building thermal loads, the v2 environment uses data-driven thermal dynamics and eliminates the need for co-simulation with building energy performance software. This work details the v2 environment and provides application examples that use reinforcement learning control to manage battery energy storage system, vehicle-to-grid control, and thermal comfort during heat pump power modulation.
Nweye, Kingsley↗
Virtual Diagnostic Suite for Electron Beam Prediction and Control at FACET-II
We discuss the implementation of a suite of virtual diagnostics at the FACET-II facility currently under commissioning at SLAC National Accelerator Laboratory. The diagnostics will be used for the prediction of the longitudinal phase space along the linac, spectral reconstruction of the bunch profile, and non-destructive inference of transverse beam quality (emittance) while using edge radiation at the injector dogleg and bunch compressor locations. These measurements will be folded into adaptive feedbacks and Machine Learning (ML)-based reinforcement learning controls to improve the stability and optimize the performance of the machine for different experimental configurations. In this paper we describe each of these diagnostics with expected measurement results that are based on simulation data and discuss progress towards implementation in regular operations.
Emma, Claudio (ORCID:0000000247191831)↗
Sensor fusion IV: Control paradigms and data structures; Proceedings of the Meeting, Boston, MA, Nov. 12-15, 1991
Various papers on control paradigms and data structures in sensor fusion are presented. The general topics addressed include: decision models and computational methods, sensor modeling and data representation, active sensing strategies, geometric planning and visualization, task-driven sensing, motion analysis, models motivated biology and psychology, decentralized detection and distributed decision, data fusion architectures, robust estimation of shapes and features, application and implementation. Some of the individual subjects considered are: the Firefly experiment on neural networks for distributed sensor data fusion, manifold traversing as a model for learning control of autonomous robots, choice of coordinate systems for multiple sensor fusion, continuous motion using task-directed stereo vision, interactive and cooperative sensing and control for advanced teleoperation, knowledge-based imaging for terrain analysis, physical and digital simulations for IVA robotics.
Schenker, Paul S.↗
Distributed reinforcement learning and consensus control of energy systems
Disclosed herein are methods, systems, and devices for utilizing distributed reinforcement learning and consensus control to most effectively generate and utilize energy. In some embodiments, individual turbines within a wind farm may communicate to reach a consensus as to the desired yaw angle based on the wind conditions.
King, Jennifer Rose Rose↗
Combining the synergistic control capabilities of modeling and experiments: Illustration of finding a minimum-time quantum objective
A common way to manipulate a quantum system, for example spins or artificial atoms, is to use properly tailored control pulses. In order to accomplish quantum information tasks before coherence is lost, it is crucial to implement the control in the shortest possible time. Here we report the near time-optimal preparation of a Bell state with fidelity higher than 99% in an NMR experiment, which is feasible by combining the synergistic capabilities of modeling and experiments operating in tandem. In this work, the pulses preparing the Bell state are found by experiments that are recursively assisted with a gradient-based optimization algorithm working with a model. Thus, we exploit the interplay between model-based numerical optimal design and experimental-based learning control. Utilizing the balanced synergism between the dual approaches, as dictated by the case specific capabilities of each approach, should have broad applications for accelerating the search for optimal quantum controls.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
In-flight simulation investigation of rotorcraft pitch-roll cross coupling
An in-flight simulation experiment investigating the handling qualities effects of the pitch-roll cross-coupling characteristic of single-main-rotor helicopters is described. The experiment was conducted using the NASA/Army CH-47B variable stability helicopter with an explicit-model-following control system. The research is an extension of an earlier ground-based investigation conducted on the NASA Ames Research Center's Vertical Motion Simulator. The model developed for the experiment is for an unaugmented helicopter with cross-coupling implemented using physical rotor parameters. The details of converting the model from the simulation to use in flight are described. A frequency-domain comparison of the model and actual aircraft responses showing the fidelity of the in-flight simulation is described. The evaluation task was representative of nap-of-the-Earth maneuvering flight. The results indicate that task demands are important in determining allowable levels of coupling. In addition, on-axis damping characteristics influence the frequency-dependent characteristics of coupling and affect the handling qualities. Pilot technique, in terms of learned control crossfeeds, can improve performance and lower workload for particular types of coupling. The results obtained in flight corroborated the simulation results.
Watson, Douglas C.↗
Modeling and Control of an Industrial Selective Catalytic Reduction Unit with Model Predictive Control and Reinforcement Learning
Hedrick, E., K. Reynolds, V. Dwivedy, D. Bhattacharyya, S.E. Zitney, and B. Omell, "Modeling and Control of an Industrial Selective Catalytic Reduction Unit with Model Predictive Control and Reinforcement Learning," Clearwater Clean Energy Conference, Clearwater, FL, July 26-29 (2021).
Hedrick, Elijah↗
Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control
Deep reinforcement learning (DRL) algorithms have successfully solved many challenging problems in various power system control scenarios. However, their decision-making process is usually regarded as black-boxes. Furthermore, how DRL models interact with human intelligence remains an open problem. Thus, this paper proposes a policy extraction framework to extract a complex DRL model into an explainable policy. This framework includes three parts: 1) DRL training and data generation. We train an agent for a specific control task and generate data, which contains the control policy of the agent. 2) Policy extraction. We propose an information gain rate based weighted oblique decision tree (IGR-WODT) for DRL policy extraction. 3) Policy evaluation. We define three metrics to evaluate the performance of the proposed approach. A case study for the under-voltage load shedding problem shows that the IGR-WODT presents a performance enhancement compared with DRL, weighted oblique decision tree, and univariate decision tree. The proposed policy extraction method could provide an intuitive explanation of the neural network decision-making process to the dispatchers when making final decisions on power grid operation. Also, the resulted rule-based controller could replace the deep neural network-based controller in many field edge devices with limited computing resources, providing comparable performance.
deep reinforcement learning↗
MPRL (Multi-Pulse Reinforcement Learning)
MPRL contains code for defining and deploying reinforcement learning control agents for multi-pulse, multi-fuel advanced compression ignition engines.
Wimer, Nicholas↗
Securing the Clean Energy Transition with Cyber-Informed Engineering
The Bipartisan Infrastructure Law kick-started a surge in clean energy investments in the United States. Securing these systems and their supporting infrastructure from the outset is imperative, starting from the conceptual design and continuing through the entire engineering lifecycle. Cyber-Informed Engineering is an ideal approach to seamlessly integrate the necessary advanced control capabilities and adaptive protection systems, especially considering the trend towards cloud-based infrastructure and process controls. Learn more about CIE at https://www.energy.gov/ceser/articles/cyber-informed-engineering-bridge-between-cyber-and-critical-infrastructure-securing.
14 SOLAR ENERGY↗
An improved algorithm for learning systems
Algorithm for implementing learning controller
Scott, R. N.↗