SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The optimal control theory of stochastic linear systems is discussed in terms of the advantages of distributed-control systems, and the control of randomly-sampled systems. An optimal solution to longitudinal control is derived and applied to the F-8 DFBW aircraft. A randomly-sampled linear process model with additive process and noise is developed.
The objective of the research has been to study the design of adaptive/learning control systems for the control of large flexible structures. In the first activity an adaptive/learning control methodology for flexible space structures was investigated. The approach was based on using a modal model of the flexible structure dynamics and an output-error identification scheme to identify modal parameters. In the second activity, a least-squares identification scheme was proposed for estimating both modal parameters and modal-to-actuator and modal-to-sensor shape functions. The technique was applied to experimental data obtained from the NASA Langley beam experiment. In the third activity, a separable nonlinear least-squares approach was developed for estimating the number of excited modes, shape functions, modal parameters, and modal amplitude and velocity time functions for a flexible structure. In the final research activity, a dual-adaptive control strategy was developed for regulating the modal dynamics and identifying modal parameters of a flexible structure. A min-max approach was used for finding an input to provide modal parameter identification while not exceeding reasonable bounds on modal displacement.
The systematic storage in neural networks of prior information to be used in the design of various control subsystems is investigated. Assuming that the prior information is available in a certain form (namely, input/output data points and specifications between the data points), a particular neural network and a corresponding parameter design method are introduced. The proposed neural network addresses the issue of effectively using prior information in the areas of dynamical system (plant and controller) modeling, fault detection and identification, information extraction, and control law scheduling.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
This study evaluates the energy performance of ASHRAE Guideline 36–compliant control (ASHRAE 36 control) and reinforcement learning (RL)–based control through experimental field tests and a simulation study. Three field tests were conducted at Oak Ridge National Laboratory’s commercial building test facility in Oak Ridge, Tennessee: a baseline with a baseline conventional control, a test with ASHRAE 36 control, and a test with RL-based control. The selected ASHRAE 36 controls were trim and respond control, as well as variable air volume (VAV) box control. We compared the measured supply air temperature of the rooftop unit, VAV box supply air temperature, and VAV box supply airflow rate across the three test cases. The field data indicated that ASHRAE 36 controls operated as specified by ASHRAE Guideline 36. Based on these data, ASHRAE 36 control achieved a 45 % reduction in hourly averaged HVAC energy consumption compared with the baseline, and RL-based control achieved a 66 % reduction. These potential annual energy savings were confirmed using a calibrated whole-building energy model. Compared with the baseline, ASHRAE 36 control reduced HVAC energy consumption by 42 %, and RL-based control achieved a 54 % reduction. Furthermore, RL-based control reduced total HVAC energy consumption by 21 % more than ASHRAE 36 control.
Reinforcement learning (RL) has shown significant success in sequential decision making in fields like autonomous vehicles, robotics, marketing and gaming industries. This success has attracted the attention to the RL control approach for building energy systems which are becoming complicated due to the need to optimize for multiple, potentially conflicting, goals like occupant comfort, energy use and grid interactivity. However, for real world applications, RL has several drawbacks like requiring large training data and time, and unstable control behavior during the early exploration process making it infeasible for an application directly to building control tasks. To address these issues, an imitation learning approach is utilized herein where the RL agents starts with a policy transferred from accepted rule based policies and heuristic policies. This approach is successful in reducing the training time, preventing the unstable early exploration behavior and improving upon an accepted rule-based policy - all of these make RL a more practical control approach for real world applications in the domain of building controls.
In this paper, we present a reinforcement-learning based distributed approach to wind farm energy capture maximization using yaw control, also known as wake steering. In order to maximize the power output of a wind farm, it is often necessary for individual turbines to decrease their own power output through yaw misalignment so as to deflect their wakes away from downstream turbines. Although using model-based methods to achieve yaw misalignment is one option, a model-free method might be better suited to incorporate factors that are difficult to model or changing conditions. We propose an algorithm that adapts concepts of temporal difference reinforcement learning distributed to a multi-agent environment that allows individual turbines to act so as to optimize overall wind farm output and react to unforeseen disturbances.
Both model predictive control (MPC) and deep reinforcement learning control (DRL) have been presented as a way to approximate the true optimality of a dynamic programming problem, and these two have shown significant operational cost saving potentials for building energy systems. Furthermore, there is still a lack of in-depth quantitative studies on their approximation levels to the true optimality, especially in the building energy domain. To fill in the gap, this paper provides a numerical framework that enables the evaluation of the optimality levels of different controllers for building energy systems. This framework is then used to comprehensively compare the optimal control performance of both MPC and DRL controllers with given computation budgets for a single zone fan coil unit system. Note the optimality is estimated based on a user-specific selection of trade-off weights among energy costs, thermal comfort and control slew rates. Compared with the best optimality we can find through expensive optimization simulations, the best DRL agent can maximally approximate the optimality by 96.54%, which outperforms the best MPC whose optimality level is 90.11%. However, due to the stochasticity, the DRL agent is only expected to approximate the optimality by 90.42%, which is almost equivalent to the best MPC. Except for Proximal Policy Optimization (PPO), all DRL agents can have a better approximation to the optimality than the best MPC, and are expected to have better approximation than the MPC with a prediction horizon of 32 steps (15 min per step). In terms of reducing energy cost and thermal discomfort, MPC can outperform the rule-based control (RBC) by 18.47%–25.44%. DRL can be expected to outperform RBC by 18.95%–25.65% ,and the best DRL control policy can outperform RBC by 20.29%–29.72%. Although the comparison of the optimality level is performed in a perfect setting, e.g., MPC assumes perfect models, and DRL assumes a perfect offline training process and online deployment process, this can shed insight on their capabilities of approximating to the original dynamic programming problem.
Advanced life support systems have many interacting processes and limited resources. Controlling and optimizing advanced life support systems presents unique challenges. In particular, advanced life support systems are nonlinear coupled dynamical systems and it is difficult for humans to take all interactions into account to design an effective control strategy. In this project. we developed several reinforcement learning controllers that actively explore the space of possible control strategies, guided by rewards from a user specified long term objective function. We evaluated these controllers using a discrete event simulation of an advanced life support system. This simulation, called BioSim, designed by Nasa scientists David Kortenkamp and Scott Bell has multiple, interacting life support modules including crew, food production, air revitalization, water recovery, solid waste incineration and power. They are implemented in a consumer/producer relationship in which certain modules produce resources that are consumed by other modules. Stores hold resources between modules. Control of this simulation is via adjusting flows of resources between modules and into/out of stores. We developed adaptive algorithms that control the flow of resources in BioSim. Our learning algorithms discovered several ingenious strategies for maximizing mission length by controlling the air and water recycling systems as well as crop planting schedules. By exploiting non-linearities in the overall system dynamics, the learned controllers easily out- performed controllers written by human experts. In sum, we accomplished three goals. We (1) developed foundations for learning models of coupled dynamical systems by active exploration of the state space, (2) developed and tested algorithms that learn to efficiently control air and water recycling processes as well as crop scheduling in Biosim, and (3) developed an understanding of the role machine learning in designing control systems for advanced life support.
Not Available
The electric grid of things (EGoT) promises great potential for innovative grid services by tapping into vast load flexibility. However, the unique characteristics of EGoT, being a part of the cyber-physical electric power system, present both opportunities and challenges, especially concerning supply-demand balancing, stability, and communication constraints. Traditionally, centralized control was employed to ensure balance and stability in power systems. However, with the massive influx of EGoT devices, new strategies are needed to efficiently coordinate and control these distributed devices for optimal grid operations. While some studies have explored efficiency and economic models, there remains a gap in ensuring reliability under everyday operations and resilience during extreme conditions. Addressing this gap, this project develops the technology for an Energy Service Interface (ESI) that includes novel pricing, control, learning, and distributed optimization algorithms, which will enable utilities to recruit EGoT assets for crucial grid services such as load flexibility, voltage regulation, and situation-awareness. The key novelty of the proposed technology is the careful distribution of learning and control functions across utility and EGoT asset owners such that provably efficient and resilient grid operations are attained while respecting communication and information-exchange constraints. Specifically, the project team develops machine-learning enhanced load modeling methods to allow EGoT asset owners to learn their load capability and flexibility, and develops pricing-based and decentralized learning-based control so that asset owners can coordinate to meet system-wide demand-supply balance and reliability goals. For extreme situations involving high-impact, low-probability catastrophic events (termed the “black-sky” events), the team also develops (1) a “Feeder-Operating Center-on-a-Laptop” (FOCAL) software that can assist utility personnel in leveraging EGoT assets to accelerate the service recovery of damaged feeders, and (2) distributed optimization algorithms that can coordinate the operation points of EGoT devices under severe communication constraints. The proposed technology has been extensively tested and evaluated through simulations and on a testbed. In summary, as we transition into a more interconnected and digital power grid era, our project’s findings and developments offer a pivotal step toward guaranteeing both efficiency and resilience in the face of both everyday operations and rare “black-sky” events.
Adaptive digitalized learning control system for launch vehicles
Adaptive patterns in central nervous tissue learning process
This report contains a complete description of a learning control system designed for the F8-DFBW aircraft. The system is parameter-adaptive with the additional feature that it 'learns' the variation of the control system gains needed over the flight envelope. It, thus, generates and modifies its gain schedule when suitable data are available. The report emphasizes the novel learning features of the system: the forms of representation of the flight envelope and the process by which identified parameters are used to modify the gain schedule. It contains data taken during piloted real-time 6 degree-of-freedom simulations that were used to develop and evaluate the system.
Hedrick, E., K. Reynolds, D. Bhattacharyya, S.E. Zitney, and B.P. Omell, "Adaptive Control Using Reinforcement Learning Augmented Model Predictive Control," Advanced Manufacturing and Processing Conference 2022, Bethesda, MD, June 1-3 (2022).
This paper presents a generalized control strategy that enhances fuzzy controllers with self-learning capability for achieving prescribed control objectives in a near-optimal manner. This methodology, termed temporal back propagation, is model-insensitive in the sense that it can deal with plants that can be represented in a piecewise-differentiable format, such as difference equations, neural networks, GMDH structures, and fuzzy models. Regardless of the numbers of inputs and outputs of the plants under consideration, the proposed approach can either refine the fuzzy if-then rules if human experts, or automatically derive the fuzzy if-then rules obtained from human experts are not available. The inverted pendulum system is employed as a test-bed to demonstrate the effectiveness of the proposed control scheme and the robustness of the acquired fuzzy controller.