Search NASA⌕ Search

SEARCH · Search NASA

Results for “Learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Deep Reinforcement Learning Based HVAC Control for Reducing Carbon Footprint of Buildings

In this paper, we present our work on deep reinforcement learning (DRL) based intelligent control of Heating, Ventilation, and Air Conditioning (HVAC) with the goal of reducing carbon emission. We performed this task using 1) Marginal Operating Emission Rates (MOER), where the objective was to shift the demand to the low emission period of the day and 2) Time-Of-Use (TOU) demand-response price where the objective was to shift the demand to low price period of the day. This was achieved by learning an optimal pre-cooing strategy. We found the carbon emission reduction in the range of 6%-16% depending on the opportunity presented by the MOER signal. Similarly, we observed the carbon emission reduction in the range of 23%-29% during the peak price period when TOU price was used. The results clearly demonstrated the applicability of our approach in reducing the carbon footprint of the building.

carbon emission↗
NASA NTRS1991

Approximate reasoning-based learning and control for proximity operations and docking in space

A recently proposed hybrid-neutral-network and fuzzy-logic-control architecture is applied to a fuzzy logic controller developed for attitude control of the Space Shuttle. A model using reinforcement learning and learning from past experience for fine-tuning its knowledge base is proposed. Two main components of this approximate reasoning-based intelligent control (ARIC) model - an action-state evaluation network and action selection network are described as well as the Space Shuttle attitude controller. An ARIC model for the controller is presented, and it is noted that the input layer in each network includes three nodes representing the angle error, angle error rate, and bias node. Preliminary results indicate that the controller can hold the pitch rate within its desired deadband and starts to use the jets at about 500 sec in the run.

Berenji, Hamid R.↗
DOE OSTI2023

Real-time machine-learning-driven control system of a deformable mirror for achieving aberration-free X-ray wavefronts

A neural-network machine learning model is developed to control a bimorph adaptive mirror to achieve and preserve aberration-free coherent X-ray wavefronts at synchrotron radiation and free electron laser beamlines. The controller is trained on a mirror actuator response directly measured at a beamline with a real-time single-shot wavefront sensor, which uses a coded mask and wavelet-transform analysis. The system has been successfully tested on a bimorph deformable mirror at the 28-ID IDEA beamline of the Advanced Photon Source at Argonne National Laboratory. It achieved a response time of a few seconds and maintained desired wavefront shapes (e.g., a spherical wavefront) with sub-wavelength accuracy at 20 keV of X-ray energy. This result is significantly better than what can be obtained using a linear model of the mirror’s response. The developed system has not been tailored to a specific mirror and can be applied, in principle, to different kinds of bending mechanisms and actuators.

47 OTHER INSTRUMENTATION↗
NASA NTRS1994

Learning fuzzy logic control system

The performance of the Learning Fuzzy Logic Control System (LFLCS), developed in this thesis, has been evaluated. The Learning Fuzzy Logic Controller (LFLC) learns to control the motor by learning the set of teaching values that are generated by a classical PI controller. It is assumed that the classical PI controller is tuned to minimize the error of a position control system of the D.C. motor. The Learning Fuzzy Logic Controller developed in this thesis is a multi-input single-output network. Training of the Learning Fuzzy Logic Controller is implemented off-line. Upon completion of the training process (using Supervised Learning, and Unsupervised Learning), the LFLC replaces the classical PI controller. In this thesis, a closed loop position control system of a D.C. motor using the LFLC is implemented. The primary focus is on the learning capabilities of the Learning Fuzzy Logic Controller. The learning includes symbolic representation of the Input Linguistic Nodes set and Output Linguistic Notes set. In addition, we investigate the knowledge-based representation for the network. As part of the design process, we implement a digital computer simulation of the LFLCS. The computer simulation program is written in 'C' computer language, and it is implemented in DOS platform. The LFLCS, designed in this thesis, has been developed on a IBM compatible 486-DX2 66 computer. First, the performance of the Learning Fuzzy Logic Controller is evaluated by comparing the angular shaft position of the D.C. motor controlled by a conventional PI controller and that controlled by the LFLC. Second, the symbolic representation of the LFLC and the knowledge-based representation for the network are investigated by observing the parameters of the Fuzzy Logic membership functions and the links at each layer of the LFLC. While there are some limitations of application with this approach, the result of the simulation shows that the LFLC is able to control the angular shaft position of the D.C. motor. Furthermore, the LFLC has better performance in rise time, settling time and steady state error than to the conventional PI controller. This abstract accurately represents the content of the candidate's thesis. I recommend its publication.

Lung, Leung Kam↗
DOE OSTI2022

LBC (Learning Building Control)

LBC encompasses the source code and data to reproduce and extend results for a manuscript that compares demand responsive control schemes for multi-zone buildings. It includes examples of model predictive control (MPC), value function approximation via CVXPYLAYERS, and differentiable predictive control (DPC). The goal of the study is to evaluate state-of-the-art controllers and establish the efficacy (if any) of learning-based approaches that leverage deep neural networks in one way or another.

Zhang, Xiangyu↗
DOE OSTI2025

Learning and Controlling Silicon Dopant Transitions in Graphene Using Scanning Transmission Electron Microscopy

A machine learning approach is introduced to determine the transition dynamics of silicon atoms on a single layer of carbon atoms, when stimulated by the electron beam of a scanning transmission electron microscope (STEM). This method is data-centric, leveraging data collected on a STEM. The data samples are processed and filtered to produce symbolic representations, which is used to train a neural network to predict transition probabilities. These learned transition dynamics are then leveraged to guide a single silicon atom throughout the lattice to pre-determined target destinations. Empirical analyses are presented that demonstrate the efficacy and generality of the approach.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Learning-Based Load Control to Support Resilient Networked Microgrid Operations

Microgrids have proven to be an effective option for increasing the resiliency of critical end-use loads during extreme events. Building on past operational experiences, some microgrid operators are examining the potential to network microgrids to further improve resiliency. However, the frequency deviations experienced on isolated microgrids during transient events, such as switching operations, step increases in load, and loss of generation, are significantly larger than those typically seen on bulk transmission systems. The larger frequency deviations can cause a loss of inverter-connected assets, resulting in a loss of power to critical end-use loads. This paper presents a method of mitigating the impact of transient events by engaging end-use loads using Grid-Friendly Appliance TM (GFA) controllers. An online, i.e., real-time, device-level algorithm is presented, which adjusts individual GFA controller frequency set-points based on the operational characteristics of each end-use load, and on the changing grid dynamic characteristics. The presented method improves the dynamic stability of the networked microgrid operations while minimizing the interruptions to end-use loads. The presented work is validated with dynamic simulations using a modified version of the IEEE 123-node test system with three microgrids, using the GridLAB-D TM simulation environment.

Radhakrishnan, Nikitha↗
DOE OSTI2025

Deep Reinforcement Learning Based Control of Wind Turbines for Fast Frequency Response

In order to fulfill vital auxiliary grid services, such as load regulation, spin and non-spin reserve provision, and frequency support during emergencies, there is often a requirement for certain wind farms to operate in de-loaded modes. Leveraging the swift response capabilities of wind farms, this study demonstrates that reserving power in de-loaded modes can significantly enhance power grid stability and reliability during system contingencies. Controlling wind farms optimally for frequency support is intricate due to the nonlinearity of models and controllers and the complexity of wind farm interactions with power systems. Here, to address this challenge, this paper introduces a novel approach that integrates wind turbines into reinforcement learning-based solutions for frequency response. This innovative methodology utilizes the state-of-the-art reinforcement learning algorithm known as the surrogate-gradient-based evolutionary strategy. The proposed learning-based algorithm provides continuous control of wind farm output to rapidly stabilize system frequency and prevent unnecessary trips of under-frequency load shedding relays. To facilitate efficient training, parallel computing techniques are employed. The proposed methodology is evaluated on a modified IEEE-39 bus system, and simulation results reveal its efficacy in reliably supporting power system frequency and preventing the need for unnecessary load shedding.

Gao, Wei [Argonne National Laboratory (ANL), Argon↗
NASA NTRS2008

Towards Intelligent Control for Next Generation Aircraft

NASA Aeronautics Subsonic Fixed Wing Project is focused on mitigating the environmental and operation impacts expected as aviation operations triple by 2025. The approach is to extend technological capabilities and explore novel civil transport configurations that reduce noise, emissions, fuel consumption and field length. Two Next Generation (NextGen) aircraft have been identified to meet the Subsonic Fixed Wing Project goals - these are the Hybrid Wing-Body (HWB) and Cruise Efficient Short Take-Off and Landing (CESTOL) aircraft. The technologies and concepts developed for these aircraft complicate the vehicle s design and operation. In this paper, flight control challenges for NextGen aircraft are described. The objective of this paper is to examine the potential of state-of-the-art control architectures and algorithms to meet the challenges and needed performance metrics for NextGen flight control. A broad range of conventional and intelligent control approaches are considered, including dynamic inversion control, integrated flight-propulsion control, control allocation, adaptive dynamic inversion control, data-based predictive control and reinforcement learning control.

Acosta, Diana Michelle↗
DOE OSTI2023

Can We Use Machine Learning to Control Nuclear Power Plants?

Slides for the INL AI/ML Symposium 11.0 focused on AI/ML in instrumentation, control, and automation. This presentation was on how we can use AI/ML for control of nuclear power plants. A recording of the presentation and other INL AI/ML symposiums can be found at https://inl.gov/artificial-intelligence/.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Improving DFIG performance under fault scenarios through evolutionary reinforcement learning based control

The double-fed induction generator (DFIG) usually experiences high rotor current and DC capacitor link voltage during system fault events. In this paper, a novel data-driven approach is proposed to enhance the DFIG performance under fault scenarios. An advanced reinforcement learning (RL) algorithm called guided surrogate-gradient-based evolution strategy (GSES) is used to control the DFIG power and capacitor DClink voltage by adjusting the optional reference signals. This controller is able to prevent the rotor of DFIG from overcurrent risk and maintain the grid-connected operation. The proposed GSES-based control algorithm was evaluated through simulations on a 3.6-MW DFIG in PSCAD/EMTDC software. Results have validated the effectiveness of the proposed GSESbased control algorithm in improving the DFIG performance under fault scenarios.

42 ENGINEERING↗
DOE OSTI2025

A Sequential Model Predictive and Deep Reinforcement Learning-Based Controller for Distribution System Outage Mitigation under Hurricane Events

This paper proposes a proactive outage mitigation framework for power distribution networks to withstand hurricane-induced disruptions. It leverages Model Predictive Control (MPC) to identify safe lines for proactive switching during hurricanes, minimizing the risk of cascading failures and voltage violations. The switching strategies optimized by MPC are sequentially integrated with a Deep Reinforcement Learning agent using the Advantage Actor-Critic algorithm, enabling dynamic line switching to maximize connected buses and minimize voltage violations in real time. Using a probabilistic hurricane model, the framework predicts line failures and adapts to varying conditions to enhance grid resilience. Simulations on the IEEE 123-bus system demonstrate its effectiveness in maintaining high connectivity and minimizing disruptions. Real-time testing with an RTDS confirms the practicality and reliability of the proposed approach.

Selim, Alaa [University of Connecticut]↗
NASA NTRS1990

Learning-based position control of a closed-kinematic chain robot end-effector

A trajectory control scheme whose design is based on learning theory, for a six-degree-of-freedom (DOF) robot end-effector built to study robotic assembly of NASA hardwares in space is presented. The control scheme consists of two control systems: the feedback control system and the learning control system. The feedback control system is designed using the concept of linearization about a selected operating point, and the method of pole placement so that the closed-loop linearized system is stabilized. The learning control scheme consisting of PD-type learning controllers, provides additional inputs to improve the end-effector performance after each trial. Experimental studies performed on a 2 DOF end-effector built at CUA, for three tracking cases show that actual trajectories approach desired trajectories as the number of trials increases. The tracking errors are substantially reduced after only five trials.

Nguyen, Charles C.↗
DOE OSTI2022

Learning-based Anticipatory Control of Microreactors

Learning-based anticipatory control advances the level of autonomy of microreactor control systems, a key consideration for the unattended operation of fission batteries. In this work, an anticipatory control system is shown to provide accurate load-following for a microreactor by utilizing a model predictive control framework.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Towards Off-policy Evaluation as a Prerequisite for Real-world Reinforcement Learning in Building Control

We present an initial study of off-policy evaluation (OPE), a problem prerequisite to real-world reinforcement learning (RL), in the context of building control. OPE is the problem of estimating a policy's performance without running it on the actual system, using historical data from the existing controller. It enables the control engineers to ensure a new, pretrained policy satisfies the performance requirements and safety constraints of a real-world system, prior to interacting with it. While many methods have been developed for OPE, no study has evaluated which ones are suitable for building operational data, which are generated by deterministic policies and have limited coverage of the state-action space. After reviewing existing works and their assumptions, we adopted the approximate model (AM) method. Furthermore, we used bootstrapping to quantify uncertainty and correct for bias. In a simulation study, we evaluated the proposed approach on 10 policies pretrained with imitation learning. On average, the AM method estimated the energy and comfort costs with 1.84% and 14.1% error, respectively.

Chen, B↗