Hypersonic Guidance and Control via Deep Reinforcement Learning Methods.
Abstract not provided.
Furfaro, Roberto↗
SEARCH · Search NASA
Results for “Learning control”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Machine Learning-Based Tuning of Control Parameters for LLRF System of Superconducting Cavities
Explore the source record for details and available documents.
Diaz Cruz, J. A.↗
Bayesian Algorithms for Practical Accelerator Control and Adaptive Machine Learning for Time-Varying Systems
Explore the source record for details and available documents.
Scheinker, Alexander↗
Adaptive Physics-Informed Machine Learning for Robust Beam Control and Diagnostics
Explore the source record for details and available documents.
Scheinker, Alexander↗
Dynamically Learning Incentives for Load Control
As electrical generation becomes more distributed and volatile, and loads become more uncertain, controllability of distributed energy resources (DERs), regardless of their ownership status, will be necessary for grid reliability. Grid operators lack direct control over end-users' grid interactions, such as energy usage, but incentives can influence behavior -- for example, an end-user that receives a grid-driven incentive may adjust their consumption or expose relevant control variables in response. A key challenge in studying such incentives is the lack of data about human behavior, which usually motivates strong assumptions, such as distributional assumptions on compliance or rational utility-maximization. In this paper, we propose a general incentive mechanism in the form of a constrained optimization problem -- our approach is distinguished from prior work by modeling human behavior (e.g., reactions to an incentive) as an arbitrary unknown function. We propose feedback-based optimization algorithms to solve this problem that each leverage different amounts of information and/or measurements. We show that each converges to an asymptotically stable incentive with (near)-optimality guarantees given mild assumptions on the problem. Finally, we evaluate our proposed techniques in voltage regulation simulations on standard test beds. We test a variety of settings, including those that break assumptions required for theoretical convergence (e.g., convexity, smoothness) to capture realistic settings. In this evaluation, our proposed algorithms are able to find near-optimal incentives even when the reaction to an incentive is modeled by a theoretically difficult (yet realistic) function.
demand response↗
CONTROLLING ENERGY STORAGE SYSTEMS: LESSONS LEARNED FROM THE WASHINGTON CLEAN ENERGY FUND DEMONSTRATION PROJECTS
Washington Clean Energy Fund energy storage demonstration projects created opportunities for learning many aspects of Energy Storage Systems (ESS) including its control for economic operation. PNNL, as the lead for use case analytics program for the demonstration projects, had the opportunity to experience practical aspects of ESS control in terms of control strategy development, control systems deployment, and performance of ESS during actual operation. This paper presents some of those learning opportunities availed to date and a few key lessons with an intention to benefit the wider energy storage community.
Washington CEF, ESS control strategy, control syst↗
Selected Flight Test Results for Online Learning Neural Network-Based Flight Control System
The NASA F-15 Intelligent Flight Control System project team developed a series of flight control concepts designed to demonstrate neural network-based adaptive controller benefits, with the objective to develop and flight-test control systems using neural network technology to optimize aircraft performance under nominal conditions and stabilize the aircraft under failure conditions. This report presents flight-test results for an adaptive controller using stability and control derivative values from an online learning neural network. A dynamic cell structure neural network is used in conjunction with a real-time parameter identification algorithm to estimate aerodynamic stability and control derivative increments to baseline aerodynamic derivatives in flight. This open-loop flight test set was performed in preparation for a future phase in which the learning neural network and parameter identification algorithm output would provide the flight controller with aerodynamic stability and control derivative updates in near real time. Two flight maneuvers are analyzed - pitch frequency sweep and automated flight-test maneuver designed to optimally excite the parameter identification algorithm in all axes. Frequency responses generated from flight data are compared to those obtained from nonlinear simulation runs. Flight data examination shows that addition of flight-identified aerodynamic derivative increments into the simulation improved aircraft pitch handling qualities.
Williams-Hayes, Peggy S.↗
Neural-Network-Development Program
NETS, software tool for development and evaluation of neural networks, provides simulation of neural-network algorithms plus computing environment for development of such algorithms. Uses back-propagation learning method for all of networks it creates. Enables user to customize patterns of connections between layers of network. Also provides features for saving, during learning process, values of weights, providing more-precise control over learning process. Written in ANSI standard C language. Machine-independent version (MSC-21588) includes only code for command-line-interface version of NETS 3.0.
Phillips, Todd A.↗
Functional Mock-up Interface - Machine Learning Center (FMI-MLC) v1
The Functional Mock-up Interface - Machine Learning Center is a tool to interface the Python-Tensorflow library with building simulation models, to develop and train machine learning-based controls. In particular, this framework allows the use of reinforcement learning through the standardized Functional Mock-up Interface, an open industry standard for co-simulation. First application was the development of a controller for dynamic façade and heating ventilation and air conditioning (HVAC) systems.
Gehbauer, Christoph↗
Selected Flight Test Results for Online Learning Neural Network-Based Flight Control System
The NASA F-15 Intelligent Flight Control System project team has developed a series of flight control concepts designed to demonstrate the benefits of a neural network-based adaptive controller. The objective of the team is to develop and flight-test control systems that use neural network technology to optimize the performance of the aircraft under nominal conditions as well as stabilize the aircraft under failure conditions. Failure conditions include locked or failed control surfaces as well as unforeseen damage that might occur to the aircraft in flight. This report presents flight-test results for an adaptive controller using stability and control derivative values from an online learning neural network. A dynamic cell structure neural network is used in conjunction with a real-time parameter identification algorithm to estimate aerodynamic stability and control derivative increments to the baseline aerodynamic derivatives in flight. This set of open-loop flight tests was performed in preparation for a future phase of flights in which the learning neural network and parameter identification algorithm output would provide the flight controller with aerodynamic stability and control derivative updates in near real time. Two flight maneuvers are analyzed a pitch frequency sweep and an automated flight-test maneuver designed to optimally excite the parameter identification algorithm in all axes. Frequency responses generated from flight data are compared to those obtained from nonlinear simulation runs. An examination of flight data shows that addition of the flight-identified aerodynamic derivative increments into the simulation improved the pitch handling qualities of the aircraft.
Williams, Peggy S.↗
A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles
The advent of connected vehicle (CV) technology offers new possibilities for a revolution in future transportation systems. With the availability of real-time traffic data from CVs, it is possible to more effectively optimize traffic signals to reduce congestion, increase fuel efficiency, and enhance road safety. The success of CV-based signal control depends on an accurate and computationally efficient model that accounts for the stochastic and nonlinear nature of the traffic flow. Without the necessity of prior knowledge of the traffic system’s model architecture, reinforcement learning (RL) is a promising tool to acquire the control policy through observing the transition of the traffic states. In this paper, we propose a novel data-driven traffic signal control method that leverages the latest in deep learning and reinforcement learning techniques. By incorporating a compressed representation of the traffic states, the proposed method overcomes the limitations of the existing methods in defining the action space to include more practical and flexible signal phases. The simulation results demonstrate the convergence and robust performance of the proposed method against several existing benchmark methods in terms of average vehicle speeds, queue length, wait time, and traffic density.
42 ENGINEERING↗
Dynamic Model Agnostic Reliability Evaluation of Machine-Learning Models Integrated in Instrumentation & Control Systems
In recent years, the field of data-driven neural network-based machine learning (ML) algorithms has grown significantly and spurred research in its applicability to instrumentation and control systems. While they are promising in operational contexts, the trustworthiness of such algorithms is not adequately assessed. Failures of ML-integrated systems are poorly understood; the lack of comprehensive risk modeling can degrade the trustworthiness of these systems. In recent reports by the National Institute for Standards and Technology, trustworthiness in ML is a critical barrier to adoption and will play a vital role in intelligent systems' safe and accountable operation. Thus, in this work, we demonstrate a real-time model-agnostic method to evaluate the relative reliability of ML predictions by incorporating out-of-distribution detection on the training dataset. It is well documented that ML algorithms excel at interpolation (or near-interpolation) tasks but significantly degrade at extrapolation. This occurs when new samples are "far" from training samples. The method, referred to as the Laplacian distributed decay for reliability (LADDR), determines the difference between the operational and training datasets, which is used to calculate a prediction's relative reliability. LADDR is demonstrated on a feedforward neural network-based model used to predict safety significant factors during different loss-of-flow transients. LADDR is intended as a "data supervisor" and determines the appropriateness of well-trained ML models in the context of operational conditions. Ultimately, LADDR illustrates how training data can be used as evidence to support the trustworthiness of ML predictions when utilized for conventional interpolation tasks.
97 MATHEMATICS AND COMPUTING↗
A Multiagent Deep Reinforcement Learning-Enabled Dual-Branch Damping Controller for Multimode Oscillation
Here, this study develops a multiagent deep reinforcement learning (MADRL)-enabled framework for the decentralized cooperative control of a novel dual-branch (DB) damping controller for both low-frequency oscillation (LFO) and ultralow-frequency oscillation (ULFO). It has two branches, each of which consists of a proportional resonance (PR) and a second-order polynomial that is designed to handle target oscillation modes. To improve the robustness of the controller to system uncertainties, MADRL is developed, where multiagents are centrally trained to obtain the coordinated adaptive control policy while being executed in a decentralized manner to provide the optimal parameter setting for each controller with only local states. Comparisons with the IEEE 10-machine 39-bus system demonstrate that the proposed method achieves better robustness to uncertainties, lower communication delay, and single-point failure, as well as damping control performances for both LFO and ULFO.
97 MATHEMATICS AND COMPUTING↗
Integrated Routing and Traffic Signal Control for CAVs via Reinforcement Learning Approach
Incorporating Connected and Automated Vehicles (CAVs) into urban traffic networks presents opportunities and challenges for traffic management systems. This paper aims to develop an integrated routing and traffic signal control system designed explicitly for CAVs, utilizing a Reinforcement Learning (RL) approach. The objective is to enhance traffic flow and improve overall transportation efficiency in the controlled areas. We propose an innovative framework that employs the Deep Reinforcement Learning (DRL) algorithm, especially the Deep Q-network (DQN), to dynamically adjust the number of vehicles in the routes and the duration of traffic signals. Our simulation results demonstrate that a DQN agent successfully optimizes the number of vehicles in the routes and traffic signal timings of traffic signal controllers, eventually reducing total travel time. The study illustrates the potential usage of RL-based systems in managing routing and traffic signals for CAVs, offering a promising opportunity for future urban traffic management strategies.
Park, Jiho [New York University]↗
Space shuttle descent flight control design requirements and experiments Learned, Pt. 1 p 617-628
Some of the lessons learned during the development of the Space Shuttle descent flight control system (FCS) are reviewed. Examples confirm the importance for requirements definition, systems level analyses, and testing. In sounding these experiences may have implication for future designs or suggest the discipline required in this engineering art.
Kafer, G.↗
NASA's Learn-to-Fly Project Overview
Learn-to-Fly (L2F) is an advanced technology development effort aimed at assessing the feasibility of real-time, self-learning flight vehicles. Specifically, research has been conducted on merging real-time aerodynamic modeling, learning adaptive control, and other disciplines with the goal of using this “learn to fly” methodology to replace the current iterative vehicle development paradigm, substantially reducing the typical ground and flight testing requirements for air vehicle design. Recent activities included an aggressive flight test program with unique fully autonomous fight test vehicles to rapidly advance L2F technology. This paper presents an overview of the project and key components.
Heim, Eugene H. D.↗
Machine learning-based discovery of molecular descriptors that control polymer gas permeation
While machine learning has found increasing use in predicting the properties of polymeric materials with only a knowledge of chain architecture, determining the molecular factors underpinning properties (“interpretable AI”) has remained less well explored. We show that encoding chain chemistry in commonly employed formats, e.g., binary-valued fingerprints, leads to uniqueness issues during the hashing process to save storage space. This is because the hashing algorithm can map several chemical moieties into the same bit. These issues carry over into the ML algorithms, especially for “inverse” design and interpretable AI, and cannot be avoided by changing the length of the fingerprint. Using MACCS key featurizations of monomer repeats resolves some of these issues, and we show that a few substructures consistently appear in top features for maximizing permeability across several gases and ML models. These are carbon–carbon double bonds (as in polyacetylenes) especially when they are associated with methyl groups (found in branching architectures). Here these results, derived from the limited data set of ~ 500 polymers with experimental gas permeation data, are in agreement with physical insight and thus provide a robust foundation which could further enable study of these material classes through detailed experiments and simulations.
36 MATERIALS SCIENCE↗
Controlling a truck with an adaptive critic temporal difference CMAC design
In this study, CMAC (Cerebellar Model Articulated Controller) neural architectures are shown to be viable for the purposes of real-time learning and control. An adaptive critic temporal difference neurocontrol design has been implemented that learns in real-time how to back up a trailer truck along a fixed straight line trajectory. The truck backer-upper experiment is a standard performance measure in the neural network literature, but previously the training of the controllers was done off-line. With the CMAC neural architectures, it was possible to train the neurocontrollers on-line in real-time on a MS-DOS PC 386.
Shelton, Robert O.↗