Search NASA⌕ Search

SEARCH · Search NASA

Results for “learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Design of nematic liquid crystals to control microscale dynamics

The dynamics of small particles, both living such as swimming bacteria and inanimate, such as colloidal spheres, has fascinated scientists for centuries. If one could learn how to control and streamline their chaotic motion, that would open technological opportunities in the transformation of stored or environmental energy into systematic motion, with applications in micro-robotics, transport of matter, guided morphogenesis. This review presents an approach to command microscale dynamics by replacing an isotropic medium with a liquid crystal. Orientational order and associated properties, such as elasticity, surface anchoring, and bulk anisotropy, enable new dynamic effects, ranging from the appearance and propagation of particle-like solitary waves to self-locomotion of an active droplet. By using photoalignment, the liquid crystal can be patterned into predesigned structures. In the presence of the electric field, these patterns enable the transport of solid and fluid particles through nonlinear electrokinetics rooted in anisotropy of conductivity and permittivity. Director patterns command the dynamics of swimming bacteria, guiding their trajectories, polarity of swimming, and distribution in space. This guidance is of a higher level of complexity than a simple following of the director by rod-like microorganisms. Namely, the director gradients mediate hydrodynamic interactions of bacteria to produce an active force and collective polar modes of swimming. The patterned director could also be engraved in a liquid crystal elastomer. When an elastomer coating is activated by heat or light, these patterns produce a deterministic surface topography. The director gradients define an activation force that shapes the elastomer in a manner similar to the active stresses triggering flows in active nematics. Finally, the patterned elastomer substrates could be used to define the orientation of cells in living tissues. The liquid-crystal guidance holds a major promise in achieving the goal of commanding microscale active flows.

36 MATERIALS SCIENCE↗
NASA NTRS2015

DSCOVR Contamination Lessons Learned

The Triana observatory was built at NASA GSFC in the late 1990's, then placed into storage. After approximately ten years it was removed from storage and repurposed as the Deep Space Climate Observatory (DSCOVR). This presentation outlines the contamination control program lessons learned during the integration, test and launch of DSCOVR.

DSCOVR↗
NASA NTRS1993

Evolving fuzzy rules in a learning classifier system

The fuzzy classifier system (FCS) combines the ideas of fuzzy logic controllers (FLC's) and learning classifier systems (LCS's). It brings together the expressive powers of fuzzy logic as it has been applied in fuzzy controllers to express relations between continuous variables, and the ability of LCS's to evolve co-adapted sets of rules. The goal of the FCS is to develop a rule-based system capable of learning in a reinforcement regime, and that can potentially be used for process control.

Valenzuela-Rendon, Manuel↗
DOE OSTI2023

Risk-Constrained Reinforcement Learning for Inverter-Dominated Power System Controls

Here, this paper develops a risk-aware controller for grid-forming inverters (GFMs) to minimize large frequency oscillations in GFM inverter-dominated power systems. To tackle the high variability from loads/renewables, we incorporate a mean-variance risk constraint into the classical linear quadratic regulator (LQR) formulation for this problem. The risk constraint aims to bound the time-averaged cost of state variability and thus can improve the worst-case performance for large disturbances. The resulting risk-constrained LQR problem is solved through the dual reformulation to a minimax problem, by using a reinforcement learning (RL) method termed as stochastic gradient-descent with max-oracle (SGDmax). In particular, the zero-order policy gradient (ZOPG) approach is used to simplify the gradient estimation using simulated system trajectories. Numerical tests conducted on the IEEE 68-bus system have validated the convergence of our proposed SGDmax for GFM model and corroborate the effectiveness of the risk constraint in improving the worst-case performance while reducing the variability of the overall control cost.

Frequency control↗
DOE OSTI2022

Safety-Related Instrumentation and Control Pilot Upgrade: Initial Scoping Phase Implementation and Lessons Learned

In May 2016, the U.S. Nuclear Regulatory Commission (NRC) staff provided a digital instrumentation and control (I&C) regulatory infrastructure integrated action plan to the NRC for approval. One of the objectives of that plan was to establish a clear regulatory structure with reduced regulatory uncertainty to enable the expanded safe use of digital I&C in commercial nuclear reactors while continuing to ensure safety and security. To achieve this end, the NRC, with collaboration from industry, developed a streamlined License Amendment Request Alternate Review (AR) process for safety-related (SR) digital I&C upgrades. In spite of this effort, the industry has remained reluctant to perform such I&C upgrades because of perceived regulatory and financial risks associated with being the first or an early adopter of the AR process for SR I&C upgrades. The U.S. Department of Energy Light Water Reactor Sustainability Program at the Idaho National Laboratory performed Initial Scoping Phase research to help break this impasse by supporting a SR I&C Pilot Upgrade, working with MPR Associates, ScottMadden Inc., and Exelon Generation. Exelon’s Limerick Generating Station (LGS) was selected as the target for this research. This paper summarizes the Initial Scoping Phase engineering and operations, licensing, and project management activities necessary to bound the scope, schedule, and estimated cost of the project sufficiently to enable utility management authorization of Conceptual Design Phase activities. These efforts and associated products are intended to provide a template to support larger industry efforts to perform similar upgrades as a foundation stone for a digital transformation that will improve plant safety, reliability, and operational performance while lowering plant total cost of ownership. As a result of the combined effort of Exelon Generation and research participants, Conceptual Design Phase activities for the subject upgrade at LGS were approved by Exelon. Further, the U.S. Department of Energy also awarded a $50 million cost share award to Exelon in order to pave the way for SR I&C modernization and associated control room upgrades across the U.S. nuclear fleet. Additional research reports are planned for the Conceptual Design Phase, Detailed Design Phase, and the Implementation Phase of the LGS project to document the process followed and promulgate lessons learned to industry.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1993

Architecture for Adaptive Intelligent Systems

We identify a class of niches to be occupied by 'adaptive intelligent systems (AISs)'. In contrast with niches occupied by typical AI agents, AIS niches present situations that vary dynamically along several key dimensions: different combinations of required tasks, different configurations of available resources, contextual conditions ranging from benign to stressful, and different performance criteria. We present a small class hierarchy of AIS niches that exhibit these dimensions of variability and describe a particular AIS niche, ICU (intensive care unit) patient monitoring, which we use for illustration throughout the paper. We have designed and implemented an agent architecture that supports all of different kinds of adaptation by exploiting a single underlying theoretical concept: An agent dynamically constructs explicit control plans to guide its choices among situation-triggered behaviors. We illustrate the architecture and its support for adaptation with examples from Guardian, an experimental agent for ICU monitoring.

ACTIVE CONTROL↗
NASA NTRS2010

Foot Pedals for Spacecraft Manual Control

Fifty years ago, NASA decided that the cockpit controls in spacecraft should be like the ones in airplanes. But controls based on the stick and rudder may not be best way to manually control a vehicle in space. A different method is based on submersible vehicles controlled with foot pedals. A new pilot can learn the sub's control scheme in minutes and drive it hands-free. We are building a pair of foot pedals for spacecraft control, and will test them in a spacecraft flight simulator.

Love, Stanley G.↗
DOE OSTI2021

Model-Free Voltage Control of Active Distribution System with PVs Using Surrogate Model-Based Deep Reinforcement Learning

Accurate knowledge of the distribution system topology and parameters is required to achieve good voltage control performance, but this is difficult to obtain in practice. This paper proposes a physical-model-free voltage control method based on a surrogate-model-enabled deep reinforcement learning approach. Specifically, a surrogate model is trained in a supervised manner using the recorded limited number of historical data to learn the relationship between the power injections and voltage fluctuations of each node. Then, the deep reinforcement learning algorithm is applied to learn an optimal control strategy from the experiences obtained by continuous interactions with the surrogate model. The proposed method can achieve physical-model-free control of unbalanced distribution network and inform real-time decisions to deal with fast voltage fluctuations caused by the rapid variation of PV generation. Simulation results on an unbalance IEEE 123-bus system show that the proposed method can achieve similar performance as that of perfect physical-model-based approaches while being advantageous over other traditional methods.

active distribution network↗
DOE OSTI2020

Uranium Mill Tailings Radiation Control Act Title II DOE Due Diligence and Lessons Learned from a Previous Site Transfer - 20352

Title II of the Uranium Mill Tailings Radiation Control Act (UMTRCA) established that a government agency will provide perpetual care for closed uranium and thorium ore-processing sites that were operating under an NRC source material license in 1978 or were licensed thereafter. Commercial owners (licensees) operating under an NRC or agreement state specific license when UMTRCA was passed are responsible for conducting reclamation of any byproduct material remaining from uranium-ore processing operations in accordance with an NRC or agreement state approved reclamation plan. Reclamation includes both surface and groundwater remedies. Upon completion of reclamation and approval by NRC, the site is required to be transferred to either the host state or the DOE for long-term surveillance and maintenance. Since UMTRCA's enactment, six Title II sites have been transferred to DOE; an additional 24 Title II sites are anticipated to be transferred before 2050. DoE's role mandated under UMTRCA Title II as the long-term care custodian is to perform 'monitoring, maintenance, and emergency measures necessary to protect the public health and safety.' UMTRCA requires that the licensee pay a long-term surveillance charge 'sufficient to cover the annual costs of site surveillance.' However, at some sites such as the Bluewater, New Mexico, Disposal Site, this mandate has required additional effort and expense by DOE, beyond the originally anticipated and intended scope within UMTRCA, but within the authority of DOE under UMTRCA. In 1997, the Bluewater site became the second UMTRCA Title II site to be transferred to DOE. The site was the location of a uranium mill operated from 1953 until 1982. The specific licensee began site reclamation in 1991, and by 1995 all tailings and contaminated materials were encapsulated in two tailings disposal cells and other disposal areas. In addition to surface contamination, milling activities impacted groundwater in the two upper aquifers. In 1989, the specific licensee attempted active groundwater remediation; however, no significant reduction in contaminant concentrations was observed. As a result, the specific licensee applied to NRC for alternate concentration limits (ACLs) in 1990, which were approved in 1996 as being protective, after additional corrective actions were performed. Since transfer of the Bluewater site to DOE, unforeseen challenges have occurred, requiring additional actions. The first challenge is the occurrence of surface depressions located on the northern section of the main tailings disposal cell. The depressions were first observed during DoE's initial inspection in 1998; however, evidence of these can be observed on satellite images taken prior to transfer. Since being first observed, the depressions have continued to grow both in depth and areal extent. Due to the design of the main tailings disposal cell, the depressions impede storm water from being effectively shed off the 101- hectare (250-acre) top slope of the main tailings disposal cell. Instead, storm water accumulates in the depressions, forming a large ephemeral pond that has stored up to 16.3 x 10{sup 6} liters (4.3 million gallons) of stormwater. The ponding poses a potential risk to the integrity of the main tailings disposal cell in the case of a large storm event, with the potential to cause the pond to overtop and erode the cover material and underlying waste. DOE has taken a number of short-term actions to monitor, measure, and reduce the ponding and is currently working with the US Army Corps of Engineers to design and construct a repair. Additional challenges are associated with groundwater at the Bluewater site. Nine wells were present on the 1335-hectare (3300-acre) site upon transfer. Groundwater compliance was called into question after the State of New Mexico reduced its uranium groundwater standard from 5.0 to 0.03 milligrams per liter in 2004, and when an ACL for uranium was exceeded in a site monitoring well in 2010. Acquiring historical groundwater data and subsequent evaluations as well as additional DOE groundwater monitoring led to installing 10 new monitoring wells and performing additional site hydrogeology recharacterization. DOE continues to evaluate groundwater conditions at the site and works with the NRC to determine regulatory requirements and a path forward. As a result of lessons learned at the Bluewater site and other Title II sites, improved processes have been implemented at a programmatic level to increase due diligence before site transfer and prevent similar issues from occurring at other UMTRCA Title II sites under long-term management. DoE's due diligence process is documented in the Process for Transition of UMTRCA Title II Disposal Sites to DOE for Long-Term Surveillance and Maintenance and is designed to ensure that DOE has no technical or compliance concerns with regulatory decisions that might compromise protectiveness following site transfer to DOE. Implementation of the due diligence process has increased DoE's role prior to site transfer and has been effective in identifying potential issues. Actions by NRC and specific licensees, in response to enhanced due diligence efforts by DOE, are expected to minimize, if not totally prevent, the need for unanticipated actions by DOE pertaining to the surface and groundwater remedies after site transfer. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS2018

Verification and Validation Challenges for Adaptive Flight Control of Complex Autonomous Systems

Autonomy of aerospace systems requires the ability for flight control systems to be able to adapt to complex uncertain dynamic environment. In spite of the five decades of research in adaptive control, the fact still remains that currently no adaptive control system has ever been deployed on any safety-critical or human-rated production systems such as passenger transport aircraft. The problem lies in the difficulty with the certification of adaptive control systems since existing certification methods cannot readily be used for nonlinear adaptive control systems. Research to address the notion of metrics for adaptive control began to appear in the recent years. These metrics, if accepted, could pave a path towards certification that would potentially lead to the adoption of adaptive control as a future control technology for safety-critical and human-rated production systems. Development of certifiable adaptive control systems represents a major challenge to overcome. Adaptive control systems with learning algorithms will never become part of the future unless it can be proven that they are highly safe and reliable. Rigorous methods for adaptive control software verification and validation must therefore be developed to ensure that adaptive control system software failures will not occur, to verify that the adaptive control system functions as required, to eliminate unintended functionality, and to demonstrate that certification requirements imposed by regulatory bodies such as the Federal Aviation Administration (FAA) can be satisfied. This presentation will discuss some of the technical issues with adaptive flight control and related V&V challenges.

Nguyen, Nhan T.↗
DOE OSTI2026

Steady Spills, Stronger Signals: Machine Learning for Slow Spill Analysis

Particle accelerator experiments rely on stable, consistent proton beams to maximize scientific discovery. This presentation introduces beam spills, duty factor, and beam stability using a meteor shower analogy before exploring how feedback control and machine learning, including recurrent neural networks (RNNs), can analyze spill data, identify patterns, and predict beam behavior. Together, these approaches support beam optimization and improve our understanding of accelerator performance for experiments such as Mu2e.

Prescott, Matthew J. [Fermilab]↗
NASA NTRS2003

Intelligent Launch and Range Operations Virtual Test Bed (ILRO-VTB)

Intelligent Launch and Range Operations Virtual Test Bed (ILRO-VTB) is a real-time web-based command and control, communication, and intelligent simulation environment of ground-vehicle, launch and range operation activities. ILRO-VTB consists of a variety of simulation models combined with commercial and indigenous software developments (NASA Ames). It creates a hybrid software/hardware environment suitable for testing various integrated control system components of launch and range. The dynamic interactions of the integrated simulated control systems are not well understood. Insight into such systems can only be achieved through simulation/emulation. For that reason, NASA has established a VTB where we can learn the actual control and dynamics of designs for future space programs, including testing and performance evaluation. The current implementation of the VTB simulates the operations of a sub-orbital vehicle of mission, control, ground-vehicle engineering, launch and range operations. The present development of the test bed simulates the operations of Space Shuttle Vehicle (SSV) at NASA Kennedy Space Center. The test bed supports a wide variety of shuttle missions with ancillary modeling capabilities like weather forecasting, lightning tracker, toxic gas dispersion model, debris dispersion model, telemetry, trajectory modeling, ground operations, payload models and etc. To achieve the simulations, all models are linked using Common Object Request Broker Architecture (CORBA). The test bed provides opportunities for government, universities, researchers and industries to do a real time of shuttle launch in cyber space.

Bardina, Jorge↗
DOE OSTI2021

Improve Learning from Crowds via Generative Augmentation

Crowdsourcing provides an efficient label collection schema for supervised machine learning. However, to control annotation cost, each instance in the crowdsourced data is typically annotated by a small number of annotators. This creates a sparsity issue and limits the quality of machine learning models trained on such data. In this paper, we study how to handle sparsity in crowdsourced data using data augmentation. Specifically, we propose to directly learn a classifier by augmenting the raw sparse annotations. We implement two principles of high-quality augmentation using Generative Adversarial Networks: 1) the generated annotations should follow the distribution of authentic ones, which is measured by a discriminator; 2) the generated annotations should have high mutual information with the ground-truth labels, which is measured by an auxiliary network. Extensive experiments and comparisons against an array of state-of-the-art learning from crowds methods on three real-world datasets proved the effectiveness of our data augmentation framework. It shows the potential of our algorithm for low-budget crowdsourcing in general.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1991

An architecture for designing fuzzy logic controllers using neural networks

Described here is an architecture for designing fuzzy controllers through a hierarchical process of control rule acquisition and by using special classes of neural network learning techniques. A new method for learning to refine a fuzzy logic controller is introduced. A reinforcement learning technique is used in conjunction with a multi-layer neural network model of a fuzzy controller. The model learns by updating its prediction of the plant's behavior and is related to the Sutton's Temporal Difference (TD) method. The method proposed here has the advantage of using the control knowledge of an experienced operator and fine-tuning it through the process of learning. The approach is applied to a cart-pole balancing system.

Berenji, Hamid R.↗
DOE OSTI2024

Resilient Control of Networked Microgrids Using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Improving system-level resiliency of networked microgrids against adversarial cyber-attacks is an important aspect in the current regime of increased inverter-based resources (IBRs). To achieve that, this paper contributes in designing a hierarchical control layer, in conjunction with the existing control layers, resilient to adversarial attack signals. Considering model complexities, unknown dynamical behaviors of IBRs, and privacy issues regarding data sharing in multi-party-owned microgrids, designing such a control layer is non-trivial. Here, to tackle these issues, a novel federated reinforcement learning (Fed-RL) method is proposed. To grasp the interconnected dynamics of networked microgrids, the paper develops Federated Soft Actor-Critic (FedSAC) algorithm following the vertical structure of implementing Fed-RL. Next, utilizing the OpenAI Gym interface, we built a custom set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark comprising 3 interconnected microgrids. Finally, the learned policies in the simulation are transferred to the real-time hardware-in-the-loop (HIL) test-bed developed using the high-fidelity Hypersim platform. Finally, experiments show that the simulator-trained RL controllers achieve desirable performance with the test-bed platform, validating the minimization of the sim-to-real gap.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Traffic Control via Connected and Automated Vehicles (CAVs): An Open-Road Field Experiment with 100 CAVs

The CIRCLES project aims to reduce instabilities in traffic flow, which are naturally occurring phenomena due to human driving behavior. Also called “phantom jams” or “stop-and-go waves,” these instabilities are a significant source of wasted energy. Toward this goal, the CIRCLES project designed a control system, referred to as the MegaController by the CIRCLES team, that could be deployed in real traffic. Our field experiment, the MegaVanderTest (MVT), leveraged a heterogeneous fleet of 100 longitudinally controlled vehicles as Lagrangian traffic actuators, each of which ran a controller with the architecture described in this article. The MegaController is a hierarchical control architecture that consists of two main layers. The upper layer is called the Speed Planner and is a centralized optimal control algorithm. It assigns speed targets to the vehicles, conveyed through the LTE cellular network. The lower layer is a control layer, running on each vehicle. It performs local actuation by overriding the stock adaptive cruise controller, using the stock onboard sensors. The Speed Planner ingests live data feeds provided by third parties as well as data from our own control vehicles and uses both to perform the speed assignment. The architecture of the Speed Planner allows for the modular use of standard control techniques, such as optimal control, model predictive control (MPC), kernel methods, and others. The architecture of the local controller allows for the flexible implementation of local controllers. Corresponding techniques include deep reinforcement learning (RL), MPC, and explicit controllers. Depending on the vehicle architecture, all onboard sensing data can be accessed by the local controllers or only some. Likewise, control inputs vary across different automakers, with inputs ranging from torque or acceleration requests for some cars to electronic selection of adaptive cruise control (ACC) setpoints in others. The proposed architecture technically allows for the combination of all possible settings proposed previously, that is {Speed Planner algorithms} × {local Vehicle Controller algorithms} × {full or partial sensing} × {torque or speed control}. As a result, most configurations were tested throughout the ramp up to the MegaVandertest (MVT).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

A Step Towards Machine Learning-based Coherent Generator Grouping for Emergency Control Applications in Modern Power Grid

A dynamic neural network (NN) based multi-class classifier is proposed for improving online prediction of coherent generator groups (CGGs), following the occurrences of various contingencies in the power grid. This is motivated by the increasing availability of the measurements from phasor measurement units (PMUs) and the number of grouping schemes is limited. The proposed method consists of three steps. First, by performing offline simulations, a library of system dynamic responses characterized by post-contingency rotor angles and speeds of individual generators is obtained. To generate sufficient data, up to N-2 contingencies and the uncertain parameters associated with the power grid including type and location of disturbance and fault clearing times are modeled. Secondly, the training data-set is produced by generating labels for individual contingencies using a hierarchical clustering method based on rotor angle and speed data. Lastly, the dynamic NN models are trained for online applications such as emergency controls and controlled islanding. The proposed method is tested on the standard 16-generator 68-bus system to demonstrate its performance. Furthermore, the impact of the sample data lengths on the CGG numbers is evaluated. It is interesting to observe that the time domain stability behaviors can be determined by examining the changes in the CGG numbers.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Machine learning enables national assessment of wind plant controls with implications for land use

Summary As deployment of wind energy continues to expand, computationally efficient tools for predicting wind plant performance over a wide range of layout designs, technology innovations, and spatial locations are increasingly important for policy and investment decisions. We demonstrate two approaches to training a surrogate model to predict annual energy production (AEP) of parameterized wind plant layouts: one using a Gaussian process (GP) and the other using a fully convolutional neural network (FCNN). We leverage the powerful FCNN architecture by encoding wind plant design parameters and output response surface as an image. The FCNN produces more accurate results than the GP with mean absolute errors equivalent to 1% and 1.9% of plant rated power, respectively, although the GP performs well under limited training data and provides useful uncertainty information. We also evaluate a surrogate model for wake steering, enabling a nationwide assessment of the impact of plant control strategies and plant layout decisions. Across two million locations, we find that wake steering strategies boost AEP with relative gains upwards of 3%. Gains are most pronounced at sites without a dominant wind direction and where layout optimization is less fruitful. Additionally, we perform a nationwide sensitivity analysis showing that wake steering can mitigate wake losses from higher density plant layouts. Our results suggest that regions which have not been previously viable for wind deployment due to moderate wind resources are especially well enhanced by wake steering strategies that could help overcome land constraints and inflexible layout options, potentially identifying new deployment opportunities.

17 WIND ENERGY↗