Control of battery charging based on reinforcement learning and long short-term memory networks
Not provided.
Computer Science↗
SEARCH · Search NASA
Results for “Learning control”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Data-Driven Control Strategies for PDE Environments using Reinforcement Learning.
Abstract not provided.
Winovich, Nickolas↗
Controlling Information and Privacy Loss in Machine Learning Models with Coarseness
Explore the source record for details and available documents.
Outkin, Alexander V.↗
International Space Station Common Cabin Air Assembly Condensing Heat Exchanger Hydrophilic Coating Failures and Lessons Learned
The ability to control the temperature and humidity of an environment or habitat is critical for human survival. These factors are important to maintaining human health and comfort, as well as maintaining mechanical and electrical equipment in good working order to support the human and to accomplish mission objectives. The temperature and humidity of the International Space Station (ISS) United States On-orbit Segment (USOS) cabin air is controlled by the Common Cabin Air Assembly (CCAA). The CCAA consists of a fan, a condensing heat exchanger (CHX), an air/water separator, temperature and liquid sensors, and electrical controlling hardware and software. The CHX is the primary component responsible for control of temperature and humidity. The CCAA CHX contains a chemical coating that was developed to be hydrophilic and thus attract water from the humid influent air. This attraction forms the basis for water removal and therefore cabin humidity control. However, there have been several instances of CHX coatings becoming hydrophobic and repelling water. When this behavior is observed in an operational CHX, the unit s ability to remove moisture from the air is compromised and the result is liquid water carryover into downstream ducting and systems. This water carryover can have detrimental effects on the cabin atmosphere quality and on the health of downstream hardware. If the water carryover is severe and widespread, this behavior can result in an inability to maintain humidity levels in the USOS. This paper will describe the operation of the five CCAAs within in the USOS, the potential causes of the hydrophobic condition, and the impacts of the resulting water carryover to downstream systems. It will describe the history of this behavior and the actual observed impacts to the ISS USOS. Information on mitigation steps to protect the health of future CHX hydrophilic coatings and potential remediation techniques will also be discussed.
Balistreri, Steven F.↗
International Space Station Common Cabin Air Assembly Condensing Heat Exchanger Hydrophilic Coating Operation, Recovery, and Lessons Learned
The ability to control the temperature and humidity of an environment or habitat is critical for human survival. These factors are important to maintaining human health and comfort, as well as maintaining mechanical and electrical equipment in good working order to support the human and to accomplish mission objectives. The temperature and humidity of the International Space Station (ISS) United States On-orbit Segment (USOS) cabin air is controlled by the Common Cabin Air Assembly (CCAA). The CCAA consists of a fan, a condensing heat exchanger (CHX), an air/water separator, temperature and liquid sensors, and electrical controlling hardware and software. The CHX is the primary component responsible for control of temperature and humidity. The CCAA CHX contains a chemical coating that was developed to be hydrophilic and thus attract water from the humid influent air. This attraction forms the basis for water removal and therefore cabin humidity control. However, there have been several instances of CHX coatings becoming hydrophobic and repelling water. When this behavior is observed in an operational CHX in the ISS segments, the unit s ability to remove moisture from the air is compromised and the result is liquid water carryover into downstream ducting and systems. This water carryover can have detrimental effects on the ISS cabin atmosphere quality and on the health of downstream hardware. If the water carryover is severe and widespread, this behavior can result in an inability to maintain humidity levels in the USOS. This paper will describe the operation of the five CCAAs within the USOS, the potential causes of the hydrophobic condition, and the impacts of the resulting water carryover to downstream systems. It will describe the history of this behavior and the actual observed impacts to the ISS USOS. Information on mitigation steps to protect the health of future CHX hydrophilic coatings as well as remediation and recovery of the full heat exchanger will be discussed.
Balistreri, Steven F.↗
Machines Learning about Machines - ML for Analysis and Control of HPC Infrastructure.
Abstract not provided.
Wang, Felix↗
Offset Free Control Using Deep Deterministic Policy Gradient Reinforcement Learning Algorithm
Hedrick, E., K. Hedrick, D. Bhattacharyya, S.E. Zitney, and B.P. Omell, "Offset-Free Deep Deterministic Policy Gradient Reinforcement Learning Algorithm," AIChE 2022 Annual Meeting, Phoenix, AZ, November 13-18 (2022).
Hedrick, Elijah↗
Agent-Based System for Transactive Control of Smart Residential Neighborhoods
Buildings consume 74% of the total electricity produced in the United States. A significant portion of the building electric load includes heating ventilation and air-conditioning (HVAC) systems and water heating (WH) systems. Enabling flexibility in the operations can improve overall electric grid efficiency. This paper describes a multi-agent system for supporting integration, learning, optimization, and control of HVAC and WH in supporting a future smart grid. The architecture supports a transactive-based negotiation strategy between homeowners and a microgrid controller to adjust consumption behavior and reduce electricity costs. The framework is deployed in a neighborhood and preliminary testing is underway. The agent architecture design is discussed along with the preliminary optimization results from the demonstration site.
Starke, Michael↗
Predicting Pulsed-Laser Deposition SrTiO 3 Homoepitaxy Growth Dynamics Using High-Speed Reflection High-Energy Electron Diffraction
Pulsed-laser deposition (PLD) is a powerful technique for growing complex oxides with controlled stoichiometry. To understand growth dynamics therein, it is common to leverage in situ spectroscopies, such as reflection high-energy electron diffraction (RHEED), to monitor surface crystallinity. Most commercial systems rely on video-rate cameras operating at 60-120 Hz that lack sufficient temporal resolution to capture growth dynamics at practical deposition frequencies. Here, a high-speed platform to record in situ dynamics via RHEED at >500 Hz is implemented. An open-source analysis package is designed to fit diffraction spots to 2D Gaussians, allowing single-pulse surface reconstruction kinetics extraction. Using homoepitaxially deposited (001)-oriented SrTiO 3 as a model system, we demonstrate how high-speed RHEED can provide real-time insight into growth processes obscured by slower acquisition systems. By fitting the single-pulse intensity to a set of exponential functions, we observe changes in the characteristic decay time and mechanism correlated to the substrate step width and surface termination. We observe distinct surface effects, with diffraction intensity decaying on lower-energy TiO 2 -terminated surfaces and stabilizing on SrO- or mixed-terminated surfaces. Similarly, using an exponential model, the extracted characteristic time of adatom deposition decreases with increased density of bonding sites associated with mixed termination and narrower step widths. Ultimately, this work shows how increasing RHEED temporal resolution can uncover new insights into growth processes, with practical implications for the design and control of PLD processes. This experimental platform provides new capabilities to enable data-driven machine learning analysis and autonomous control systems to enhance the complexity and fecundity of PLD.
(SrO)↗
Off-policy deep reinforcement learning with automatic entropy adjustment for adaptive online grid emergency control
Electric overloading conditions and contingencies put modern power systems at risk of voltage collapse and blackouts. Load shedding is crucial to maintain voltage stability for grid emergency control. However, the rule- or model-based schemes rely on accurate dynamic system models and face considerable challenges in adapting to various operating conditions and uncertain event occurrences. Here, to address these issues, this paper proposes a novel deep reinforcement learning (DRL)-based voltage stability control algorithm with automatic entropy adjustment (AEA) for grid emergency control. Various dynamic network components for complex system operations are modeled to construct the DRL environment. An off-policy soft actor-critic architecture is developed to maximize the expected reward and policy entropy simultaneously. The AEA mechanism is proposed to facilitate the policy maximum entropy procedure, and the proposed method can automatically provide effective discrete and continuous actions against various fault scenarios. Our approach accomplishes high sampling efficiency, scalability, and auto-adaptivity of the control policies under high uncertainties. Comparative studies with the existing DRL-based control methods in IEEE benchmarks indicate salient performance improvement of the proposed method for dynamic system emergency control.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Reinforcement Learning‐Based Adaptation of Grid Following Inverter's Internal Controller to Networked Microgrids' Strengths
The varying topological configurations, generator commitments and dispatches, and dynamic load demand lead to changing system's strengths during the operations of networked microgrids. When the system's strengths significantly change, the fixed control gains at large devices may result in unsatisfactory system performance; this necessitates the tuning of the control gains at large devices to adapt to the changing system's strengths. In this paper, observer-based reinforcement learning (RL) is utilised to automatically tune the proportional-integral (PI) gains of phase lock loop (PLL) controller of grid-following (GFL) inverters to adapt to the changing strengths of microgrids and networked microgrids. The RL agent in this framework augments an observer predicting system's strengths, from which the RL control policy will adjust accordingly to tune the PLL controller's gains towards the system's strengths. Also, to enhance the control performance, the recently introduced Barrier function-based RL framework is leveraged for the design of reward function to prevent the high frequency nadir. An operational 26 kV electric distribution system, which is modelled as networked microgrids, is used to illustrate the need and effectiveness of the proposed RL-tuned control.
frequency response↗
An Artificial-Intelligence and Machine-Learning-Based Methodology to Conduct Seemingly Strain-Controlled Fatigue Test in a Pressurized-Water-Reactor-Test-Loop-Autoclave, While Not Controlling the Strain
In general, low cycle fatigue analysis of pressurized water reactor (PWR) components, requires strain-controlled fatigue test data such as using strain versus life (ε–N) curves. Conducting strain-controlled fatigue tests under in-air conditions is not an issue. However, controlling strain in a PWR-test-loop-autoclave is a challenge, since an extensometer cannot be placed in a narrow autoclave (typically used in a high-temperature-pressure PWR-test-loop). This is due to lack of space inside an autoclave that houses the test specimen. In addition, installing a contact-type extensometer in the path of a high-pressure flow can be a challenge. These difficulties of using an extensometer inside an autoclave led us to use an outside-autoclave displacement sensor which measures the displacement of pull-rod-specimen assembly. However, in our study (based on in-air fatigue test data), we found that a pull-rod-controlled based fatigue test can lead to substantial cyclic hardening/softening resulting in substantially different cyclic strain amplitudes and their rates compared to the desired cyclic strain amplitudes and its rates. In this paper, we propose an Artificial-Intelligence and Machine-Learning based technique such as using k-means clustering technique to improve the pull-rod-control based fatigue test method, such that the gage-area strain amplitude and rates can reasonably be achieved. In support of this, we present the fatigue test results for both 316 SS base and 81/182 dissimilar-metal-weld specimens.
42 ENGINEERING↗
Development of a Countermeasure to Enhance Postflight Locomotor Adaptability
Astronauts returning from space flight experience locomotor dysfunction following their return to Earth. Our laboratory is currently developing a gait adaptability training program that is designed to facilitate recovery of locomotor function following a return to a gravitational environment. The training program exploits the ability of the sensorimotor system to generalize from exposure to multiple adaptive challenges during training so that the gait control system essentially learns to learn and therefore can reorganize more rapidly when faced with a novel adaptive challenge. We have previously confirmed that subjects participating in adaptive generalization training programs using a variety of visuomotor distortions can enhance their ability to adapt to a novel sensorimotor environment. Importantly, this increased adaptability was retained even one month after completion of the training period. Adaptive generalization has been observed in a variety of other tasks requiring sensorimotor transformations including manual control tasks and reaching (Bock et al., 2001, Seidler, 2003) and obstacle avoidance during walking (Lam and Dietz, 2004). Taken together, the evidence suggests that a training regimen exposing crewmembers to variation in locomotor conditions, with repeated transitions among states, may enhance their ability to learn how to reassemble appropriate locomotor patterns upon return from microgravity. We believe exposure to this type of training will extend crewmembers locomotor behavioral repertoires, facilitating the return of functional mobility after long duration space flight. Our proposed training protocol will compel subjects to develop new behavioral solutions under varying sensorimotor demands. Over time subjects will learn to create appropriate locomotor solution more rapidly enabling acquisition of mobility sooner after long-duration space flight. Our laboratory is currently developing adaptive generalization training procedures and the associated flight hardware to implement such a training program during regular inflight treadmill operations. A visual display system will provide variation in visual flow patterns during treadmill exercise. Crewmembers will be exposed to a virtual scene that can translate and rotate in six-degrees-of freedom during their regular treadmill exercise period. Associated ground based studies are focused on determining optimal combinations of sensory manipulations (visual flow, body loading and support surface variation) and training schedules that will produce the greatest potential for adaptive flexibility in gait function during exposure to challenging and novel environments. An overview of our progress in these areas will be discussed during the presentation.
Bloomberg, Jacob J.↗
Towards Learning-Based Architectures for Sensor Impact Evaluation in Building Controls
Advanced control algorithms for building systems are known to have significant potential in reducing energy consumption while optimizing thermal comfort. The success of such algorithms is critically contingent on several different types of sensor systems, which are in turn, used for continuous monitoring, identification and estimation of several important building states, such as temperatures, humidity, air quality, power consumption and occupancy status. Nonidealities in any of these sensors can lead to significant performance degradation of the control functionalities, and may lead to unwanted sub-optimal building operation. In this paper, we provide a simulation example with a high-fidelity building model, for a particular use-case of advanced optimization-based control in buildings, i.e., occupancy-based controls. We show how imperfections in occupancy sensing can offset performance. Subsequently, we discuss a novel learning-based architecture to efficiently evaluate the impact of sensor nonidealities for building systems, in context of advanced control algorithms.
Bhattacharya, Saptarshi↗
Physics and Control of Locked Modes in the DIII-D Tokamak
This Final Technical Report summarizes an investigation, carried out under the auspices of the DOE Early Career Award, of the physics and control of non-rotating magnetic islands (“locked modes”) in tokamak plasmas. Locked modes are one of the main causes of disruptions in present tokamaks, and could be an even bigger concern in ITER, due to its relatively high beta (favoring the formation of Neoclassical Tearing Mode islands) and low rotation (favoring locking). For these reasons, this research had the goal of studying and learning how to control locked modes in the DIII-D National Fusion Facility under ITER-relevant conditions of high pressure and low rotation. Major results included: the first full suppression of locked modes and avoidance of the associated disruptions; the demonstration of error field detection from the interaction between locked modes, applied rotating fields and intrinsic errors; the analysis of a vast database of disruptive locked modes, which led to criteria for disruption prediction and avoidance.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Automatic learning rate adjustment for self-supervising autonomous robot control
Described is an application in which an Artificial Neural Network (ANN) controls the positioning of a robot arm with five degrees of freedom by using visual feedback provided by two cameras. This application and the specific ANN model, local liner maps, are based on the work of Ritter, Martinetz, and Schulten. We extended their approach by generating a filtered, average positioning error from the continuous camera feedback and by coupling the learning rate to this error. When the network learns to position the arm, the positioning error decreases and so does the learning rate until the system stabilizes at a minimum error and learning rate. This abolishes the need for a predetermined cooling schedule. The automatic cooling procedure results in a closed loop control with no distinction between a learning phase and a production phase. If the positioning error suddenly starts to increase due to an internal failure such as a broken joint, or an environmental change such as a camera moving, the learning rate increases accordingly. Thus, learning is automatically activated and the network adapts to the new condition after which the error decreases again and learning is 'shut off'. The automatic cooling is therefore a prerequisite for the autonomy and the fault tolerance of the system.
Arras, Michael K.↗
Towards Autonomous Lunar Resource Excavation via Reinforcement Learning
To continue on a sustainable and flexible path, NASA needs to address the challenge of collecting and moving large amounts of regolith at the destination. NASA’s Regolith Advanced Surface Systems Operations Robot (RASSOR) is principally designed to mine and deliver regolith for In-Situ Resource Utilization (ISRU) processing. RASSOR’s design enables it to efficiently collect and deposit regolith, return collected material for processing, and myriad related ISRU activities. To reliably perform these operations on the lunar surface, RASSOR software and sensory systems need to be robust and maximize the information extracted from a reduced sensor payload. Herein, we present preliminary findings from the Intelligent Capabilities Enhanced RASSOR project. We created reduced-order simulation environments to develop autonomous trenching controllers via reinforcement learning and prototype state estimation architectures. The goal of reinforcement learning is for an agent to learn a policy (task strategy) through interactions with an environment. When the agent performs an action, a change occurs in environment state and a numerical reward is received which informs the agent whether the action performed was good or not. Since reinforcement learning algorithms learn through trial-and-error, a simulation is a desirable first environment for development and learning. We developed two simulations, the first is a 2D excavation simulation developed to facilitate parameter selection, and a 3D simulation developed using a game physics engine, to simulate simplified soil interactions and increase the fidelity of the dynamic models of the robotic agents. The development of this 3D simulation has enabled the training of additional sensing capabilities and research both at the granular mechanics and operations levels. We experimented with various virtual sensor payloads to identify a combination that enabled efficient excavation operation and learning. Our reward function is based on how much material is excavated per step. A penalty is also received for leaving the dig site and to smooth the acceleration of the drum arms. We implemented pseudo time-of-flight sensors to report distance from each drum to ground and the height above ground which was found to be more efficient than existing solutions. Our findings suggest that reinforcement learning for autonomous operations has learned viable trenching strategies within 3000 training episodes in our simplified 2D environment and helped identify desirable sensing capabilities, arrangements, and considerations such as the positioning of time-of-flight sensors. Future work includes expanding our simulation to more complex environments and scenarios, and transfer learning from simulation to RASSOR 2.0 hardware for deployment in the Regolith Test Bin at NASA's Kennedy Space Center.
rassor↗