Search NASA⌕ Search

Engineering topics

Zhang, Baosen

Publications and source records attributed to Zhang, Baosen.

Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning: Preprint

We study multiagent reinforcement learning (MARL) with constraints. This setting is gaining importance as MARL algorithms find new applications in real-world systems ranging from power grids to drone swarms. Most constrained MARL (C-MARL) algorithms use a primal-dual approach to enforce constraints through a penalty function added to the reward. In this paper, we study the structural effects of the primal-dual approach on the constraints and value function. First, we show that using the constraint evaluation as the penalty leads to a weak notion of safety, but by making simple modifications to the penalty function, we can enforce meaningful probabilistic safety constraints. Second, we show that the penalty term changes the value function in a way that is easy to model, and demonstrate the consequences of not doing so. We conclude with simulations in a simple constrained multiagent environment to back up the theoretical results.

data-driven control↗

Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning

Constrained multiagent reinforcement learning (C-MARL) is gaining importance as MARL algorithms find new applications in real-world systems ranging from energy systems to drone swarms. Most C-MARL algorithms use a primal-dual approach to enforce constraints through a penalty function added to the reward. In this paper, we study the structural effects of this penalty term on the MARL problem. First, we show that the standard practice of using the constraint function as the penalty leads to a weak notion of safety. However, by making simple modifications to the penalty term, we can enforce meaningful probabilistic (chance and conditional value at risk) constraints. Second, we quantify the effect of the penalty term on the value function, uncovering an improved value estimation procedure. We use these insights to propose a constrained multiagent advantage actor critic (C-MAA2C) algorithm. Simulations in a simple constrained multiagent environment affirm that our reinterpretation of the primal-dual method in terms of probabilistic constraints is effective, and that our proposed value estimate accelerates convergence to a safe joint policy.

chance constraints↗

Transfer Learning for HVAC System Fault Detection

Faults in HVAC systems degrade thermal comfort and energy efficiency in buildings and have received significant attention from the research community, with data driven methods gaining in popularity. Yet the lack of labeled data, such as normal versus faulty operational status, has slowed the application of machine learning to HVAC systems. In addition, for any particular building, there may be an insufficient number of observed faults over a reasonable amount of time for training. To overcome these challenges, we present a transfer methodology for a novel Bayesian classifier designed to distinguish between normal operations and faulty operations. The key is to train this classifier on a building with a large amount of sensor and fault data (for example, via simulation or standard test data) then transfer the classifier to a new building using a small amount of normal operations data from the new building. We demonstrate a proof-of-concept for transferring a classifier between architecturally similar buildings in different climates and show few samples are required to maintain classification precision and recall.

transfer learning, Building HVAC, Bayesian framewo↗

Electrochemical Corrosion Behavior of Plasma-Sprayed FeCrNiMoCBSi Amorphous/Nanocrystalline Coatings in Simulated Seawater Medium

A novel FeCrNiMoCBSi coating was fabricated on F22 marine steel by plasma spraying. The coating microstructure and composition were characterized by scanning electron microscope, transmission electron microscope, x-ray diffraction, differential scanning calorimetry, and x-ray photoelectron spectroscopy. The results showed that the coating consisted of amorphous and nanocrystalline phases. Additionally, the corrosion resistance in simulated seawater was evaluated by electrochemical corrosion methods. It was found that the coating exhibited an excellent resistance to pitting corrosion. For long-time immersion tests, the corrosion resistance was gradually decreased with the increasing through-porosity due to the permeation of the corrosive solution into the coating. The coating lost its efficacy through three stages, namely the protection stage, the permeation stage, and the failure stage.

36 MATERIALS SCIENCE↗

Tribological Properties of Surface-Textured and Plasma-Nitrided Pure Titanium Under Oil Lubrication Condition

Plasma nitriding was conducted as post-treatment for surface texture on pure titanium to obtain a continuous nitriding layer. Supersonic fine particles bombarding (SFPB) was carried out to prepare surface texture. The surface morphologies and chemical composition were analyzed using scanning electron microscope and energy disperse spectroscopy. The microstructures of modified layers were characterized by transmission electron microscope. The tribological properties of surface-textured and duplex-treated pure titanium under oil lubrication condition were systematically investigated in the ball-on-plate reciprocating mode. The effects of applied load and sliding velocity on the tribological behavior were analyzed. The results show that after duplex treatments, the grains size in modified layer becomes slightly larger, and hardness is obviously improved. Wear resistance of duplex-treated pure titanium is significantly improved referenced to untreated and surface-textured pure titanium, which is 3.22 times as much as untreated pure titanium and 2.15 times of that for surface-textured pure titanium, respectively.

36 MATERIALS SCIENCE↗