Search NASA⌕ Search

Engineering topics

Ni, Zhen

Publications and source records attributed to Ni, Zhen.

4 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A Modified Maximum Entropy Inverse Reinforcement Learning Approach for Microgrid Energy Scheduling

Increasing popularity of integrating distributed energy resources (DERs) into the power system brings a challenge to optimize the microgrid dispatch policy. The reinforcement learning methods suffer from a long-time problem with the theoretical assumption of the objective/reward function for the microgrid system. Although the traditional inverse reinforcement learning (IRL) approaches can solve this problem to some extent, they encounter a limitation of complex computations for state visitation frequency in the large and continuous state space. To alleviate this limitation, we propose a modified maximum entropy IRL (MMIRL) method to extract the reward function from the expert demonstrations for solving the microgrid energy scheduling problem. The proposed MMIRL algorithm is promising in recovering the reward function and learning the dispatch policy compared to conventional approaches. Case studies are performed in an energy arbitrage problem and a microgrid system with DERs. Results substantiate that the proposed MMIRL approach can learn the dispatch policy with more than 99% efficiency and outperforms other comparative methods.

artificial intelligence, reinforcement learning, m↗
DOE OSTI2023

Microgrid energy scheduling under uncertain extreme weather: Adaptation from parallelized reinforcement learning agents

Microgrids are useful solutions for integrating renewable energy resources and providing seamless green electricity to minimize carbon footprint. In recent years, extreme weather events happened often worldwide and caused significant economic and societal losses. Such events bring uncertainties to the microgrid energy scheduling problems and increase the challenges of microgrid operation. Traditional optimization approaches suffer from the inaccuracy of the uncertain microgrid model and the unseen events. Existing reinforcement learning (RL) - based approaches are also hampered by the limited generalization and the increasing computational burden when stochastic formulations are required to accommodate the uncertainties. This paper proposes a new parallelized reinforcement learning (PRL) method based on the probabilistic events to handle the microgrid energy uncertainties. Specifically, several local learning agents are employed to interact with pertinent microgrid environments in a distributed manner and report outcomes to the global agent, which will optimize microgrid energy resources online during extreme events. The stochastic microgrid energy optimization problem is reformulated to include all possible scenarios with probabilities. The advantage estimate functions of learning agents are designed with a backward sweep to transfer the outcomes to the value function updating process. Two simulation studies, stochastic optimization and online testing, are performed to compare with several existing RL approaches. Results substantiate that the proposed PRL method can achieve up to 20% optimization performance improvement with 4 and 28 times less computation cost than Q-learning with experience replay and multi-agent Q-learning approaches, respectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

An Efficient Distributed Reinforcement Learning for Enhanced Multi-Microgrid Management

Economic dispatch in multi-microgrid (MMG) systems requires coordinating distributed energy resources (DERs) of different microgrids, which leads to a significant increase in the number of states for energy management. In these cases, traditional reinforcement learning (RL) approaches become computationally expensive or output a solution that causes extra-operating costs for the system. This paper proposes an RL approach that employs local learning agents to interact with microgrid environments in a distributed manner and aggregates the outcomes to train the global agent to learn the policy for the MMG system. This distributed exploration and aggregation process provides an effective solution and guides the global agent to learn the dispatch policy efficiently. Case studies are performed on a system with three microgrids with different types of DERs. Results obtained using the proposed RL and comparisons with conventional methods substantiate the effectiveness of the proposed approach in terms of operation costs, computation time, and peak-to-average ratio.

Das, Avijit↗