Deep reinforcement learning assisted co-optimization of Volt-VAR grid service in distribution networks

Hossain, Rakib; Gautam, Mukesh; Thapa, Jitendra; Livani, Hanif; Benidris, Mohammed

doi:10.1016/j.segan.2023.101086

Home/Archive/DOE OSTI

DOE OSTI · 2418735

Deep reinforcement learning assisted co-optimization of Volt-VAR grid service in distribution networks

Hossain, Rakib·Gautam, Mukesh·Thapa, Jitendra·Livani, Hanif·Benidris, Mohammed

View original source ↗Read source document ↗Export metadata ↓

Abstract

With the increasing penetration of distributed energy resources in distribution networks, Volt-VAR control and optimization (VVC/VVO) have become very important to ensure an acceptable quality of service to all customers. System operators can rely on slow-responding utility devices, including capacitor banks and on-load tap changing transformers, along with fast-responding battery and photovoltaic (PV) inverters for the VVC/VVO implementation. Because of variations in response time of these two classes of devices, and different control actions (discrete versus continuous), coordinated and optimal scheduling and operation have become of utmost importance. Here, this paper develops a look-ahead deep reinforcement learning (DRL)-based multi-objective VVO technique to improve the voltage profile of active distribution networks, decrease network and inverter power loss, and save the operational cost of the grid. It proposes a deep deterministic policy gradient (DDPG)-based approach to schedule the optimal reactive and/or active power set-points of fast-responding inverters, and a deep Q-network (DQN)-based DRL agent to schedule the discrete decisions variables of slow-responding assets. The reactive power output of PV and battery smart inverters are scheduled at 30-minute intervals and the capacitors’ commitment status is scheduled with several hour intervals. The proposed framework is validated on the modified IEEE 34-bus and 123-bus test cases with embedded PV and PV-plus-storage. To validate the efficacy of the proposed VVO, it is compared with several scenarios, including the base case without VVO, localized droop control of DERs, DDPG-only, and twin delayed DDPG (TD3) agent-based DRL techniques. The results justify the superior performance of the proposed method to improve the voltage profile, reduce network power loss, and minimize the look-ahead grid operational cost while minimizing the undesirable power losses in inverters as a result of power factor adjustments.

Explore related subjects

24 POWER TRANSMISSION AND DISTRIBUTION Deep Q network Deep deterministic policy gradient Distributed energy resources Distribution grids Volt-VAR optimization

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Hossain, Rakib, Gautam, Mukesh, Thapa, Jitendra, Livani, Hanif, Benidris, Mohammed. 2023-06-12. Deep reinforcement learning assisted co-optimization of Volt-VAR grid service in distribution networks. https://doi.org/10.1016/j.segan.2023.101086

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

DOE OSTI2027

Nodal capacity expansion planning with flexible large-scale load siting

We propose explicitly incorporating large-scale load siting into a stochastic nodal power system capacity expansion planning model that concurrently co-optimizes generation, transmission, and storage expansion. The potential operational flexibility of some of these large loads is also taken into account by considering them as consisting of a set of tranches with different reliability requirements, which are modeled as a constraint on expected served energy across operational scenarios. We implement our model as a two-stage stochastic mixed-integer optimization problem with cross-scenario expectation constraints. To overcome the challenge of scalability, we build upon existing work to implement this model on a high performance computing platform and exploit scenario parallelization using an augmented Progressive Hedging Algorithm. The algorithm is implemented using the bounding features of mpisppy, which have shown to provide satisfactory provable optimality gaps despite the absence of theoretical guarantees of convergence. We test our approach and assess the value of this proactive planning framework on total system cost and reliability metrics using realistic testcases geographically assigned to San Diego and South Carolina, with datacenter and direct air capture facilities as large loads.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2027

From zonal to nodal capacity expansion planning: Spatial aggregation impacts on a realistic test-case

Solving power system capacity expansion planning (CEP) problems at realistic spatial resolutions is computationally challenging. Thus, a common practice is to solve CEP over zonal models with low spatial resolution rather than over full-scale nodal power networks. Due to improvements in solving large-scale stochastic mixed integer programs, these computational limitations are becoming less relevant, and the assumption that zonal models are realistic and useful approximations of nodal CEP is worth revisiting. Here, this work is the first to conduct a systematic computational study on the assumption that spatial aggregation can reasonably be used for ISO-scale CEP. By considering a realistic, large-scale test network based on the state of California with over 8000 buses, we find that well-designed small spatial aggregations can yield good approximations but that coarser zonal models may result in large distortions of investment decisions, e.g., capacity under-investment of up to 41% for the lowest resolution model considered.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2026

Multi-facility analysis using metered power data to quantify MRI energy use and utility bill costs across scanner operating modes

This study quantifies the energy consumption of magnetic resonance imaging (MRI) scanners across discrete operating modes during routine clinical workflows, based solely on electrical power measurements. Although previous studies have investigated MRI energy consumption within single hospitals or specific clinical settings, this research provides a broader and more systematic analysis. Researchers analyzed electrical power data and applied a previously developed semi-automatic method for identifying MRI operating modes using load duration curves for 20 MRI scanners across four different U.S. healthcare facilities, encompassing outpatient, inpatient, and mixed-use clinical settings. A key innovation is the inclusion of localized hourly utility rates to estimate costs, a parameter absent in prior literature. Key findings indicate significant variability in energy and cost profiles between weekdays and weekends. Scanner characteristics, including magnet strength, manufacturer, vintage, location, and clinical setting, influenced average daily energy consumption and power thresholds for operating modes. Notably, the clinical setting of a scanner predominantly determines its energy use. For example, the scanners in outpatient facilities consumed more energy. The breakdown of energy usage and costs by operating modes showed scanners spend between 61% and 93% of their time in nonproductive modes, with one outlier spending 34%. Average daily energy use for the scanners in the study ranged from 160 to 1069 kWh, with energy costs ranging from $\$$9 to $\$$149. This study uses an existing framework to quantify MRI energy behavior, leading to insights that can enable improved performance and cost savings across different healthcare environments.

24 POWER TRANSMISSION AND DISTRIBUTION↗