Deep Reinforcement Learning for Resilient Power and Energy Systems: Progress, Prospects, and Future Avenues

Gautam, Mukesh

doi:10.3390/electricity4040020

Home/Archive/DOE OSTI

DOE OSTI · 2229634

Deep Reinforcement Learning for Resilient Power and Energy Systems: Progress, Prospects, and Future Avenues

Gautam, Mukesh

View original source ↗Read source document ↗Export metadata ↓

Abstract

In recent years, deep reinforcement learning (DRL) has garnered substantial attention in the context of enhancing resilience in power and energy systems. Resilience, characterized by the ability to withstand, absorb, and quickly recover from natural disasters and human-induced disruptions, has become paramount in ensuring the stability and dependability of critical infrastructure. This comprehensive review delves into the latest advancements and applications of DRL in enhancing the resilience of power and energy systems, highlighting significant contributions and key insights. The exploration commences with a concise elucidation of the fundamental principles of DRL, highlighting the intricate interplay among reinforcement learning (RL), deep learning, and the emergence of DRL. Furthermore, it categorizes and describes various DRL algorithms, laying a robust foundation for comprehending the applicability of DRL. The linkage between DRL and power system resilience is forged through a systematic classification of DRL applications into five pivotal dimensions: dynamic response, recovery and restoration, energy management and control, communications and cybersecurity, and resilience planning and metrics development. This structured categorization facilitates a methodical exploration of how DRL methodologies can effectively tackle critical challenges within the domain of power and energy system resilience. The review meticulously examines the inherent challenges and limitations entailed in integrating DRL into power and energy system resilience, shedding light on practical challenges and potential pitfalls. Additionally, it offers insights into promising avenues for future research, with the aim of inspiring innovative solutions and further progress in this vital domain.

Explore related subjects

24 POWER TRANSMISSION AND DISTRIBUTION 42 ENGINEERING communications and cybersecurity deep learning deep reinforcement learning dynamic response energy management and control power and energy system resilience resilience review

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Gautam, Mukesh. 2023-12-01. Deep Reinforcement Learning for Resilient Power and Energy Systems: Progress, Prospects, and Future Avenues. https://doi.org/10.3390/electricity4040020

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

DOE OSTI2027

Nodal capacity expansion planning with flexible large-scale load siting

We propose explicitly incorporating large-scale load siting into a stochastic nodal power system capacity expansion planning model that concurrently co-optimizes generation, transmission, and storage expansion. The potential operational flexibility of some of these large loads is also taken into account by considering them as consisting of a set of tranches with different reliability requirements, which are modeled as a constraint on expected served energy across operational scenarios. We implement our model as a two-stage stochastic mixed-integer optimization problem with cross-scenario expectation constraints. To overcome the challenge of scalability, we build upon existing work to implement this model on a high performance computing platform and exploit scenario parallelization using an augmented Progressive Hedging Algorithm. The algorithm is implemented using the bounding features of mpisppy, which have shown to provide satisfactory provable optimality gaps despite the absence of theoretical guarantees of convergence. We test our approach and assess the value of this proactive planning framework on total system cost and reliability metrics using realistic testcases geographically assigned to San Diego and South Carolina, with datacenter and direct air capture facilities as large loads.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2027

From zonal to nodal capacity expansion planning: Spatial aggregation impacts on a realistic test-case

Solving power system capacity expansion planning (CEP) problems at realistic spatial resolutions is computationally challenging. Thus, a common practice is to solve CEP over zonal models with low spatial resolution rather than over full-scale nodal power networks. Due to improvements in solving large-scale stochastic mixed integer programs, these computational limitations are becoming less relevant, and the assumption that zonal models are realistic and useful approximations of nodal CEP is worth revisiting. Here, this work is the first to conduct a systematic computational study on the assumption that spatial aggregation can reasonably be used for ISO-scale CEP. By considering a realistic, large-scale test network based on the state of California with over 8000 buses, we find that well-designed small spatial aggregations can yield good approximations but that coarser zonal models may result in large distortions of investment decisions, e.g., capacity under-investment of up to 41% for the lowest resolution model considered.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2026

Multi-facility analysis using metered power data to quantify MRI energy use and utility bill costs across scanner operating modes

This study quantifies the energy consumption of magnetic resonance imaging (MRI) scanners across discrete operating modes during routine clinical workflows, based solely on electrical power measurements. Although previous studies have investigated MRI energy consumption within single hospitals or specific clinical settings, this research provides a broader and more systematic analysis. Researchers analyzed electrical power data and applied a previously developed semi-automatic method for identifying MRI operating modes using load duration curves for 20 MRI scanners across four different U.S. healthcare facilities, encompassing outpatient, inpatient, and mixed-use clinical settings. A key innovation is the inclusion of localized hourly utility rates to estimate costs, a parameter absent in prior literature. Key findings indicate significant variability in energy and cost profiles between weekdays and weekends. Scanner characteristics, including magnet strength, manufacturer, vintage, location, and clinical setting, influenced average daily energy consumption and power thresholds for operating modes. Notably, the clinical setting of a scanner predominantly determines its energy use. For example, the scanners in outpatient facilities consumed more energy. The breakdown of energy usage and costs by operating modes showed scanners spend between 61% and 93% of their time in nonproductive modes, with one outlier spending 34%. Average daily energy use for the scanners in the study ranged from 160 to 1069 kWh, with energy costs ranging from $\$$9 to $\$$149. This study uses an existing framework to quantify MRI energy behavior, leading to insights that can enable improved performance and cost savings across different healthcare environments.

24 POWER TRANSMISSION AND DISTRIBUTION↗