Search NASA⌕ Search
Home/Archive/DOE OSTI

DOE OSTI · 2246633

Global-Local Policy Search and its Application in Grid-Interactive Building Control

Zhang, Xiangyu·Chen, Yue·Bernstein, Andrey
View original source ↗Export metadata ↓

Abstract

As the buildings sector represents over 70% of the total U.S. electricity consumption, it offers a great amount of untapped demand-side resources to tackle many critical grid-side problems and improve the overall energy system's efficiency. To help make buildings grid-interactive, this paper proposes a global-local policy search method to train a reinforcement learning (RL) based controller which optimizes building operation during both normal hours and demand response (DR) events. Experiments on a simulated five-zone commercial building demonstrate that by adding a local fine-tuning stage to the evolution strategy policy training process, the control costs can be further reduced by 7.55% in unseen testing scenarios. Baseline comparison also indicates that the learned RL controller outperforms a pragmatic linear model predictive controller (MPC), while not requiring intensive online computation.

Explore related subjects

demand responseENERGY CONSERVATION, CONSUMPTION, AND UTILIZATIONgrid-interactive building controlreinforcement learning

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Zhang, Xiangyu, Chen, Yue, Bernstein, Andrey. 2023-05-04. Global-Local Policy Search and its Application in Grid-Interactive Building Control. https://www.osti.gov/biblio/2246633

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

DOE OSTI2024

Dynamically Learning Incentives for Load Control

As electrical generation becomes more distributed and volatile, and loads become more uncertain, controllability of distributed energy resources (DERs), regardless of their ownership status, will be necessary for grid reliability. Grid operators lack direct control over end-users' grid interactions, such as energy usage, but incentives can influence behavior -- for example, an end-user that receives a grid-driven incentive may adjust their consumption or expose relevant control variables in response. A key challenge in studying such incentives is the lack of data about human behavior, which usually motivates strong assumptions, such as distributional assumptions on compliance or rational utility-maximization. In this paper, we propose a general incentive mechanism in the form of a constrained optimization problem -- our approach is distinguished from prior work by modeling human behavior (e.g., reactions to an incentive) as an arbitrary unknown function. We propose feedback-based optimization algorithms to solve this problem that each leverage different amounts of information and/or measurements. We show that each converges to an asymptotically stable incentive with (near)-optimality guarantees given mild assumptions on the problem. Finally, we evaluate our proposed techniques in voltage regulation simulations on standard test beds. We test a variety of settings, including those that break assumptions required for theoretical convergence (e.g., convexity, smoothness) to capture realistic settings. In this evaluation, our proposed algorithms are able to find near-optimal incentives even when the reaction to an incentive is modeled by a theoretically difficult (yet realistic) function.

demand response↗
DOE OSTI2024

VPP Participation Models: What is the Objective?

Virtual power plants (VPPs) can provide grid services by aggregating and controlling many distributed energy resources. However, their design, operation, level of participation, and outcomes depend on the regulatory and market environments they operate in, DER owner preferences, grid service value, and technical capabilities (e.g., communications, forecasting, controls, user interfaces). Although VPPs are inherently complex, they could enable high shares of behind-the-meter solar plus storage and reduce the cost of electrifying, e.g., personal vehicles and space heating.

demand response↗
DOE OSTI2023

The Empirical Effect of Fleet Optimization on Synchronization and Rebound Effects in Heat Pump Water Heaters

Demand response is a growing concept in light of the internet of things and an increasing need for grid flexibility. Water heaters are one of the preferred devices for providing demand response for grid services and peak management due to their capability to store energy. The efficient use of water heaters for demand response requires consideration of the associated load effects such as synchronization of device schedules and rebound effect. These effects present a significant challenge. Despite the importance of the mentioned effects for water heater queuing and scheduling, there has been no effort to quantify and empirically validate their impact. This study attempts to address this gap by offering two methods - Ward clustering and Euclidean K-means - to evaluate the extent of synchronization in a fleet of 42 water heaters in Atlanta, GA. Using the aforementioned methods on the measured data, we find evidence of convergence of water heater loads as a result of optimization compared to an idle period and analyzed their impact.

demand response↗