Wang, Jiyu | Search NASA

DOE OSTI2024

Deep Reinforcement Learning for Microgrid Cost Optimization Considering Load Flexibility

This paper proposes a novel Soft-Actor-Critic (SAC) based Deep Reinforcement Learning (DRL) method for optimizing the cost of microgrid operation by leveraging load flexibility. The proposed SAC-DRL method is designed to coordinate the control of distributed energy resources (DERs) and flexible load, addressing practical energy billing formation by power distribution utilities. Key contributions include an innovative reward function to mitigate sparse reward challenges and a mixed control strategy for discrete and continuous variables, ensuring radial network topology and minimizing power loss. We evaluate the proposed method on the model of a real microgrid located in Southern California, U.S.. The SAC-DRL model is tested to demonstrate its efficacy in reducing grid dependence, optimizing resource use, and minimizing costs. The results highlight the potential of DRL in modern energy systems, offering a sustainable and economically efficient solution for energy management in microgrids.

deep reinforcement learning↗

DOE OSTI2024

Simulation-Based Analysis of Feeder Operation with Different PV Smart Inverter Functions on an Actual Distribution System: Preprint

High penetration of photovoltaics (PV) in distribution feeders can cause problems, such as overvoltage, reverse power flow, and large net load changes. Traditional voltage regulation devices, such as capacitors and voltage regulators, can solve some of these problems but might have some delays. Today, smart inverters are gradually being used to provide voltage regulation and frequency support in distribution systems. Different smart inverter settings have been recommended in various rules and standards; however, the potential benefits and their impacts on distribution system operation are not well compared and studied. This paper presents a comparison of different smart inverter settings as applied to a distribution system. An actual feeder model from San Diego Gas & Electric Company is used to conduct the simulation. Additionally, a load disaggregation method is proposed to disaggregate the load and PV profile for each load location using advanced metering infrastructure net load measurements. Then, different smart inverter settings are applied to the PV systems in the feeder, and the simulation results are compared. The results show that the implementation of specific functions of smart inverters can reduce voltage exceedances, and the utility can determine the specific inverter setting based on its operational requirements.

distribution system↗

DOE OSTI2024

Optimal Electric Vehicle Charging and Discharging Strategies Under DER Compensation Programs: Preprint

The adoption of electric vehicles (EVs) is becoming increasingly popular because of environmental concerns, the greater availability of models, and increased cost-competitiveness with gas vehicles. Because EVs have both charging and discharging capabilities, they provide great potential to help electric utilities with grid operation. When the grid demand is high, EVs can discharge to the grid to reduce the peak load, and vice versa; therefore, electric utilities have designed different policies to encourage EV charging station operators to charge or discharge at certain time periods. The New York State Public Service Commission established the Value of Distributed Energy Resources (VDER), or the Value Stack, to compensate for energy created by distributed energy resources, including EVs. This paper presents an optimization-based approach to identify the "golden hours" and "golden spots," i.e., the effective time periods and geographic locations for EV charging station operators to charge or discharge under the VDER program that can provide them the highest benefit. The proposed methodology can be applied to other compensation mechanisms and distribution systems as well. By working with industry partner NineDot Energy, realistic charging station information is used in this study, and the proposed approach is tested on a distribution feeder. The results from this study can help electric utilities and EV charging station operators determine the ideal charging/discharging time and the ideal locations for the charging station(s) in their distribution systems to achieve maximized benefit.

electric vehicle↗

DOE OSTI2024

Learning-Based Building Flexibility Estimation and Control to Improve Microgrid Economics and Resilience: Preprint

This paper proposes a learning-based building flexibility estimation and control framework to improve system economics and resilience. A data-driven building load flexibility model consisting of weather forecasting and estimating load consumption is proposed to quantify building heating, ventilation, and air conditioning (HVAC) load flexibility. A reinforcement learning-based microgrid controller is proposed to dispatch distributed generators, distributed energy resources, and build HVAC loads while taking flexibility information as one of the inputs. Simulation analysis is conducted on the model of a real microgrid in California. The effectiveness of the proposed learning-based building flexibility estimation and control in reducing microgrid energy costs and improving the sustainability of critical loads is demonstrated.

building load flexibility↗

DOE OSTI2024

Study of eVTOL Charging Impact on Airport Electrical Grids

Growth in commercial air traffic has increased the aviation industry's contribution to global emissions. Aircraft electrification can reduce greenhouse gas emissions along with being efficient and reliable. Advanced air mobility introduces the idea of an advanced and more efficient air transportation system. Electric and hybrid aircraft designs, such as electric vertical takeoff and landing (eVTOL), are being considered for a number of applications, such as urban air mobility, cargo delivery, and medical emergencies. Although the electrification of aircraft provides environmental benefits, their charging needs will place significant demand on the electrical grid. This work presents a grid impact study and infrastructure upgrade requirements analysis of eVTOL charging on a realistic distribution grid through power flow simulation analysis. The case study results show that the eVTOL charging can lead to power quality issues, such as undervoltage and equipment capacity overloading. Grid infrastructure, such as lines and transformers, would need to be upgraded, and voltage management strategies would need to be deployed to accommodate eVTOL charging at an existing airport.

advanced air mobility (AAM)↗

DOE OSTI2024

Commercialization of Distribution System Load Modeling Tool for Improved DER Interconnection Studies

Residential and commercial buildings have huge potential to contribute value to improve grid resilience by participating grid services. To reveal the significant value, it is critical to estimate the grid service capability from these buildings. Unlike the large-scale distributed energy resources such as wind and solar farms, those buildings need to participate grid services in aggregation, not by individual. Therefore, it is important to appropriately group buildings for aggregation. The load profiles in the same group will have similar characteristics at the same time step, so grid operators can send the grid service signal to the customer group with a higher chance to respond at that time step. In this paper, we develop a load profile clustering method to classify the building-level load profiles for grid service capability estimation. In our two-step clustering approach, we first calculate the total load consumption for each building, clustering the load profiles based on energy consumption level. Then, we further cluster the load profiles in each energy cluster based on the load shape. The parameter selection for each clustering step is discussed. The proposed method is applied on actual building-level load profiles, and the results have proved the effectiveness of this method.

14 SOLAR ENERGY↗

DOE OSTI2023

Federal Aviation Administration Vertiport Electrical Infrastructure Study

In this detailed analysis, the authors assess the charging infrastructure needed for the deployment of advanced air mobility involving electrified vertical take-off and landing technologies. The report covers four research areas: (1) Identifying charging infrastructure requirements for existing facilities based on flight operational parameters, potential use cases, charging strategy, and other constraints. (2) Assessing sites on power availability to meet charging demand, the impact on grid infrastructure, potential hazards, and cybersecurity needs, and using technoeconomic analysis to identify opportunities for onsite distributed energy resources, primarily solar photovoltaics and battery energy storage systems. (3) Calculating greenhouse gas emission based on total energy consumption attributable to each site. (4) Analyzing the job and economic development impact for sites adopting new infrastructure.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2023

Two-Stage Deep Reinforcement Learning for Distribution System Voltage Regulation and Peak Load Management

The growing integration of distributed solar photovoltaic (PV) in distribution systems could result in adverse effects during grid operation. This paper develops a two-agent soft actor critic-based deep reinforcement learning (SAC-DRL) solution to simultaneously control PV inverters and battery energy storage systems for voltage regulation and peak demand reduction. The novel two-stage framework, featured with two different control agents, is applied for daytime and nighttime operations to enhance control performance. Comparison results with other control methods on a real feeder in Western Colorado demonstrate that the proposed method can provide advanced voltage regulation with modest active power curtailment and reduce peak load demand from feeder's head.

deep reinforcement learning↗

DOE OSTI2023

Conservation Voltage Reduction with Distributed Energy Resource Management System, Grid-Edge, and Legacy Devices

Distribution utilities use conservation voltage reduction (CVR) to obtain energy savings and lower peak demand by reducing bus voltages. Traditionally, the CVR is accomplished by controlling the legacy assets such as load tap changers, voltage regulators, and capacitor banks. The deployment of the advanced distribution management system (ADMS) and distributed energy resource management system (DERMS) enables the integration of distributed energy resources into the distribution networks and provide the grid services including CVR. This paper studies the coordinated operation of an ADMS and a DERMS in achieving CVR and voltage regulation. A commercial ADMS uses legacy devices and Edge-of-Network Grid Optimization (ENGO) devices to obtain energy savings through CVR. A prototype DERMS dispatches the photovoltaic smart inverters based on real-time optimal power low to ensure voltage regulation across the feeder. The results show that the coordinated operation of ADMS and DERMS is effective in achieving CVR and voltage regulation. Specifically, energy savings of up to 4.7% are observed in the real utility distribution system used in this study.

advanced distribution management system↗

DOE OSTI2023

Multi-Agent Reinforcement Learning for Distribution System Critical Load Restoration

Grid resilience has become a critical topic recently because of the increasing occurrence of extreme events and the growing integration of intermittent renewable energy sources. To build a resilient distribution system, this paper develops a multiagent reinforcement learning-based (MARL) method to coordinate distribution energy resources (DERs) dispatch, load pickup, and network reconfiguration for load restoration after a system outage. With the help of two types of control agents, namely critical load restoration (CLR) and coordination (COR) agents, system loads can be restored efficiently, given available resources. The effectiveness and superiority of the proposed algorithm are demonstrated through simulations and comparative studies on a real distribution feeder in Western Colorado.

distribution system↗

DOE OSTI2023

A Two-Step Time-Series Data Clustering Method for Building-Level Load Profile

Residential and commercial buildings have huge potential to contribute value to improve grid resilience by participating grid services. To reveal the significant value, it is critical to estimate the grid service capability from these buildings. Unlike the large-scale distributed energy resources such as wind and solar farms, those buildings need to participate grid services in aggregation, not by individual. Therefore, it is important to appropriately group buildings for aggregation. The load profiles in the same group will have similar characteristics at the same time step, so grid operators can send the grid service signal to the customer group with a higher chance to respond at that time step. In this paper, we develop a load profile clustering method to classify the building-level load profiles for grid service capability estimation. In our two-step clustering approach, we first calculate the total load consumption for each building, clustering the load profiles based on energy consumption level. Then, we further cluster the load profiles in each energy cluster based on the load shape. The parameter selection for each clustering step is discussed. The proposed method is applied on actual building-level load profiles, and the results have proved the effectiveness of this method.

advanced metering infrastructure (AMI)↗

DOE OSTI2023

Demographic Information Incorporated Household Energy Consumption Analysis

The high energy consumption from residential buildings provides them large potential to participate in demand response programs. To design appropriate demand response programs for residential buildings, it is important for electric utilities to know the energy consumption characteristics for different types of households so that utilities can send requests to the groups with a higher possibility to successfully respond. In this paper, we develop a load model to generate synthetic load profiles for different types of households incorporating demographical information including Current Population Survey data set and American Time Use Survey data set. The details of each data set and the details of the load models are presented. The synthetic household load profiles are generated by the load model and clustered into different groups based on state, age, number of occupants, income level, and city of the household. The average energy consumption characteristics for different groups of households are analyzed and compared, which will help electric utilities issue demand response signals to appropriate households.

building loads↗

DOE OSTI2023

Demographic Information Incorporated Household Energy Consumption Analysis

The high energy consumption from residential buildings provides them large potential to participate in demand response programs. To design appropriate demand response programs for residential buildings, it is important for electric utilities to know the energy consumption characteristics for different types of households so that utilities can send requests to the groups with a higher possibility to successfully respond. In this paper, we develop a load model to generate synthetic load profiles for different types of households incorporating demographical information including Current Population Survey data set and American Time Use Survey data set. The details of each data set and the details of the load models are presented. The synthetic household load profiles are generated by the load model and clustered into different groups based on state, age, number of occupants, income level, and city of the household. The average energy consumption characteristics for different groups of households are analyzed and compared, which will help electric utilities issue demand response signals to appropriate households.

building loads↗

DOE OSTI2023

A Two-Step Time-Series Data Clustering Method for Building-Level Load Profile

Residential and commercial buildings have huge potential to contribute value to improve grid resilience by participating grid services. To reveal the significant value, it is critical to estimate the grid service capability from these buildings. Unlike the large-scale distributed energy resources such as wind and solar farms, those buildings need to participate grid services in aggregation, not by individual. Therefore, it is important to appropriately group buildings for aggregation. The load profiles in the same group will have similar characteristics at the same time step, so grid operators can send the grid service signal to the customer group with a higher chance to respond at that time step. In this paper, we develop a load profile clustering method to classify the building-level load profiles for grid service capability estimation and the results have proved the effectiveness of this method.

AMI↗

DOE OSTI2023

A Two-Step Time-Series Data Clustering Method for Building-Level Load Profile: Preprint

Residential and commercial buildings have huge potential to contribute value to improve grid resilience by participating grid services. To reveal the significant value, it is critical to estimate the grid service capability from these buildings. Unlike the large-scale distributed energy resources such as wind and solar farms, those buildings need to participate grid services in aggregation, not by individual. Therefore, it is important to appropriately group buildings for aggregation. In this paper, we develop a load profile clustering method to classify the building-level load profiles for grid service capability estimation. In our two-step clustering approach, we first calculate the total load consumption for each building, clustering the load profiles based on energy consumption level. Then, we further cluster the load profiles in each energy cluster based on the load shape. The parameter selection for each clustering step is discussed. The proposed method is applied on actual building-level load profiles, and the results have proved the effectiveness of this method.

advanced metering infrastructure (AMI)↗

DOE OSTI2023

Demographic Information Incorporated Household Energy Consumption Analysis: Preprint

The high energy consumption from residential buildings provides them large potential to participate in demand response programs. To design appropriate demand response programs for residential buildings, it is important for electric utilities to know the energy consumption characteristics for different types of households so that utilities can send requests to the groups with a higher possibility to successfully respond. In this paper, we develop a load model to generate synthetic load profiles for different types of households incorporating demographical information including Current Population Survey data set and American Time Use Survey data set. The details of each data set and the details of the load models are presented. The synthetic household load profiles are generated by the load model and clustered into different groups based on state, age, number of occupants, income level, and city of the household. The average energy consumption characteristics for different groups of households are analyzed and compared, which will help electric utilities issue demand response signals to appropriate households.

building loads↗

DOE OSTI2023

Conservation Voltage Reduction with Distributed Energy Resource Management System, Grid-Edge, and Legacy Devices

Distribution utilities use conservation voltage reduction (CVR) to obtain energy savings and lower peak demand by reducing bus voltages. Traditionally, the CVR is accomplished by controlling the legacy assets such as load tap changers, voltage regulators, and capacitor banks. The deployment of the advanced distribution management system (ADMS) and distributed energy resource management system (DERMS) enables the integration of distributed energy resources into the distribution networks and provide the grid services including CVR. This paper studies the coordinated operation of an ADMS and a DERMS in achieving CVR and voltage regulation. A commercial ADMS uses legacy devices and Edge-of-Network Grid Optimization (ENGO) devices to obtain energy savings through CVR. A prototype DERMS dispatches the photovoltaic smart inverters based on real-time optimal power low to ensure voltage regulation across the feeder. The results show that the coordinated operation of ADMS and DERMS is effective in achieving CVR and voltage regulation. Specifically, energy savings of up to 4.7% are observed in the real utility distribution system used in this study.

advanced distribution management system↗

DOE OSTI2023

Multi-Agent Reinforcement Learning for Distribution System Critical Load Restoration: Preprint

Grid resilience has become a critical topic recently because of the increasing occurrence of extreme events and the growing integration of intermittent renewable energy sources. To build a resilient distribution system, this paper develops a multi-agent reinforcement learning-based (MARL) method to coordinate distribution energy resources (DERs) dispatch, load pickup, and network reconfiguration for load restoration after a system outage. With the help of two types of control agents, namely critical load restoration (CLR) and coordination (COR) agents, system loads can be restored efficiently, given available resources. The effectiveness and superiority of the proposed algorithm are demonstrated through simulations and comparative studies on a real distribution feeder in Western Colorado.

distribution system↗