Search NASA⌕ Search

Engineering topics

Zandi, Helia

Publications and source records attributed to Zandi, Helia.

At least 37 records · Page 2

Reinforcement-Learning-Based Smart Water Heater Control: An Actual Deployment

Utilizing smart control algorithms for electric water heaters (EWHs) is essential for fully harnessing the demand response (DR) potential of EWHs. For this reason, the use of reinforcement learning (RL) algorithms for EWHs has received increasing attention in recent years. However, existing RL approaches are either simulation-based or use pretrained RL agents. To this end, this paper presents the real-world deployment of a set of model-free RL approaches that aim to minimize the electricity cost of a EWH under a time-of-use electricity pricing policy using standard DR commands (e.g., shed, load up). The experiment results showed that the RL agents can help save electricity cost in the range of 11% to 14% compared to the baseline operation. This study demonstrated that RL-based EWH controllers can be deployed in real world without any prior training and can still save electricity cost.

deep learning↗

Virtual Infrastructure Twins: Software Testing Platforms for Computing-Instrument Ecosystems

Science ecosystems are being built by federating computing systems and instruments located at geographically distributed sites over wide-area networks. These computing-instrument ecosystems are expected to support complex workflows that incorporate remote, automated AI-driven science experiments. Their realization, however, requires various designs to be explored and software components to be developed, in order to support the orchestration of distributed computations and experiments. It is often too expensive, infeasible, or disruptive for the entire ecosystem to be available during the typically long software development and testing periods. We propose a Virtual Infrastructure Twin (VIT) of the ecosystem that emulates its network and computing components, and incorporates its instrument software simulators. It provides a software environment nearly identical to the ecosystem to support early development and testing, and design space exploration. We present a brief overview of previous digital infrastructure twins that culminated in the VIT concept, including (i) the virtual science network environment for developing software-defined networking solutions, and (ii) the virtual federated science instrument environment for testing the federation software stack and remote instrument control software. We briefly describe VITs for Nion microscope steering and access to GPU systems.

Rao, Nageswara↗

Evidence of Residential Demand Flexibility in a 46 Townhome Neighborhood

Demand response is an important emerging part of smart grids and it has been well researched from the theoretical and modeling perspectives. The empirical evidence on demand response is scarce, resulting in a limited understanding of many important aspects of demand response, including latency, cycling, and overrides - including acceptable impacts to customer comfort, convenience, and productivity in the new remote work era. We attempt to provide additional information to address this knowledge gap by sharing early results from the 46-townhome testbed located in Atlanta, GA. We report our findings from the first three months of experimental work. These focus on the delays associated with device status updates, characteristics of user overrides of control signals, and the share of devices available for demand response. We also provide some information about the duration of demand response events and a discussion of some properties of the testbed neighborhood.

Tsybina, Eve↗

Virtual Infrastructure Twins: Software Testing Platforms for Computing-Instrument Ecosystems

Science ecosystems are being built by federating computing systems and instruments located at geographically distributed sites over wide-area networks. These computing-instrument ecosystems are expected to support complex workflows that incorporate remote, automated AI-driven science experiments. Their realization, however, requires various designs to be explored and software components to be developed, in order to support the orchestration of distributed computations and experiments. It is often too expensive, infeasible, or disruptive for the entire ecosystem to be available during the typically long software development and testing periods. We propose a Virtual Infrastructure Twin (VIT) of the ecosystem that emulates its network and computing components, and incorporates its instrument software simulators. It provides a software environment nearly identical to the ecosystem to support early development and testing, and design space exploration. We present a brief overview of previous digital infrastructure twins that culminated in the VIT concept, including (i) the virtual science network environment for developing software-defined networking solutions, and (ii) the virtual federated science instrument environment for testing the federation software stack and remote instrument control software. We briefly describe VITs for Nion microscope steering and access to GPU systems.

Rao, Nageswara↗

Deep reinforcement learning with online data augmentation to improve sample efficiency for intelligent HVAC control

Deep Reinforcement Learning (DRL) has started showing success in real-world applications such as building energy optimization. Much of the research in this space utilized simulated environments to train RL-agent in an offline mode. Very few research have used DRL-based control in real-world systems due to two main reasons: 1) sample efficiency challenge---DRL approaches need to perform a lot of interactions with the environment to collect sufficient experiences to learn from, which is difficult in real systems, and 2) comfort or safety related constraints---user's comfort must never or at least rarely be violated. In this work, we propose a novel deep Reinforcement Learning framework with online Data Augmentation (RLDA) to address the sample efficiency challenge of real-world RL. We used a time series Generative Adversarial Network (TimeGAN) architecture as a data generator. We further evaluated the proposed RLDA framework using a case study of an intelligent HVAC control. With a ≈28% improvement in the sample efficiency, RLDA framework lays the way towards increased adoption of DRL-based intelligent control in real-world building energy management systems.

Kurte, Kuldeep↗

Seamlessly Fuel Flexible Heat Pump with Optimal Model-based Control Strategies to Reduce Peak Demand, Utility Cost and CO2 Emission

This research develops a novel hybrid fuel heat pump system for space heating of residential and small commercial buildings with built-in optimization and control. Whereas conventional dual fuel systems either run on gas or electricity at any given moment, the proposed seamlessly fuel flexible heat pump (SFFHP) simultaneously consumes gas and electricity and continuously optimizes the proportion of each. The building air flows across the heat pump condenser first and then flows across the furnace coil, and this reduces the heat pump temperature lift. The SFFHP delivers energy savings by allowing each subsystem (gas furnace and electric heat pump) to operate where it performs best to improve energy efficiency, minimize energy cost, and minimize carbon footprint. The capacities of the electric heat pump and gas furnace are continuously adjusted based on ambient conditions, utility price signals, and marginal grid emission signals. An optimal model predictive control strategy was developed with the goal of minimizing utility cost and minimizing CO2 emission. Two case studies were conducted to simulate the performance of SFFHP during the heating season in Chicago and Los Angeles, respectively. Compared with a conventional electric heat pump, SFFHP yields 33% utility cost reduction and 49% CO2 emission reduction in Chicago. Similarly, it achieves 23% utility cost reduction and 17% CO2 emission reduction in Los Angeles. Case studies demonstrate that SFFHP can deliver significant reductions in peak demand, utility cost, and CO2 emission. Due to the hybrid fuel nature of this novel equipment, user comfort will always be maintained. The fuel flexibility makes it an attractive option for demand response programs.

Li, Zhenning↗

Hybrid Heat Pump Controls: Conventional Dual Fuel versus Seamlessly Fuel Flexible Heat Pump

This paper compares the performance of a novel seamlessly fuel flexible heat pump (SFFHP) and conventional dual fuel heat pump (DFHP) for space heating. The conventional dual fuel systems either run on the gas furnace or electric heat pump at any given moment, as a comparison, the proposed seamlessly fuel flexible heat pump simultaneously consumes gas and electricity by continuously optimizing the proportion of each. The process air flows across the heat pump condenser first and then flows across the furnace coil, therefore, the heat pump temperature lift is reduced. SFFHP delivers energy savings by allowing each subsystem, i.e., gas furnace and electric heat pump, to operate where it performs best.For DFHP, two operation control strategies, i.e., non-restricted control and restricted control, are available on market. For the non-restricted mode, the thermostat has a switching temperature-programmed according to the balancing point of heating load and capacity curve. Heat pump operates above the switching temperature, while the furnace takes over under the switching temperature. For the restricted control, the compressor of a heat pump is disabled below a predefined lockout outdoor temperature to let the furnace take over. For SFFHP, a model predictive control strategy is developed to continuously adjust the capacities of the electric heat pump and gas furnace based on the foreseen weather data, utility price signals, and marginal grid emission signals with the goal of minimizing the utility cost and CO2 emission while guaranteeing comfort requirements.In this paper, DFHP and SFFHP are simulated using high-fidelity heat pump performance curves generated from DOE/ORNL heat pump design model. Performance comparison of DFHP and SFFHP during 2019-2020 heating season in Los Angeles shows that SFFHP with model predictive control achieves 23% utility cost reduction and 17 % CO2 emission reduction. Case studies demonstrate that SFFHP can deliver significant reductions in peak demand, utility cost, and CO2 emission. As a result, SFFHP can deliver superior benefits for utility cost reduction and CO2 emission reduction over conventional dual fuel heat pump.

Li, Zhenning↗

Demonstration of Intelligent HVAC Load Management With Deep Reinforcement Learning: Real-World Experience of Machine Learning in Demand Control

We report that buildings account for 40% of total primary energy consumption and 30% of all CO 2 emissions worldwide. A large portion of building energy consumption is due to heating, ventilation, and air-conditioning (HVAC) systems. In the summer, for example, more than 50% of a building’s electricity consumption is used for cooling. With proper energy management, buildings can provide load shifting, peak shaving, frequency regulation, and many other demand response services.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

COVID-19 pandemic ramifications on residential Smart homes energy use load profiles

The COVID-19 pandemic has significantly affected people’s behavioral patterns and schedules because of stay-at-home orders and a reduction of social interactions. Therefore, the shape of electrical loads associated with residential buildings has also changed. In this paper, we quantify the changes and perform a detailed analysis on how the load shapes have changed, and we make potential recommendations for utilities to handle peak load and demand response. Our analysis incorporates data from before and after the onset of the COVID-19 pandemic, from an Alabama Power Smart Neighborhood with energy-efficient/smart devices, using around 40 advanced metering infrastructure data points. This paper highlights the energy usage pattern changes between weekdays and weekends pre– and post–COVID-19 pandemic times. The weekend usage patterns look similar pre– and post–COVID-19 pandemic, but weekday patterns show significant changes. We also compare energy use of the Smart Neighborhood with a traditional neighborhood to better understand how energy-efficient/smart devices can provide energy savings, especially because of increased work-from-home situations. HVAC and water heating remain the largest consumers of electricity in residential homes, and our findings indicate an even further increase in energy use by these systems.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Comparative analysis of model-free and model-based HVAC control for residential demand response

In this paper, we present a comparative analysis of model-free reinforcement learning (RL) and model predictive control (MPC) approaches for intelligent control of heating, ventilation, and air-conditioning (HVAC). Deep-Q-network (DQN) is used as a candidate for model-free RL algorithm. The two control strategies were developed for residential demand-response (DR) HVAC system. We considered MPC as our golden standard to compare DQN's performance. The question we tried to answer through this work was, What % of MPC's performance can be achieved by model-free RL approach for intelligent HVAC control?. Based on our test result, RL achieved an average of ≈ 62% daily cost saving of MPC. Considering the pure optimization and model-based nature of MPC methods, the RL showed very promising performance. We believe that the interpretations derived from this comparative analysis provide useful insights to choose from various DR approaches and further enhance the performance of the RL-based methods for building energy managements.

Kurte, Kuldeep↗

A System of Agents for Supporting Optimization and Control of a Connected Community

The residential sector consumes a significant portion of the electricity sold in the United States. Above 60% of the energy used in the sector is used to operate heating, ventilation, and air conditioning (HVAC) systems and water heating (WH) systems. With the increase of intelligence in the grid and the new decision and control options enabled by the Internet of Things; control of these devices can be used to support the grid. Therefore, this article presents a scalable multiagent system for optimizing HVAC and WH systems while maintaining comfort. It allows a utility to orchestrate the shifting of energy from critical periods without direct control, but instead by using a price signal. The architecture, optimization formulation, implementation strategy and results from an implementation project are discussed.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Impact of Connected Communities

Buildings account for 35% of CO 2 emissions and almost 40% of the United States’ energy use. High-performance homes and neighborhoods play an important role in supporting efforts to decarbonize the US power system by 2035. Significant reductions in CO 2 emissions within the residential sector can be realized through electrification of loads paired with the flexibility created by leveraging smart Internet of Things (IoT) capabilities to shift energy use based on grid signals, thus improving generation/distribution efficiency and maximizing the use of renewable generation capacity. All of this can be achieved while allowing smart home appliances and equipment to meet homeowner needs – including reducing power bills - while optimizing operation in conjunction with the grid using novel control techniques. The Grid-Interactive Efficient Buildings Roadmap by the US Department of Energy’s (DOE’s) Building Technologies Office (BTO) notes that implementing grid-interactive efficient building (GEB) technology has the potential to reduce CO 2 emissions by 80 million tons/year—roughly equivalent to 17 million cars. To achieve this vision, the US Department of Energy’s Oak Ridge National Laboratory (ORNL)—in collaboration with Southern Company Research & Development, Alabama Power, Georgia Power, BTO and the US DOE’s Office of Electricity (OE) —is developing and demonstrating novel connected communities at two locations. Southern Company in turn engaged with industry partners, including design firms, residential developers, and residential HVAC and appliance manufacturers because their participation would be critical to the success of the initial research project, as well as the future scaling to the Southern Company service territory and beyond. Impacts of the Connected Communities projects in Alabama and Georgia are outlined including: energy, grid services and data management learnings; homeowner feedback; vendor engagement; adoption by utilities; technical, policy and business model challenges.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Approximating Nash Equilibrium in Day-ahead Electricity Market Bidding with Multi-agent Deep Reinforcement Learning

In this paper, a day-ahead electricity market bidding problem with multiple strategic generation company (GEN-CO) bidders is studied. The problem is formulated as a Markov game model, where GENCO bidders interact with each other todevelop their optimal day-ahead bidding strategies. Considering unobservable information in the problem, a model-free and data-driven approach, known as multi-agent deep deterministic policy gradient (MADDPG), is applied for approximating the Nash equilibrium (NE) in the above Markov game. The MADDPG algorithm has the advantage of generalization due to the automatic feature extraction ability of the deep neural networks. The algorithm is tested on an IEEE 30-bus system with three competitive GENCO bidders in both an uncongested caseand a congested case. Comparisons with a truthful bidding strategy and state-of-the-art deep reinforcement learning methods including deep Q network and deep deterministic policy gradient (DDPG) demonstrate that the applied MADDPG algorithm can find a superior bidding strategy for all the market participants with increased profit gains. In addition, the comparison with a conventional model-based method shows that the MADDPG algorithm has higher computational efficiency, which is feasible for real-world applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Methodology for interpretable reinforcement learning model for HVAC energy control

Deep reinforcement learning (DRL) approaches have been used in various application areas to improve efficiency, optimization, or automation. However, very little is known about how the DRL algorithms make decisions and what features affect their performance. Using a case study of a DRL based Heating, Ventilation and Air Conditioning (HVAC) optimization methodology, we demonstrate how we can address these challenges by applying interpretability tools and systematically exploring the model inputs for better understanding the DRL behaviour and decision making process. We developed a methodology for interpretable reinforcement learning and evaluated our approach in real-world house located in Knoxville, TN. Our findings explain the reasoning behind DRL-based optimization decisions under different circumstances which has been discussed and confirmed by the experts in the field.

Kotevska, Olivera↗

A Real-time Agent Based Optimization and Control Approach for Residential Building Heating Ventilation and Air Conditioning Systems

The prevalence of the loT (Internet of Things) is fostering the development of new control options and decision making that was not previously available. This is creating a wealth of opportunities for real-time control approaches and systems that can optimize for a common goal. This paper presents a smart residential neighborhood with optimization at the residential level utilizing a system of agents. The optimization utilizes information and modeling to optimize variable speed HVAC real-time operation in actual residential buildings. Data is presented showing the performance of the optimization and conclusions are drawn on next steps for development.

Hall, Joni↗

Double Deep Q-Networks for Optimizing Electricity Cost of a Water Heater

Electric water heaters represent 14% of the electricity consumption in the residential buildings and the cost associated with domestic water heating account for a good portion of the household expenses in the United States. In this context, intelligent control of water heaters gained a lot of research attention. In recent years, a significant number of intelligent water heater controllers, with various methods and intended uses, have been proposed. However, existing studies are mostly model-based approaches that require an accurate modelling of the water heater. Towards addressing this research gap, this paper presents a model-free reinforcement learning-based controller for a day-ahead price market. The controller aims to minimize the cost of domestic water heating while maintaining the user comfort. The results showed that the developed controller can help save energy cost while maintaining the temperatures within the desired comfort band.

Amasyali, Kadir↗

Data Analysis Approach for Large Data Volumes in a Connected Community

Recent advancements within smart neighborhoods where utilities are enabling automatic control of appliances such as heating, ventilation, and air conditioning (HVAC) and water heater (WH) systems are providing new opportunities to minimize energy costs through reduced peak load. This requires systematic collection, storage, management, and in-memory processing of large volumes of streaming data for fast performance. In this paper, we propose a multi-tier layered IoT software framework that enables effective descriptive and predictive data analysis for understanding live operation of the neighborhood, fault identification, and future opportunities for further optimization of load curves. We then demonstrate how we achieve live situational awareness of the connected neighborhood through a suite of visualization components. Finally, we discuss a few analytic dashboards that address questions such as peak load reductions obtained due to optimization, customer preference for automatic control of appliances (do they override the automatic control of HVAC?, etc.). 1 1 This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a nonexclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Chinthavali, Supriya↗