Search NASA⌕ Search

Engineering topics

Munk, Jeffrey

Publications and source records attributed to Munk, Jeffrey.

At least 55 records · Page 3

Deep Reinforcement Learning for Residential HVAC Control with Consideration of Human Occupancy

The Artificial Intelligence (AI) development described herein uses model-free Deep Reinforcement Learning (DRL) to minimize energy cost during residential heating, ventilation, and air conditioning (HVAC) operation. Building cooling loads and HVAC operation are difficult to accurately model due to complexity, lack of measurements and data, and model specific performance, so online machine learning is used to allow for real-time readjustment in performance. Energy costs for the multi-zone cooling unit shown in this work are minimized by scheduling on/off commands around dynamic prices. By taking advantage of precooling events that take place when the price is low, the agent is able to reduce operational cost without violating user comfort. The DRL controller was tested in simulation where the learner achieved a 43.89% cost reduction when compared to traditional, fixed-setpoint operation. The system is now ready for the next phase of testing in a live, real-time home environment.

Mckee, Evan↗

Multi-task deep reinforcement learning for intelligent multi-zone residential HVAC control

In this short communication, a data-driven deep reinforcement learning (deep RL) method is applied to minimize HVAC users’ energy consumption costs while maintaining users’ comfort. The applied deep RL method's efficiency is enhanced by conducting multi-task learning that can achieve an economic control strategy for a multi-zone residential HVAC system in both cooling and heating scenarios. The applied multi-task deep RL method is compared with a rule-based benchmark case and a single-task deep deterministic policy gradient algorithm to verify its effective and generalized application in optimizing HVAC operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control

Recently, deep reinforcement learning (DRL) based intelligent control of Heating, Ventilation, and Air Conditioning (HVAC) has gained a lot of attention due to DRL's ability to optimally control HVAC for minimizing operational cost while maintaining resident's comfort. The success of such DRL-based techniques largely depends on the articulation of the problem in terms of states, actions, and reward function. Inclusion of the electricity pricing information in the problem formulation can play an important role in saving the cost of HVAC operation. However, less attention has been given in the literature on formulating well-crafted state features based on electricity pricing. In this work, we propose an approach for training the DRL model with a specific focus on feature engineering based on electricity pricing. During training, we generate random but sufficiently realistic electricity price signals so that the pre-trained DRL model is robust and adaptive to the dynamic and variable electricity prices. The validation results are encouraging and show the potential of ≈12%-15% savings in the one day cost of HVAC operation, proving the usefulness of including electricity pricing related features as state features.

Kurte, Kuldeep↗

Virtual storage capability of residential buildings for sustainable smart city via model-based predictive control

This paper evaluates the virtual storage capability of a residential air-conditioning (AC) system by utilizing the building mass as a thermal storage to enable sustainable cities through model-based predictive control (MPC). The control-oriented building model was developed with a grey-box model structure based on the experimental data to predict the indoor air temperature. Based on the model, the operation cost-saving potential of the MPC was investigated in a single house with different comfort levels, electricity prices, and prediction horizons. The cost-saving of the MPC was approximately 10.6 % compared to the conventional control. The MPC was expanded to a residential building cluster considering the peak demand charge. In an hourly shedding and load-up scenario, the performances of the MPC and the heuristic prior-based control (PBC) are promising. When shedding was enforced all day, the peak demand saving of the MPC was 36~38 %, whereas that of the PBC was 25.7 % compared to the baseline. In addition, the monthly electricity bill, including the operation and peak demand cost, was investigated with different demand charge rates and weather conditions. The total saving decreased with a low demand charge rate and hot conditions, and the operation cost saving was compromised with more aggressive demand shedding.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Aggregation and data driven identification of building thermal dynamic model and unmeasured disturbance

An aggregate model is a single-zone equivalent of a multi-zone building, and is useful for many purposes, including model based control of large heating, ventilation and air conditioning (HVAC) equipment. This paper deals with the problem of simultaneously identifying an aggregate thermal dynamic model and unknown disturbances from input–output data of multi-zone buildings. The unknown disturbance is a key challenge since it is not measurable but non-negligible. In this paper, we first present a principled method to aggregate a multi-zone building model into a single zone model, and show the aggregation is not as trivial as it has been assumed in the prior art. We then provide a method to identify the parameters of the model and the unknown disturbance for this aggregate (single-zone) model. Finally, we test our proposed identification algorithm to data collected from a multi-zone building testbed in Oak Ridge National Laboratory. A key insight provided by the aggregation method allows us to recognize under what conditions the estimation of the disturbance signal will be necessarily poor and uncertain, even in the case of a specially designed test in which the disturbances affecting each zone are known (as the case of our experimental testbed). This insight is used to provide a heuristic that can be used to assess when the identification results are likely to have high or low accuracy.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Battery Equivalent Model for Residential HVAC

Flexible loads, especially heating, ventilation, and air-conditioning (HVAC) systems can be used to provide a battery-like service to the power grid by varying their demand up and down over a baseline. Recent work has reported that the round-trip efficiency for providing grid services with HVAC systems can be close to 1. Inspired by this promising result, in this work, we rigorously develop a battery equivalent model (BEM) for residential HVAC systems. Relying only on typically available data, the proposed BEM will bridge the observability and controllability gaps between utility companies and customers. The proposed BEM will enable the utility companies to manage the flexible HVAC devices as easy as charging/discharging the conventional energy storage devices. Validated physics-based models are used to train the BEM, whose prediction accuracy is illustrated through a simulation study.

Dong, Jin↗

Development and Evaluation of Distributed Energy Resource Device Models: Electric Vehicles, Electric Water Heaters, and Commercial Refrigeration Systems

In order to validate distributed energy resource (DER) models operated with grid services in the GMLC 1.4.2 team’s February (GMLC 2019a) and July (GMLC 2019b) reports, a test and measurement program using actual DER devices was conducted by national laboratories for three devices: (1) electric vehicles (EVs), (2) water heaters, and (3) commercial refrigeration. Test procedures were developed and carried out to identify the mathematical models and their parameters that describe the operational function, characterize the physics, and obey transient response of the devices. This report focuses on the experimental results obtained to develop and verify simulations of three specific EV models. This section summarizes elements of the mathematical model and shows the necessity data collected from the EVs while charging and discharging (driving). In the course of the study, we found it necessary to make some modifications to the model to deal with observed transient behavior. Section 1.2 details the model assumptions and equations, Section 1.3 provides details of the EVs tested, Section 1.4 outlines the test procedures developed for the projects, and Section 1.5 presents the experimental results obtained for the three different EVs tested and how these results compare to the models running the same test profiles.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Scalable and Distributed Algorithm for Managing Residential Demand Response Programs using Alternating Direction Method of Multipliers (ADMM)

For effective engagement of residential demand-side resources and to ensure efficient operation of distribution networks, we must overcome the challenges of controlling and coordinating residential components and devices at scale. To overcome this challenge, we present a distributed and scalable algorithm with a three-level hierarchical information exchange architecture for managing the residential demand response programs. First, a centralized optimization model is formulated to maximize community social welfare. Then, this centralized model is solved in a distributed manner with alternating direction method of multipliers (ADMM) by decomposing the original problem to utility-level and house-level problems. The information exchange between the different layers is limited to the primary residual (i.e., supply-demand mismatch), Lagrangian multipliers, and the total load of each house to protect each customer’s privacy. Simulation studies are performed on the IEEE 33 bus test system with 605 residential customers. The results demonstrate that the proposed approach can save customers’ electricity bills and reduce the peak load at the utility level without much affecting customers’ comfort and privacy. Finally, a quantitative comparison of the distributed and centralized algorithms shows the scalability advantage of the proposed ADMM-based approach, and it gives benchmarking results with achievable value for future research works.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Power Grid Simulation Testbed for Transactive Energy Management Systems

To effectively engage demand-side and distributed energy resources (DERs) for dynamically maintaining the electric power balance, the challenges of controlling and coordinating building equipment and DERs on a large scale must be overcome. Although several control techniques have been proposed in the literature, a significant obstacle to applying these techniques in practice is having access to an effective testing platform. Performing tests at scale using real equipment is impractical, so simulation offers the only viable route to developmental testing at scales of practical interest. Existing power-grid testbeds are unable to model individual residential end-use devices for developing detailed control formulations for responsive loads and DERs. Furthermore, they cannot simulate the control and communications at subminute timescales. To address these issues, this paper presents a novel power-grid simulation testbed for transactive energy management systems. Detailed models of primary home appliances (e.g., heating and cooling systems, water heaters, photovoltaic panels, energy storage systems) are provided to simulate realistic load behaviors in response to environmental parameters and control commands. The proposed testbed incorporates software as it will be deployed, and enables deployable software to interact with various building equipment models for end-to-end performance evaluation at scale.

24 POWER TRANSMISSION AND DISTRIBUTION↗

An Automatic Learning Framework for Smart Residential Communities

Predictability has been foundational to matching supply and demand in the day-to-day operation of the electric power system. Demand predictability is eroding because of the increased use of renewable energy resources and more sophisticated loads, such as electric vehicles and smart appliances. In this paper, an automatic software framework is described which can be used for load forecasting in smart communities. A time-varying clustering-based Markov chain approach is used to predict the energy consumption of residential buildings in a smart community. The training data is based on 1-minute meter data of occupied homes over one month. The data points are first clustered based on the energy consumption and the time of the day. Then, the original data is converted using the Centroids of the clusters. A time-varying Markov chain is subsequently trained to model the energy consumption behavior of residents for each home using the transformed data. The trained model is shown to successfully predict load in 5-minute intervals over a 24 hours period.

Zandi, Helia↗

Smart technologies enable homes to be efficient and interactive with the grid

Oak Ridge National Laboratory researchers compare two different approaches to test how advanced, energy-efficient building technologies such as smart thermostats, heat pump water heaters, and advanced heat pump HVAC (heating, ventilation and air conditioning) can be optimized within a home and connected at a neighborhood-scale load to a community microgrid in the Alabama Power Smart Neighborhood located in Hoover. Working with Southern Company and Alabama Power, ORNL researchers are pioneering that future where smart homes and smart neighborhoods can benefit both homeowners and utilities, by reducing energy consumption by 44% and peak demand by 34%.This project is one of two neighborhoods in the U.S Department of Energy’s (DOE’s) Connected Neighborhood project, supported by Building Technologies Office , where ORNL researchers leverages DOE investment in micro-grids and responsive, flexible building loads research to improve grid reliability – a goal of DOE’s Grid-interactive Efficient Buildings (GEB) Initiative. Researchers control the neighborhood and microgrid to enable utilities achieve their desired load and cost profiles while ensuring the comfort of homeowners in the Smart Neighborhood. This transactive control approach maximizes the utilization of technical resources of the microgrid and controllable loads, while reducing costs for both the homeowners and Alabama Power. These tests partially seek to determine a more precise range of tolerance with respect to occupant comfort as researchers work to facilitate customer acceptance and perception of new building technologies that enable energy savings.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Real-Time MPC for Residential Building Water Heater Systems to Support the Electric Grid

The world of IoT (internet of things) is spawning new control options and decision making that has not been available before. This is providing a wealth of opportunities for transactive type controls and systems that can negotiate for a common goal. This paper discusses a transactive residential neighborhood with optimization at the home level utilizing a system of agents. The real-time optimization utilizes information and modeling to optimize heat pump water heater operation in occupied homes. Data is presented showing the performance of the optimization and conclusions are drawn on next steps for development.

Starke, Michael↗

RL-HEMS: Reinforcement Learning Based Home Energy Management System for HVAC Energy Optimization

Heating, ventilation, and air conditioning (HVAC) is one of the major energy consumers in the residential sector. It is important to be able to monitor and control the energy consumed to provide utility services such as load shaping while satisfying the comfort and economic constraints of the homeowner. The objective of this work is to create the optimal schedule for HVAC operation to reduce the cost while satisfying the home-owner and equipment’s constraints using a model-free Reinforcement Learning (RL)-based optimization. The specific goal is to find the right balance between reducing energy cost, consumption, and customer comfort level. Our research effort addresses this optimization problem using multiple components: the development of initial learning test-bed and implementation of RL techniques on a real home. This will enable the rapid evaluation of the RL techniques and provide an early baseline to train before implementation on site. The RL algorithm is designed to learn the energy use patterns and generate the optimized schedule for HVAC within an acceptable time-interval to satisfy the homeowner’s comfort and minimize the energy usage. Our preliminary results are promising and we have observed a 17% reduction in the total cost and a 15% reduction in the power utilization using our RL-based HVAC model—RL-HEMS.

Kotevska, Olivera↗

Identification of aggregate building thermal dynamic model and unmeasured internal heat load from data

This paper is on the problem of simultaneously identifying the parameters of an aggregate thermal dynamic model of a multi-zone building and unknown disturbances from input-output data. An aggregate model is a single-zone equivalent of a multi-zone building, and is useful for many purposes, including model based control of large heating, ventilation and air conditioning (HVAC) equipment that delivers thermal energy to the entire building. A key challenge in identification is the presence of unknown disturbance since it is not measurable but non-negligible.We first present a principled method to aggregate a multizone building model into a single zone model. We then provide a method to identify thermal parameters and the unknown disturbance for this aggregate (single-zone) model. Finally, we test our proposed identification algorithm to data generated from a virtual building. A key insight provided by the aggregation method allows us to recognize under what conditions the estimation of the disturbance signal will be necessarily poor and uncertain.

Guo, Zhong↗

Quantification of Energy Cost Savings through Optimization and Control of Appliances within Smart Neighborhood Homes

Electric utilities are driving towards enabling automatic scheduling and control of the consumption pattern of appliances such as heating, ventilation, and air conditioning (HVAC) and water heater (WH) systems (e.g., through preheating and pre-cooling, etc.) within smart neighborhoods to minimize energy cost and peak load demand. Quantifying economic savings through direct comparison of the optimized energy usage profile on a specific day with the typical non-optimized usage profile on another day is not a fair comparison because energy usage highly depends on weather conditions and human behaviour especially appliance like HVAC on those days. In this paper, we propose a novel approach of identifying similar weather day pairs which can then be used to compare the energy use profiles within homes between the identified pairs. We then demonstrate how the proposed approach can be used to compute cost savings due to optimization and control of smart appliances at home and neighborhood-level within a future-focused smart neighborhood of 62 residential homes. We also demonstrate a simulation based approach to quantify cost savings and showcase our findings through customized and interactive visualizations.

Chinthavali, Supriya↗

Load Flexibility Analysis of Residential HVAC and Water Heating and Commercial Refrigeration

The emergence of connected devices and the Internet of Things has opened the door to the concept of widespread use of building equipment for load-side flexibility. This stems from the increased use of distributed and clean energy resources for electric generation and the growing demand for a more resilient and reliable grid. A significant challenge associated with the use of building equipment as a distributed energy resource is quantifying the magnitude of power or energy flexibility. Unlike batteries that have well-defined and relatively constant capacity and maximum charge/discharge rates, building equipment often has characteristics that vary based on external factors (e.g., weather or occupant behavior). This makes quantifying the availability and flexibility of these resources difficult. Controlling a fleet of equipment while minimizing any adverse impacts to the occupants (e.g., discomfort or food spoilage) is a nontrivial matter. Although there are many different control methodologies available, this study focuses on two: priority-based control (PBC) and model-predictive control (MPC). These were selected because they represent two ends of the control spectrum. PBC is a relatively simple control strategy that requires only the current state of the devices to be controlled and an estimate of how much power each device consumes when turned on. MPC is much more complex and requires a model of the system and forecasts of any disturbances that influence the system. Because MPC has a predictive element, it can prepare for a desired response before the event and therefore is expected to provide a more optimal control relative to PBC. Both MPC and PBC are compared with a baseline simulation that uses feedback control typical of the thermostatically controlled devices being evaluated. This report investigates the load flexibility of three building systems: residential air conditioners, residential water heaters, and commercial refrigeration. Air heating and cooling and water heating account for 45% of residential electricity use, making them good candidates for providing load flexibility. Refrigeration in commercial buildings accounts for 16% of commercial building electricity use, making it another good candidate. Simplified models of each system were generated and validated based on measured data from actual equipment. These models were used to simulate the thermal and power response of the systems using PBC and MPC for a fleet of devices trying to achieve a variety of different responses, including hourly load shed, hourly load up, and daily peak demand reduction. Significant findings from each piece of equipment follow.

42 ENGINEERING↗

Agent-Based System for Transactive Control of Smart Residential Neighborhoods

Buildings consume 74% of the total electricity produced in the United States. A significant portion of the building electric load includes heating ventilation and air-conditioning (HVAC) systems and water heating (WH) systems. Enabling flexibility in the operations can improve overall electric grid efficiency. This paper describes a multi-agent system for supporting integration, learning, optimization, and control of HVAC and WH in supporting a future smart grid. The architecture supports a transactive-based negotiation strategy between homeowners and a microgrid controller to adjust consumption behavior and reduce electricity costs. The framework is deployed in a neighborhood and preliminary testing is underway. The agent architecture design is discussed along with the preliminary optimization results from the demonstration site.

Starke, Michael↗

Novel PCM Integration with Electrical Heat Pump for Demand Response

In this paper, we examine the potential for using phase change material (PCM) as a thermal storage medium for improving the efficiency as well as enabling grid-responsive control of heat pump (HP) systems. Previous research in PCM integration in buildings has primarily been focused on integration with building envelopes. An objective of our research is to design a novel and practical PCM integration with existing HP systems. High-fidelity models for building, HP and PCM are developed to facilitate the model-based predictive control (MPC) design. To optimize the operation of HP and PCM, we introduce an optimization problem considering various constraints for both comfort and hardware constraints. We include experimental results in which a house fleet are optimized to minimize their power consumption or electricity cost. We numerically demonstrate the designs potential for reducing power consumption or electricity cost by 13% or 19%, respectively.

Dong, Jin↗