Search NASA⌕ Search

Engineering topics

Kurte, Kuldeep

Publications and source records attributed to Kurte, Kuldeep.

34 records · Page 2

Methodology for interpretable reinforcement learning model for HVAC energy control

Deep reinforcement learning (DRL) approaches have been used in various application areas to improve efficiency, optimization, or automation. However, very little is known about how the DRL algorithms make decisions and what features affect their performance. Using a case study of a DRL based Heating, Ventilation and Air Conditioning (HVAC) optimization methodology, we demonstrate how we can address these challenges by applying interpretability tools and systematically exploring the model inputs for better understanding the DRL behaviour and decision making process. We developed a methodology for interpretable reinforcement learning and evaluated our approach in real-world house located in Knoxville, TN. Our findings explain the reasoning behind DRL-based optimization decisions under different circumstances which has been discussed and confirmed by the experts in the field.

Kotevska, Olivera↗

Double Deep Q-Networks for Optimizing Electricity Cost of a Water Heater

Electric water heaters represent 14% of the electricity consumption in the residential buildings and the cost associated with domestic water heating account for a good portion of the household expenses in the United States. In this context, intelligent control of water heaters gained a lot of research attention. In recent years, a significant number of intelligent water heater controllers, with various methods and intended uses, have been proposed. However, existing studies are mostly model-based approaches that require an accurate modelling of the water heater. Towards addressing this research gap, this paper presents a model-free reinforcement learning-based controller for a day-ahead price market. The controller aims to minimize the cost of domestic water heating while maintaining the user comfort. The results showed that the developed controller can help save energy cost while maintaining the temperatures within the desired comfort band.

Amasyali, Kadir↗

Smoky Mountain Data Challenge 2020: An Open Call to Solve Data Problems in the Areas of Neutron Science, Material Science, Urban Modeling and Dynamics, Geophysics, and Biomedical Informatics

The 2020 Smoky Mountains Computational Sciences and Engineering Conference enlists research scientists from across Oak Ridge National Laboratory (ORNL) to be data sponsors and help create data analytics challenges for eminent data sets at the laboratory. This work describes the significance of each of the seven data sets and their as- sociated challenge questions. The challenge questions for each data set were required to cover multiple difficulty levels. An international call for participation was sent to students, and researchers asking them to form teams of up to four people to apply novel data analytics techniques to these data sets.

Parete-Koon, Suzanne↗

Deep Reinforcement Learning for Residential HVAC Control with Consideration of Human Occupancy

The Artificial Intelligence (AI) development described herein uses model-free Deep Reinforcement Learning (DRL) to minimize energy cost during residential heating, ventilation, and air conditioning (HVAC) operation. Building cooling loads and HVAC operation are difficult to accurately model due to complexity, lack of measurements and data, and model specific performance, so online machine learning is used to allow for real-time readjustment in performance. Energy costs for the multi-zone cooling unit shown in this work are minimized by scheduling on/off commands around dynamic prices. By taking advantage of precooling events that take place when the price is low, the agent is able to reduce operational cost without violating user comfort. The DRL controller was tested in simulation where the learner achieved a 43.89% cost reduction when compared to traditional, fixed-setpoint operation. The system is now ready for the next phase of testing in a live, real-time home environment.

Mckee, Evan↗

Multi-task deep reinforcement learning for intelligent multi-zone residential HVAC control

In this short communication, a data-driven deep reinforcement learning (deep RL) method is applied to minimize HVAC users’ energy consumption costs while maintaining users’ comfort. The applied deep RL method's efficiency is enhanced by conducting multi-task learning that can achieve an economic control strategy for a multi-zone residential HVAC system in both cooling and heating scenarios. The applied multi-task deep RL method is compared with a rule-based benchmark case and a single-task deep deterministic policy gradient algorithm to verify its effective and generalized application in optimizing HVAC operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Residential heating, ventilation, and air conditioning (HVAC) has been considered as an important demand response resource. However, the optimization of residential HVAC control is no trivial task due to the complexity of the thermal dynamic models of buildings and uncertainty associated with both occupant-driven heat loads and weather forecasts. In this paper, we apply a novel model-free deep reinforcement learning (RL) method, known as the deep deterministic policy gradient (DDPG), to generate an optimal control strategy for a multi-zone residential HVAC system with the goal of minimizing energy consumption cost while maintaining the users’ comfort. Here, the applied deep RL-based method learns through continuous interaction with a simulated building environment and without referring to any prior model knowledge. Simulation results show that compared with the state-of-art deep Q network (DQN), the DDPG-based HVAC control strategy can reduce the energy consumption cost by 15% and reduce the comfort violation by 79%; and when compared with a rule-based HVAC control strategy, the comfort violation can be reduced by 98%. In addition, experiments with different building models and retail price models demonstrate that the well-trained DDPG-based HVAC control strategy has high generalization and adaptability to unseen environments, which indicates its practicability for real-world implementation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Electricity Pricing aware Deep Reinforcement Learning based Intelligent HVAC Control

Recently, deep reinforcement learning (DRL) based intelligent control of Heating, Ventilation, and Air Conditioning (HVAC) has gained a lot of attention due to DRL's ability to optimally control HVAC for minimizing operational cost while maintaining resident's comfort. The success of such DRL-based techniques largely depends on the articulation of the problem in terms of states, actions, and reward function. Inclusion of the electricity pricing information in the problem formulation can play an important role in saving the cost of HVAC operation. However, less attention has been given in the literature on formulating well-crafted state features based on electricity pricing. In this work, we propose an approach for training the DRL model with a specific focus on feature engineering based on electricity pricing. During training, we generate random but sufficiently realistic electricity price signals so that the pre-trained DRL model is robust and adaptive to the dynamic and variable electricity prices. The validation results are encouraging and show the potential of ≈12%-15% savings in the one day cost of HVAC operation, proving the usefulness of including electricity pricing related features as state features.

Kurte, Kuldeep↗

RL-HEMS: Reinforcement Learning Based Home Energy Management System for HVAC Energy Optimization

Heating, ventilation, and air conditioning (HVAC) is one of the major energy consumers in the residential sector. It is important to be able to monitor and control the energy consumed to provide utility services such as load shaping while satisfying the comfort and economic constraints of the homeowner. The objective of this work is to create the optimal schedule for HVAC operation to reduce the cost while satisfying the home-owner and equipment’s constraints using a model-free Reinforcement Learning (RL)-based optimization. The specific goal is to find the right balance between reducing energy cost, consumption, and customer comfort level. Our research effort addresses this optimization problem using multiple components: the development of initial learning test-bed and implementation of RL techniques on a real home. This will enable the rapid evaluation of the RL techniques and provide an early baseline to train before implementation on site. The RL algorithm is designed to learn the energy use patterns and generate the optimized schedule for HVAC within an acceptable time-interval to satisfy the homeowner’s comfort and minimize the energy usage. Our preliminary results are promising and we have observed a 17% reduction in the total cost and a 15% reduction in the power utilization using our RL-based HVAC model—RL-HEMS.

Kotevska, Olivera↗

A Mobility-Driven Approach to Modeling Building Energy

Buildings are one of the primary energy consumers in any city's energy use [12]. Presence and absence of humans is a major contributing factor to the energy use in a building. In this paper, we present an approach to generating a realistic model of human building occupancy throughout a typical work week. We use the Toolbox for Urban Mobility Systems (TUMS) to generate a synthetic population based on population distribution estimates, we schedule the population's daily commute based on National Household Travel Survey (NHTS) survey data, and we simulate their daily travel patterns using an agent-based transportation simulation (TRANSIMS). We process and fuse the simulation output to produce a list of the first and last seen location of each agent in the simulation. Based on the arrival at the last destination, we map each agent to one of the nearby buildings. Using these agent arrivals, as well as NHTS data, we create an hourly occupancy schedule for each building. We successfully demonstrate this workflow at the example of the Chicago Loop, a major business district in Chicago, Illinois.

Berres, Andy↗

Semantics-enabled Spatio-Temporal Modeling of Earth Observation Data: An application to Flood Monitoring

Extreme events such as urban floods are dynamic in nature, i.e. they evolve with time. The spatiotemporal analysis of such disastrous events is important for understanding the resiliency of an urban system during these events. Remote Sensing (RS) data is one of the crucial earth observation (EO) data sources that can facilitate such spatiotemporal analysis due to its wide spatial coverage and high temporal availability. In this paper, we propose a discrete mereotopology (DM) based approach to enable representation and querying of spatiotemporal information from a series of multitemporal RS images that are acquired during a flood disaster event. We represent this spatiotemporal information using a semantic model called Dynamic Flood Ontology (DFO). To establish the effectiveness and applicability of the proposed approach, spatiotemporal queries relevant during an urban flood scenario such as, show me road segments that were partially flooded during the time interval t1 have been demonstrated with promising results.

Kurte, Kuldeep↗

Regional-scale Spatio-Temporal Analysis of Impacts of Weather on Traffic Speed in Chicago using Probe Data

Understanding a regional-scale impact of the weather on the transportation system and how that impact varies geographically, is important from a sustainability standpoint. In this work, we have performed a city-scale analysis on the impact of weather on the traffic speed for the City of Chicago. We have found that there is a significant variation in the average hourly speed due to different weather patterns which also varies geographically. We have also observed that the rainfall has an evident impact on the average hourly speed on the freeways that are influenced by urbanized residential and commercial areas, and non-urbanized areas. Also, low visibility has shown a significant reduction in the average hourly traffic speed during the congested hours on the freeways that are influenced by non urbanized areas. We anticipate the contributions of this work in estimating emission and fuel economy at a regional scale which are important sustainability measures for transportation.

Kurte, Kuldeep↗

High Performance Geocomputation for Assessing Human Dynamics at Planet Scale

Settlements are key indicators of human presence on the landscape. Large scale mapping of human settlements and their morphology from very high-resolution satellite images is a critical step towards developing an interpretative understanding of population distribution and the sociocultural attributes of the built environment they live in. Convolutional neural network (CNN) based Deep Learning experiments indicates that such computations can be scaled to some of the largest high-performance computing (HPC) architectures. While early results are encouraging for developing settlement and corresponding population maps at unprecedented speed and spatial resolutions, characterizing human dynamics at planet scale with high temporal resolution will require the community to develop novel geocomputational infrastructures and ecosystems.

Bhaduri, Budhu↗

Semantic Framework for Spatial Query Reformulation for Disaster Monitoring Applications

In disasters, since time is of the essence, quick decision making based on actionable insights is desired. In our earlier work, we have demonstrated that the spatial relationships-based queries can play a vital role in the disaster response phase. However, we found that the utilization of spatial relationships rules (i.e. encoded spatial knowledge) via rule reasoning process do not scale well with the increased number of image regions. Most of the available Resource Description Framework (RDF) triplestores do not support rule reasoning due to the computational complexity and undecidable nature of the rule reasoning process. In this paper, we propose an alternative approach for utilizing spatial knowledge encoded in the form of spatial relationship rules. The proposed approach reformulates the spatial query by expanding it with the configuration encoded in the corresponding spatial relationship rule. The preliminary results are promising and show the applicability of the proposed approach during the time critical events such as flood disaster.

Kurte, Kuldeep↗

Performance analysis and optimization for scalable deployment of deep learning models for country-scale settlement mapping on Titan supercomputer

Here, we present a scalable object detection workflow for detecting objects, such as settlements, from remotely sensed (RS) imagery. We have successfully deployed this workflow on Titan supercomputer and utilized it for the task of mapping human settlement at a country scale. The performance of various stages in the workflow was analyzed before making it operational. The workflow implemented various strategies to address issues such as suboptimal resource utilization and long-tail effects due to unbalanced image workload, data loss due to runtime failures, and maximum wall-time constraints imposed by Titan's job scheduling policy. A mean shift clustering–based static load balancing strategy was implemented, which partitions the image load such that each partition contained similar-sized images. Furthermore, a checkpoint-restart strategy was added in the workflow as a fault-tolerance mechanism to prevent the data losses due to unforeseen runtime failures. The performance of the above-mentioned strategies was observed in various scenarios, such as node failure, exceeding wall time, and successful completion. Using this workflow, we have examined an RS data set that has a spatial resolution of 0.31 m and is comprised of 685 675 km 2 of area of the Republic of Zambia in under six hours using 5426 nodes of the Titan supercomputer.

97 MATHEMATICS AND COMPUTING↗