SEARCH · Search NASA
Results for “Building control”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Towards Off-policy Evaluation as a Prerequisite for Real-world Reinforcement Learning in Building Control
We present an initial study of off-policy evaluation (OPE), a problem prerequisite to real-world reinforcement learning (RL), in the context of building control. OPE is the problem of estimating a policy's performance without running it on the actual system, using historical data from the existing controller. It enables the control engineers to ensure a new, pretrained policy satisfies the performance requirements and safety constraints of a real-world system, prior to interacting with it. While many methods have been developed for OPE, no study has evaluated which ones are suitable for building operational data, which are generated by deterministic policies and have limited coverage of the state-action space. After reviewing existing works and their assumptions, we adopted the approximate model (AM) method. Furthermore, we used bootstrapping to quantify uncertainty and correct for bias. In a simulation study, we evaluated the proposed approach on 10 policies pretrained with imitation learning. On average, the AM method estimated the energy and comfort costs with 1.84% and 14.1% error, respectively.
On-policy learning-based deep reinforcement learning assessment for building control efficiency and stability
Artificial intelligence technologies have emerged as a game changer not only in specific applications such as image recognition and machine translation but also in many scientific domains. In particular, as deep reinforcement learning (DRL) has shown great success in complex control problems, DRL-based control has been considered as a potential solution to efficiently control and manage building systems. However, broad assessment of DRL-based building control is still required to characterize their pros and cons in comparison with conventional building control methods (e.g., rule-based feedback controls). In this paper, we assessed DRL-based controls with on-policy learning-based algorithms and continuous control actions for cooling control of large office buildings in the summer season to minimize whole-building energy use and occupant discomfort. We compared DRL-based control methods with two baseline control methods: (1) a pre-determined schedule with supply temperature and static pressure setpoints, and (2) advanced reset method that adjusts setpoints based on heuristic rules, i.e., ASHRAE Guideline 36. We also tested the DRL algorithms to evaluate their performances in multiple climate locations. We found that DRL-based control methods outperformed the baseline control methods in terms of energy savings while maintaining a thermal comfort. DRL reduced energy use between ~4%–22% on average compared to the baseline methods, depending on climate location. We also evaluated DRL-based control in terms of control stability and showed that DRL-based methods should address the span of hardware lifetimes in practical operations.
A guideline to document occupant behavior models for advanced building controls
The availability of computational power, and a wealth of data from sensors have boosted the development of model-based predictive control for smart and effective control of advanced buildings in the last decade. More recently occupant-behavior models have been developed for including people in the building control loops. However, while important objectives of scientific research are reproducibility and replicability of results, not all information is available from published documents. Therefore, the aim of this paper is to propose a guideline for a thorough and standardized occupant-behavior model documentation. For that purpose, the literature screening for the existing occupant behavior models in building control was conducted, and the occupant behavior modeling processes were studied to extract practices and gaps for each of the following phases: problem statement, data collection, and preprocessing, model development, model evaluation, and model implementation. Here, the literature screening pointed out that the current state-of-the-art on model documentation shows little unification, which poses a particular burden for the model application and replication in field studies. In addition to the standardized model documentation, this work presented a model-evaluation schema that enabled benchmarking of different models in field settings as well as the recommendations on how OB models are integrated with the building system.
BOPTest As a Platform for Building Controls and Grid-Interactive Buildings Workforce Training
Building automation and controls are becoming increasingly complex with the emergence of Grid Integrated Efficient Buildings (GEBs) as well as new highly efficient sequences of operation and data-driven control schemes. However, there remains a significant gap in hands-on training opportunities for building operators and technicians to gain practical experience with advanced control systems in a low-risk environment. This paper presents BOPTEST (Building Optimization Performance Test) as a suitable platform for workforce training in building controls and GEB technologies. BOPTEST provides a suite of standardized building simulation test cases with a REST API, real-time control interfaces through BACnet, semantic models connecting users to building data, and built-in calculation of control metrics and performance indicators. The platform enables trainees to interact with virtual buildings using industry-standard protocols while learning how to implement and innovate control strategies. The training platform is designed to offer a structured and interactive learning experience for building engineers, helping them effectively develop, learn, and retain skills in fault identification, troubleshooting, and correction. The workflow is divided into three main phases: 1) Setup, 2) Exercise, and 3) Review, each comprising specific activities performed by either the instructor or the student. Initial pilot training sessions have yielded positive feedback from instructors and participants and demonstrates that BOPTEST effectively fills an industry need for a low-risk training resource via simulation of real building control systems, allowing trainees to gain practical experience before working in the field. The platform's ability to provide immediate performance feedback while maintaining familiar industry interfaces makes it particularly suitable for workforce development programs. This work provides a replicable model for leveraging building simulation in control education and training.
BOPTEST as a Platform for Building Controls and Grid-Interactive Buildings Workforce Training
Building automation and controls are becoming increasingly complex with the emergence of Grid Integrated Efficient Buildings (GEBs) as well as new highly efficient sequences of operation and data-driven control schemes. However, there remains a significant gap in hands-on training opportunities for building operators and technicians to gain practical experience with advanced control systems in a low-risk environment. This paper presents BOPTEST (Building Optimization Performance Test) as a suitable platform for workforce training in building controls and GEB technologies. BOPTEST provides a suite of standardized building simulation test cases with a REST API, real-time control interfaces through BACnet, semantic models connecting users to building data, and built-in calculation of control metrics and performance indicators. The platform enables trainees to interact with virtual buildings using industry-standard protocols while learning how to implement and innovate control strategies. The training platform is designed to offer a structured and interactive learning experience for building engineers, helping them effectively develop, learn, and retain skills in fault identification, troubleshooting, and correction. The workflow is divided into three main phases: 1) Setup, 2) Exercise, and 3) Review, each comprising specific activities performed by either the instructor or the student. Initial pilot training sessions have yielded positive feedback from instructors and participants and demonstrates that BOPTEST effectively fills an industry need for a low-risk training resource via simulation of real building control systems, allowing trainees to gain practical experience before working in the field. The platform's ability to provide immediate performance feedback while maintaining familiar industry interfaces makes it particularly suitable for workforce development programs. This work provides a replicable model for leveraging building simulation in control education and training.
Global-Local Policy Search and its Application in Grid-Interactive Building Control
As the buildings sector represents over 70% of the total U.S. electricity consumption, it offers a great amount of untapped demand-side resources to tackle many critical grid-side problems and improve the overall energy system's efficiency. To help make buildings grid-interactive, this paper proposes a global-local policy search method to train a reinforcement learning (RL) based controller which optimizes building operation during both normal hours and demand response (DR) events. Experiments on a simulated five-zone commercial building demonstrate that by adding a local fine-tuning stage to the evolution strategy policy training process, the control costs can be further reduced by 7.55% in unseen testing scenarios. Baseline comparison also indicates that the learned RL controller outperforms a pragmatic linear model predictive controller (MPC), while not requiring intensive online computation.
Flexible Reinforcement Learning Framework for Building Control using EnergyPlus-Modelica Energy Models
In recent years, reinforcement learning (RL) methods have been greatly enhanced by leveraging deep learning approaches. RL methods applied to building control have shown potential in many applications due to their ability to complement or replace conventional methods such as model-based or rule-based controls. However, RL-based building control software is likely tailored either to one target building system or to a specific RL method so that significant additional effort would be required to customize the RL-based controller for use in other building systems or with other RL approaches. Also, RL-based building controls usually depend on building energy simulations to train controllers, so emulating building dynamics (i.e., thermal dynamics and control dynamics) and capturing sub-hourly dynamic profiles are crucial to further the development of effective RL-based building control methods. To address these challenges, we present an open source RL-based control software employing a high-fidelity hybrid EnergyPlus-Modelica building energy model which emulates building dynamics at 1-minute resolution. This software consists of decoupled components (environment, building emulator, control agent, and RL algorithm), which allows for quick prototyping and benchmarking of standard RL algorithms in different systems; for example, a single component can be replaced without revising all of the software. To demonstrate this software framework, we conducted a benchmark study using an EnergyPlus-Modelica building energy model for a Chicago office building with an RL-based controller to dynamically control the chilled water temperature setpoint and the air handling unit supply air temperature setpoint on selected floors.
Reinforcement learning building control approach harnessing imitation learning
Reinforcement learning (RL) has shown significant success in sequential decision making in fields like autonomous vehicles, robotics, marketing and gaming industries. This success has attracted the attention to the RL control approach for building energy systems which are becoming complicated due to the need to optimize for multiple, potentially conflicting, goals like occupant comfort, energy use and grid interactivity. However, for real world applications, RL has several drawbacks like requiring large training data and time, and unstable control behavior during the early exploration process making it infeasible for an application directly to building control tasks. To address these issues, an imitation learning approach is utilized herein where the RL agents starts with a policy transferred from accepted rule based policies and heuristic policies. This approach is successful in reducing the training time, preventing the unstable early exploration behavior and improving upon an accepted rule-based policy - all of these make RL a more practical control approach for real world applications in the domain of building controls.
Two-Stage Reinforcement Learning Policy Search for Grid-Interactive Building Control
This paper develops an intelligent grid-interactive building controller, which optimizes building operation during both normal hours and demand response (DR) events. To avoid costly on-demand computation and to adapt to non-linear building models, the controller utilizes reinforcement learning (RL) and makes real-time decisions based on a near-optimal control policy. Learning such a policy typically amounts to solving a hard non-convex optimization problem. We propose to address this problem with a novel global-local policy search method. In the first stage, an RL algorithm based on zero-order gradient estimation is leveraged to search for the optimal policy globally, due to its scalability and the potential to escape some poor performing local optima. The obtained policy is then fine-tuned locally to bring the first-stage solution closer to that of the original unsmoothed problem. Experiments on a simulated five-zone commercial building demonstrate the advantages of the proposed method over existing learning approaches. They also show that the learned control policy outperforms a pragmatic linear model predictive controller (MPC) and approaches the performance of an oracle MPC in testing scenarios. Using a state-of-the-art advanced computing system, we demonstrate that the controller can be learned and deployed within hours of training.
Grid-Interactive Building Control Via Reinforcement Learning
In this work, we present the proposed two-stage reinforcement learning approach for training building controllers to help buildings to participate in DR events. The original conference paper for this research idea can be found at https://www.nrel.gov/docs/fy21osti/78000.pdf.
Towards Learning-Based Architectures for Sensor Impact Evaluation in Building Controls
Advanced control algorithms for building systems are known to have significant potential in reducing energy consumption while optimizing thermal comfort. The success of such algorithms is critically contingent on several different types of sensor systems, which are in turn, used for continuous monitoring, identification and estimation of several important building states, such as temperatures, humidity, air quality, power consumption and occupancy status. Nonidealities in any of these sensors can lead to significant performance degradation of the control functionalities, and may lead to unwanted sub-optimal building operation. In this paper, we provide a simulation example with a high-fidelity building model, for a particular use-case of advanced optimization-based control in buildings, i.e., occupancy-based controls. We show how imperfections in occupancy sensing can offset performance. Subsequently, we discuss a novel learning-based architecture to efficiently evaluate the impact of sensor nonidealities for building systems, in context of advanced control algorithms.
All-Digital Plug and Play Passive RFID Sensors for Energy Efficient Building Control
This is the final report for the project “All-Digital Plug and Play Passive RFID Sensors for Energy Efficient Building Control”, funded by DOE, and performed by Clemson University, Phase IV Engineering and Harvard University from October 1, 2016 to December 31, 2020. The main objective of this project is to develop, demonstrate and pre-commercialize a novel, plug & play, battery-free, wireless sensor technology to enable low-cost (<$10 per node) indoor and outdoor temperature and humidity measurement for energy efficient building controls and operations. The proposed technology is based on the novel concept of all-digital sensing and its seamless integration with the passive RFID technology. This project focuses on the design, fabrication, material optimization, interrogation electronics, validation, and demonstration of the novel sensor nodes for building applications. The specific objectives of this research program include: (1) Design, fabricate and optimize a compact, robust, and high-resolution digitizer, which could convert rotation angle into digital numbers. (2) Develop the multi-physics-based modeling and simulation of the temperature/humidity transducer to derive a rational design of the architecture, dimension, structure, materials (i.e., mechanical, electrical, thermal and hygroscopic) properties and functions of the sensor node. (3) Design, fabricate and optimize the bi-material based humidity sensitive coil which could linearly transduce the environmental relative humidity variations to rotation angles. (4) Design, fabricate and optimize an UHF RFID platform which could support long-range passive wireless communications of 8-bit digital numbers. (5) Design, fabricate and optimize the miniaturized all-digital sensor using MEMS technology. (6) Validate the integrated all-digital sensing system in a real building environment to test the system’s performance.
An open-source framework for simulation-based testing of buildings control strategies
Here this paper presents a simulation framework for evaluating building control strategies (BCSs), developed with VOLTTRON, an opensource platform that integrates data, devices, and systems for sensing and control applications, and is based on co-simulation interfaces. It realizes an integrated environment for both testing and deploying BCSs, thereby eliminating the need for having a dedicated implementation of BCS for testing. For the first time, this framework provides critical functionalities, including scalable communication management and time-drive simulation advance, i.e., advancing the simulation based on clock time. We applied this framework to evaluating two BCSs from ASHRAE Guideline 36-2008 and ASHRAE Standard 90.1-2019. The evaluation results show that those BCSs primarily benefit the heating operation by reducing gas usage but yield insignificant savings in electricity consumption. The results also emphasize the importance of tuning the parameters of the BCSs to achieve better performances.
Grid-Interactive Multi-Zone Building Control Using Reinforcement Learning with Global-Local Policy Search
In this paper, we develop a grid-interactive multi-zone building controller based on a deep reinforcement learning (RL) approach. The controller is designed to facilitate building operation during normal conditions and demand response events, while ensuring occupants comfort and energy efficiency. We leverage a continuous action space RL formulation, and devise a two-stage global-local RL training framework. In the first stage, a global fast policy search is performed using a gradient-free RL algorithm. In the second stage, a local fine-tuning is conducted using a policy gradient method. In contrast to the state-of-the-art model predictive control (MPC) approach, the proposed RL controller does not require complex computation during real-time operation and can adapt to nonlinear building models. We illustrate the controller performance numerically using a five-zone commercial building.
Grid-Interactive Multi-Zone Building Control Using Reinforcement Learning with Global-Local Policy Search: Preprint
In this paper, we develop a grid-interactive multi-zone building controller based on a deep reinforcement learning (RL) approach. The controller is designed to facilitate building operation during normal conditions and demand response events, while ensuring occupants comfort and energy efficiency. We leverage a continuous action space RL formulation, and devise a two-stage global-local RL training framework. In the first stage, a global fast policy search is performed using a gradient-free RL algorithm. In the second stage, a local fine-tuning is conducted using a policy gradient method. In contrast to the state-of-the-art model predictive control (MPC) approach, the proposed RL controller does not require complex computation during real-time operation and can adapt to non-linear building models. We illustrate the controller performance numerically using a five-zone commercial building.
Preliminary Sensitivity Analysis for Sensors Impacts on Building Control Performance
This report describes the preliminary sensitivity analysis for sensor impacts on building control performance through the US Department of Energy’s Oak Ridge National Laboratory’s Flexible Research Platform (FRP-2) building. The rooftop unit system provides cooling and heating to the building. The main heating coil is a gas heating coil. Each zone is served by a variable air volume box with an electricity reheat coil. The rooftop unit and variable air volume box controls adopted the practical control sequences from ASHRAE Guideline 36-2018: High-Performance Sequences of Operation. For sensors, the incipient (time-changing) sensor errors, including bias sensor error and precision sensor error, are the inputs of interest. The outputs are energy consumption and thermal comfort (e.g., the predicted percentage of dissatisfied occupants). The large-scale simulation (3,600 cases) was conducted on a cloud platform by integrating sensor errors and ASHRAE Guideline 36 control sequences into an emulator based on the EnergyPlus simulation program with Python energy management system feature. The surrogate models were developed based on cloud simulation results. The uncertainty analysis showed that the sensor errors substantially affect building energy consumption and thermal comfort. The sensitivity analysis shows a ranking of sensor error impacts for each interested output item (e.g., cooling energy, reheat coil heating energy, predicted percentage of dissatisfied occupants). In FY 2022, sensor locations, types, and costs will be evaluated. The field test in Oak Ridge National Laboratory’s Flexible Research Platform building regarding sensor impacts will also be performed. Finally, a comparative analysis will be conducted based on the field test results and emulator results.
Exploration of Domain Aware Machine Learning for Grid Analytics: Transfer-Learnt Energy Models to Assist Buildings Control with Sparse Field Data
Buildings are a primary consumer of energy in the United States and are also increasingly being perceived as providers of grid services such as load shifting, shedding and modulation. High fidelity models of building energy consumption are needed to set appropriate baselines for measurement and verification (M&V) of controllers designed for energy efficient operation of buildings and to enable buildings to provide grid services via. participation in demand response programs. State-of-the-art building energy modeling techniques either rely on Physics based models, or extensive instrumentation of the building envelope to gather “big” data to train machine learning based models such as deep neural networks. While Physics based models are often limited by their accuracy, it is not always feasible to gather a significant amount of field data required to train machine learning based models with sufficient accuracy. In this paper, we explore the use of transfer learning-based strategies to address unsatisfactory accuracy of models for estimating building energy consumption when available field data for training is sparse or of unacceptable quality. In particular, we transfer knowledge in the form of data and parameters, from Physics based simulation frameworks to the field to improve the model accuracy, thus resulting in a Physics-informed Machine Learning framework. We evaluated the efficacy of our approach on field data collected from six commercial buildings and our results indicate that the proposed transfer learning based models provide comparative (and in some cases better) accuracy than state-of-the-art machine learning and deep learning solutions, with just one month of field data.