Automated point mapping for building control systems: Recent advances and future research needs
Not Available
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Not Available
We present an initial study of off-policy evaluation (OPE), a problem prerequisite to real-world reinforcement learning (RL), in the context of building control. OPE is the problem of estimating a policy's performance without running it on the actual system, using historical data from the existing controller. It enables the control engineers to ensure a new, pretrained policy satisfies the performance requirements and safety constraints of a real-world system, prior to interacting with it. While many methods have been developed for OPE, no study has evaluated which ones are suitable for building operational data, which are generated by deterministic policies and have limited coverage of the state-action space. After reviewing existing works and their assumptions, we adopted the approximate model (AM) method. Furthermore, we used bootstrapping to quantify uncertainty and correct for bias. In a simulation study, we evaluated the proposed approach on 10 policies pretrained with imitation learning. On average, the AM method estimated the energy and comfort costs with 1.84% and 14.1% error, respectively.
Abstract not provided.
Artificial intelligence technologies have emerged as a game changer not only in specific applications such as image recognition and machine translation but also in many scientific domains. In particular, as deep reinforcement learning (DRL) has shown great success in complex control problems, DRL-based control has been considered as a potential solution to efficiently control and manage building systems. However, broad assessment of DRL-based building control is still required to characterize their pros and cons in comparison with conventional building control methods (e.g., rule-based feedback controls). In this paper, we assessed DRL-based controls with on-policy learning-based algorithms and continuous control actions for cooling control of large office buildings in the summer season to minimize whole-building energy use and occupant discomfort. We compared DRL-based control methods with two baseline control methods: (1) a pre-determined schedule with supply temperature and static pressure setpoints, and (2) advanced reset method that adjusts setpoints based on heuristic rules, i.e., ASHRAE Guideline 36. We also tested the DRL algorithms to evaluate their performances in multiple climate locations. We found that DRL-based control methods outperformed the baseline control methods in terms of energy savings while maintaining a thermal comfort. DRL reduced energy use between ~4%–22% on average compared to the baseline methods, depending on climate location. We also evaluated DRL-based control in terms of control stability and showed that DRL-based methods should address the span of hardware lifetimes in practical operations.
The availability of computational power, and a wealth of data from sensors have boosted the development of model-based predictive control for smart and effective control of advanced buildings in the last decade. More recently occupant-behavior models have been developed for including people in the building control loops. However, while important objectives of scientific research are reproducibility and replicability of results, not all information is available from published documents. Therefore, the aim of this paper is to propose a guideline for a thorough and standardized occupant-behavior model documentation. For that purpose, the literature screening for the existing occupant behavior models in building control was conducted, and the occupant behavior modeling processes were studied to extract practices and gaps for each of the following phases: problem statement, data collection, and preprocessing, model development, model evaluation, and model implementation. Here, the literature screening pointed out that the current state-of-the-art on model documentation shows little unification, which poses a particular burden for the model application and replication in field studies. In addition to the standardized model documentation, this work presented a model-evaluation schema that enabled benchmarking of different models in field settings as well as the recommendations on how OB models are integrated with the building system.
Building automation and controls are becoming increasingly complex with the emergence of Grid Integrated Efficient Buildings (GEBs) as well as new highly efficient sequences of operation and data-driven control schemes. However, there remains a significant gap in hands-on training opportunities for building operators and technicians to gain practical experience with advanced control systems in a low-risk environment. This paper presents BOPTEST (Building Optimization Performance Test) as a suitable platform for workforce training in building controls and GEB technologies. BOPTEST provides a suite of standardized building simulation test cases with a REST API, real-time control interfaces through BACnet, semantic models connecting users to building data, and built-in calculation of control metrics and performance indicators. The platform enables trainees to interact with virtual buildings using industry-standard protocols while learning how to implement and innovate control strategies. The training platform is designed to offer a structured and interactive learning experience for building engineers, helping them effectively develop, learn, and retain skills in fault identification, troubleshooting, and correction. The workflow is divided into three main phases: 1) Setup, 2) Exercise, and 3) Review, each comprising specific activities performed by either the instructor or the student. Initial pilot training sessions have yielded positive feedback from instructors and participants and demonstrates that BOPTEST effectively fills an industry need for a low-risk training resource via simulation of real building control systems, allowing trainees to gain practical experience before working in the field. The platform's ability to provide immediate performance feedback while maintaining familiar industry interfaces makes it particularly suitable for workforce development programs. This work provides a replicable model for leveraging building simulation in control education and training.
Building automation and controls are becoming increasingly complex with the emergence of Grid Integrated Efficient Buildings (GEBs) as well as new highly efficient sequences of operation and data-driven control schemes. However, there remains a significant gap in hands-on training opportunities for building operators and technicians to gain practical experience with advanced control systems in a low-risk environment. This paper presents BOPTEST (Building Optimization Performance Test) as a suitable platform for workforce training in building controls and GEB technologies. BOPTEST provides a suite of standardized building simulation test cases with a REST API, real-time control interfaces through BACnet, semantic models connecting users to building data, and built-in calculation of control metrics and performance indicators. The platform enables trainees to interact with virtual buildings using industry-standard protocols while learning how to implement and innovate control strategies. The training platform is designed to offer a structured and interactive learning experience for building engineers, helping them effectively develop, learn, and retain skills in fault identification, troubleshooting, and correction. The workflow is divided into three main phases: 1) Setup, 2) Exercise, and 3) Review, each comprising specific activities performed by either the instructor or the student. Initial pilot training sessions have yielded positive feedback from instructors and participants and demonstrates that BOPTEST effectively fills an industry need for a low-risk training resource via simulation of real building control systems, allowing trainees to gain practical experience before working in the field. The platform's ability to provide immediate performance feedback while maintaining familiar industry interfaces makes it particularly suitable for workforce development programs. This work provides a replicable model for leveraging building simulation in control education and training.
The cyber-physical security threat to buildings is complex, non-linear, and rapidly evolving as operational and information technologies converge and connect buildings to cyberspace. Cyberattacks on buildings can exploit smart building controls and breach corporate networks, causing financial and reputational damage. This may result in the loss of sensitive building information or the disruption of, or damage to, the systems necessary for the safe and efficient operation of buildings. For the buildings and facility infrastructure, there is a need for a robust national cybersecurity strategy for buildings, guidance on the selection and implementation of appropriate cybersecurity controls for buildings, an approach to evaluate the maturity and adequacy of the cybersecurity programs. To provide an approach for evaluating the maturity of the cybersecurity programs for building control systems, the US Department of Energy’s widely used Cybersecurity Capability and Maturity Model (C2M2) has been adapted into a building control systems version. The revised model, the Buildings-C2M2 (B-C2M2) provides maturity level indicators for cybersecurity programmatic domains. A “B-C2M2 Lite” version allows facility managers and building control system engineers, or information technology personnel to perform rapid self-assessments of their cybersecurity program. Both tools have been pilot tested on several facilities. This paper outlines the concept of a maturity model, describes the B-C2M2 tools, presents results and observations from the pilot assessments, and lays out plans for future work.
The U.S. Department of Energy’s (DOE’s) Building Technologies Office (BTO) sponsored a study to determine the potential national savings achievable in the commercial building sector through widespread deployment of best practice controls, elimination of system and component faults, and use of better sensing. Detailed characterization of potential savings was one source of input to set research, development, and deployment (RD&D) goals in the field of building sensors and controls. DOE’s building energy simulation software, EnergyPlus, was employed to estimate the potential savings from 34 measures in 9 building types and across 16 climates representing almost 57% of commercial building sector energy consumption. In addition to estimating savings from individual measures, three packages of measures were created to estimate savings from the packages. These packages represented an 1) efficient building, 2) typical building, and 3) inefficient building. To scale the results from individual measures or a package to the national scale, building weights by building type and climate locations from the Energy Information Administration’s 2012 Commercial Building Energy Consumption Survey (CBECS) were used. The results showed significant potential for energy savings across all building types and climates. The total site potential savings from individual measures by building type and climate location ranged between 0% and 25%. The total site potential savings by building type aggregated across all climates (using the CBECS building weights) for each measure varied between 0% and 16%. The total site potential savings aggregated across all building types and climates for each measure varied between 0% and 11%. Some individual measures had negative savings because correcting underlying operational problems (e.g., inadequate ventilation) resulted in increased energy consumption. When combined into packages, the overall national savings potential is estimated to be 29%; seven of the nine building types were in the range of 23 to 29% and two exceeded 40%. The total potential national site savings in for each building type ranged between 95x106 GJ (0.09 Quadrillion British thermal units [Quads]; Large Hotels) to 222x106 GJ (0.21 Quads; Large Office, Hospital Administrative areas, and College/University), resulting in total site savings of 1,393x106 GJ (1.32 Quads) when the three packages are applied to the U.S. commercial buildings stock. Using the source (or primary) energy conversion factors of 1.05 for natural gas and 3.14 for electricity resulted in an approximate potential primary energy savings of 2,912x106 GJ (2.76 Quads), which would be 15% of the sector’s 2015 use of approximately 18,991x106 GJ (18 Quads). Extrapolating the results for other building types not analyzed as part of this study, the primary energy savings could be in the range of 4,220x106 GJ to 5,275x106 GJ (4 to 5 Quads). If this savings potential is realized, it would be equivalent to not combusting 180 to 230 million tons of coal or reducing the energy impacts, at today’s energy intensities, of the per capita consumption of 12 to 15 million people in the U.S. To realize most of this potential savings, many gaps can be addressed through RD&D, as recommended in this paper.
As the buildings sector represents over 70% of the total U.S. electricity consumption, it offers a great amount of untapped demand-side resources to tackle many critical grid-side problems and improve the overall energy system's efficiency. To help make buildings grid-interactive, this paper proposes a global-local policy search method to train a reinforcement learning (RL) based controller which optimizes building operation during both normal hours and demand response (DR) events. Experiments on a simulated five-zone commercial building demonstrate that by adding a local fine-tuning stage to the evolution strategy policy training process, the control costs can be further reduced by 7.55% in unseen testing scenarios. Baseline comparison also indicates that the learned RL controller outperforms a pragmatic linear model predictive controller (MPC), while not requiring intensive online computation.
In recent years, reinforcement learning (RL) methods have been greatly enhanced by leveraging deep learning approaches. RL methods applied to building control have shown potential in many applications due to their ability to complement or replace conventional methods such as model-based or rule-based controls. However, RL-based building control software is likely tailored either to one target building system or to a specific RL method so that significant additional effort would be required to customize the RL-based controller for use in other building systems or with other RL approaches. Also, RL-based building controls usually depend on building energy simulations to train controllers, so emulating building dynamics (i.e., thermal dynamics and control dynamics) and capturing sub-hourly dynamic profiles are crucial to further the development of effective RL-based building control methods. To address these challenges, we present an open source RL-based control software employing a high-fidelity hybrid EnergyPlus-Modelica building energy model which emulates building dynamics at 1-minute resolution. This software consists of decoupled components (environment, building emulator, control agent, and RL algorithm), which allows for quick prototyping and benchmarking of standard RL algorithms in different systems; for example, a single component can be replaced without revising all of the software. To demonstrate this software framework, we conducted a benchmark study using an EnergyPlus-Modelica building energy model for a Chicago office building with an RL-based controller to dynamically control the chilled water temperature setpoint and the air handling unit supply air temperature setpoint on selected floors.
Reinforcement learning (RL) has shown significant success in sequential decision making in fields like autonomous vehicles, robotics, marketing and gaming industries. This success has attracted the attention to the RL control approach for building energy systems which are becoming complicated due to the need to optimize for multiple, potentially conflicting, goals like occupant comfort, energy use and grid interactivity. However, for real world applications, RL has several drawbacks like requiring large training data and time, and unstable control behavior during the early exploration process making it infeasible for an application directly to building control tasks. To address these issues, an imitation learning approach is utilized herein where the RL agents starts with a policy transferred from accepted rule based policies and heuristic policies. This approach is successful in reducing the training time, preventing the unstable early exploration behavior and improving upon an accepted rule-based policy - all of these make RL a more practical control approach for real world applications in the domain of building controls.
This paper develops an intelligent grid-interactive building controller, which optimizes building operation during both normal hours and demand response (DR) events. To avoid costly on-demand computation and to adapt to non-linear building models, the controller utilizes reinforcement learning (RL) and makes real-time decisions based on a near-optimal control policy. Learning such a policy typically amounts to solving a hard non-convex optimization problem. We propose to address this problem with a novel global-local policy search method. In the first stage, an RL algorithm based on zero-order gradient estimation is leveraged to search for the optimal policy globally, due to its scalability and the potential to escape some poor performing local optima. The obtained policy is then fine-tuned locally to bring the first-stage solution closer to that of the original unsmoothed problem. Experiments on a simulated five-zone commercial building demonstrate the advantages of the proposed method over existing learning approaches. They also show that the learned control policy outperforms a pragmatic linear model predictive controller (MPC) and approaches the performance of an oracle MPC in testing scenarios. Using a state-of-the-art advanced computing system, we demonstrate that the controller can be learned and deployed within hours of training.
This paper gives an overview of recent developments in the Building Controls Virtual Test Bed (BCVTB), a framework for co-simulation and hardware-in-the-loop. First, a general overview of the BCVTB is presented. Second, we describe the BACnet interface, a link which has been implemented to couple BACnet devices to the BCVTB. We present a case study where the interface was used to couple a whole building simulation program to a building control system to assess in real-time the performance of a real building. Third, we present the ADInterfaceMCC, an analog/digital interface that allows a USB-based analog/digital converter to be linked to the BCVTB. In a case study, we show how the link was used to couple the analog/digital converter to a building simulation model for local loop control.
This report summarizes a comprehensive literature review for the selected sensor suites used in building controls and fault detection and diagnostics (FDD), which is a Q2 deliverable of the Sensor Impact Evaluation and Verification project. The project’s overarching goal is to develop a framework for investigating the impact of sensor deployment and configuration for building energy optimization, fault detection and diagnosis (FDD), occupants’ thermal comfort, and potential grid efficiency. While the first phase of the literature review performed in Q1 reviewed the current methods of various sensor selection and placements and their impact, this report focuses on high impactful potential use cases for the selected sensor suites in building control and FDD. The use cases identified in this literature review will be further investigated and be included in the final sensor impact evaluation framework. For the review, the multi-laboratory team reviewed 241 research papers, technical reports, and books to identify existing technologies, the potential impact on energy savings and thermal comfort, and future research opportunities for the selected use cases in building controls and FDD. Along with the literature review, the team conducted a series of interviews with experts in sensors and controls area to identify the state-of-art sensor technologies, and the most impactful use cases in controls and FDD areas. The interview results will also be incorporated with the findings from the literature review in the next quarter, and the final use cases to be evaluated will be selected.
In this work, we present the proposed two-stage reinforcement learning approach for training building controllers to help buildings to participate in DR events. The original conference paper for this research idea can be found at https://www.nrel.gov/docs/fy21osti/78000.pdf.
Advanced control algorithms for building systems are known to have significant potential in reducing energy consumption while optimizing thermal comfort. The success of such algorithms is critically contingent on several different types of sensor systems, which are in turn, used for continuous monitoring, identification and estimation of several important building states, such as temperatures, humidity, air quality, power consumption and occupancy status. Nonidealities in any of these sensors can lead to significant performance degradation of the control functionalities, and may lead to unwanted sub-optimal building operation. In this paper, we provide a simulation example with a high-fidelity building model, for a particular use-case of advanced optimization-based control in buildings, i.e., occupancy-based controls. We show how imperfections in occupancy sensing can offset performance. Subsequently, we discuss a novel learning-based architecture to efficiently evaluate the impact of sensor nonidealities for building systems, in context of advanced control algorithms.