Search NASA⌕ Search

Engineering topics

O’Neill, Zheng

Publications and source records attributed to O’Neill, Zheng.

11 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Efficient and assured reinforcement learning-based building HVAC control with heterogeneous expert-guided training

Abstract Building heating, ventilation, and air conditioning (HVAC) systems account for nearly half of building energy consumption and $$20\%$$ of total energy consumption in the US. Their operation is also crucial for ensuring the physical and mental health of building occupants. Compared with traditional model-based HVAC control methods, the recent model-free deep reinforcement learning (DRL) based methods have shown good performance while do not require the development of detailed and costly physical models. However, these model-free DRL approaches often suffer from long training time to reach a good performance, which is a major obstacle for their practical deployment. In this work, we present a systematic approach to accelerate online reinforcement learning for HVAC control by taking full advantage of the knowledge from domain experts in various forms . Specifically, the algorithm stages include learning expert functions from existing abstract physical models and from historical data via offline reinforcement learning, integrating the expert functions with rule-based guidelines, conducting training guided by the integrated expert function and performing policy initialization from distilled expert function. Moreover, to ensure that the learned DRL-based HVAC controller can effectively keep room temperature within the comfortable range for occupants, we design a runtime shielding framework to reduce the temperature violation rate and incorporate the learned controller into it. Experimental results demonstrate up to 8.8 X speedup in DRL training from our approach over previous methods, with low temperature violation rate.

Xu, Shichao↗
DOE OSTI2024

Advanced co-simulation framework for assessing the interplay between occupant behaviors and demand flexibility in commercial buildings

With buildings contributing significantly to electricity usage, enabling demand flexibility becomes a challenge, especially when accounting for occupant comfort. This study introduces an innovative co-simulation framework integrating multiple models: heating, ventilation, and air conditioning (HVAC) system, building zone load, indoor airflow, supervisory control, and occupant comfort and behavior. Uniquely, this framework allows for a comprehensive and dynamic analysis of building systems and occupant interactions in demand response events. Using this framework, we conducted a case study using a typical small office building model. Specifically, we focused on three areas: (1) the impact of indoor airflow modeling on energy use, occupant comfort, and behaviors forecasting, (2) the impact of occupant behaviors on demand flexibility, and (3) occupant comfort and behaviors under demand response events. Key performance indicators such as energy use, flexibility factor, durations of occupant discomfort and occupant behaviors were analyzed. Our findings indicated variations in energy usage and occupant comfort within demand flexibility events, marked by uncertainty boundaries, with variability in demand shedding up to 57.9%. Here, we concluded that this framework is suitable for analyzing typical commercial buildings and their HVAC systems in terms of demand flexibility potential under the impact of occupant behaviors.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Emulation and detection of physical faults and cyber-attacks on building energy systems through real-time hardware-in-the-loop experiments

The increasing use of remote or mobile access, integrated wearable technologies, data exchange, and cloud-based data analytics in modern smart buildings is steering the building industry towards open communication technologies. The increased connectivity and accessibility could lead to more cyber-attacks in smart buildings. On the other hand, physical faults (e.g., HVAC -heating, ventilation, and air-conditioning faults) may have similar adverse impacts as those from the cyber-attacks on building energy systems, such as occupant discomfort, energy wastage, and equipment downtime. However, current physical behavior-based anomaly detection methods fail to differentiate between cyber-attacks and physical faults in building energy systems. Moreover, the challenge in collecting real-world threat data with ground truth has led researchers to rely on numerical models with user-defined assumptions, which may not accurately reflect real-world conditions due to the lack of in-situ experimental datasets. To address these challenges and gaps, this paper presents a flexible hardware-in-the-loop (HIL) testbed for generating cyber-attack and physical fault datasets and demonstrating threat detection algorithms in a real building automation system (BAS) environment. This testbed combines hardware (i.e., real BAS with local HVAC controllers and a physical network) with software (i.e., high-fidelity models to represent behaviors of building envelope and HVAC energy systems), enabling emulations of realistic threats. Five HIL experiments, including one baseline without any threats, two with physical faults, and two with cyber-attacks, were conducted to generate datasets containing detailed network traffic and system states. A joint classification framework, incorporating a network analyzer and a physical HVAC fault detector, was proposed to automatically detect cyber-physical abnormalities on BAS at both the network and the physical HVAC levels. The network analyzer comprises a conditional random fields (CRF) based command validator and a statistics-based detection strategy. The fault detector employs a weather and schedule-based pattern matching and feature-based principal component analysis (WPM-FPCA) method. Evaluation of the classification using four metrics from the multi-class confusion matrix revealed an average accuracy of 90.2%, recall of 89.7%, precision of 88.5% and F1-score of 89.2%. Finally, these results demonstrate that the proposed joint classification framework can effectively differentiate between specific types of cyber-attacks (e.g., device reinitialization attack, network Denial-of-Service attack) and physical faults (e.g., air handling unit operational fault, cooling coil valve stuck) in real time for improved building energy management.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Performance assessment of a real water source heat pump within a hardware-in-the-loop (HIL) testing environment

Over the last decade, the global fight against climate change through electrification has led to an increase in research on building heating, ventilation, and air conditioning (HVAC) systems that utilize intelligent control algorithms to provide demand-side grid service while also maintaining the thermal comfort of building occupants. As the pivotal point between building electricity consumption and indoor thermal comfort, high-efficiency electrical vapor-compression heat pumps are at the center of these emerging studies, and various grid-interactive and occupant-comfort control algorithms have been developed for them. The impact of these algorithms on heat pump operation and performance when subjected to different weather conditions, building loads, and grid requests calls for investigation and verification via experimental testing with actual heat pumps integrated with real-time building and grid responses. This study introduces a Water-Source Heat Pump (WSHP) Hardware-in-The-Loop (HIL) Test Facility that is the first of its kind. This testbed utilizes a 2-ton variable speed water-to-air heat pump that is capable of interacting with a virtual environment currently comprised of an EnergyPlus (E+) building simulation, an agent-based occupant behavioral model, and a single U-tube ground-loop heat exchanger (GLHE) model. Detailed descriptions of the testbed’s physical design and operation, virtual environment, as well as their mutual communication is provided. An uncertainty analysis is also performed under manufacturer specified heating and cooling design conditions. This analysis shows that the total load across the WSHP’s demand side heat exchanger, i.e., the sum of its latent and sensible components, can be measured with a relative uncertainty of ± 10.4% and ± 3.6% in cooling and heating mode respectively. The WSHP’s coefficient of performance (COP) can be measured with relative uncertainties of ± 10.4% in cooling mode, and ± 3.7% in heating mode. A preliminary 24-h experimental demonstration is then performed utilizing the DOE prototype small commercial office building model in E+. The simulation takes place in Atlanta, GA on the date of 08/26/15 from 12:00 AM to 11:59 PM using TMY3 weather data. Here, the results from this demonstration show that over the course of this experiment the simulated outputs of zone dry-bulb temperature, zone humidity ratio, and WSHP inlet water temperature can be tracked by testbed emulators up to a root mean squared error (RMSE) of ± 0.27 °C, ± 0.376 g/kg, and ± 0.85 °C respectively. The WSHP’s dynamic behavioral characteristics and performance are also captured, and correspond well with the authors’ previous understanding of heat pump efficiency as a function of evaporator and condenser fluid inlet conditions respectively.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

A framework for calibrating and validating an air loop dynamic model in an HVAC system in Modelica

The use of Modelica for simulating the dynamic behaviors of building heating, ventilation, and air conditioning (HVAC) systems has gained popularity. Calibration of a model that represents large and complex HVAC systems in Modelica involves the determination of hundreds of parameters using real-world operation data. Considering the coupling effects among various components, such a calibration process is complex and time-consuming. In this study, we propose a systematic framework to efficiently calibrate and validate a complex HVAC system model in Modelica. The framework includes strategies to solicit real-system operation data and to decouple the model. The goal of this calibration framework is to accurately and efficiently determine a set of Modelica model parameters that provide a good match between the simulated and real system behaviors. To demonstrate the validity of the framework, it was applied to calibrate the parameters of a Modelica model of the air loop subsystem of a real AHU-VAV system. The results show that the calibrated model can generate simulated results, including VAV air flow rate, outdoor air flow, and fan power consumption, that match well with the operational data of the real system. The coefficient of the variation of the root mean square error, CV(RMSE), of the air flow rate and power consumption are 14.6 % and 11.5 %, respectively. The results prove that the framework is valid and effective, and it can be used to calibrate other complex HVAC system Modelica models in the future.

Li, Yicheng↗
DOE OSTI2023

How good are learning-based control v.s. model-based control for load shifting? Investigations on a single zone building energy system

Both model predictive control (MPC) and deep reinforcement learning control (DRL) have been presented as a way to approximate the true optimality of a dynamic programming problem, and these two have shown significant operational cost saving potentials for building energy systems. Furthermore, there is still a lack of in-depth quantitative studies on their approximation levels to the true optimality, especially in the building energy domain. To fill in the gap, this paper provides a numerical framework that enables the evaluation of the optimality levels of different controllers for building energy systems. This framework is then used to comprehensively compare the optimal control performance of both MPC and DRL controllers with given computation budgets for a single zone fan coil unit system. Note the optimality is estimated based on a user-specific selection of trade-off weights among energy costs, thermal comfort and control slew rates. Compared with the best optimality we can find through expensive optimization simulations, the best DRL agent can maximally approximate the optimality by 96.54%, which outperforms the best MPC whose optimality level is 90.11%. However, due to the stochasticity, the DRL agent is only expected to approximate the optimality by 90.42%, which is almost equivalent to the best MPC. Except for Proximal Policy Optimization (PPO), all DRL agents can have a better approximation to the optimality than the best MPC, and are expected to have better approximation than the MPC with a prediction horizon of 32 steps (15 min per step). In terms of reducing energy cost and thermal discomfort, MPC can outperform the rule-based control (RBC) by 18.47%–25.44%. DRL can be expected to outperform RBC by 18.95%–25.65% ,and the best DRL control policy can outperform RBC by 20.29%–29.72%. Although the comparison of the optimality level is performed in a perfect setting, e.g., MPC assumes perfect models, and DRL assumes a perfect offline training process and online deployment process, this can shed insight on their capabilities of approximating to the original dynamic programming problem.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

From occupants to occupants: A review of the occupant information understanding for building HVAC occupant-centric control

Occupants are the core of the built environment. Traditional Heating, Ventilation, and Air-Conditioning (HVAC) systems operate with predefined schedules and maximum occupancy assumptions with no consideration of specific occupant information. These generalized assumptions usually do not align with the actual demand and result in over-conditioning and occupant discomfort. In recent years, with the aid of Information & Communication Technology (ICT) and Computer Science (CS), it is possible to acquire real-time and accurate occupant information to satisfy the exact thermal requirement through specific HVAC control in one particular built environment. This mechanism is called HVAC “Occupant-centric Control (OCC).” HVAC OCC strategy starts with collecting the occupant’s information (e.g., presence/absence) and then applies it to meet the occupant’s requirement (e.g., thermal comfort). However, even though some research studies and field pilot demonstrations have been devoted to the field of OCC, there is a lack of systematic knowledge about occupant data, which is the principal component of OCC for HVAC researchers and practitioners. To fill this gap, this review paper discusses OCC with a particular emphasis on occupant information and investigates how this information can assist HVAC operation in providing an acceptable built environment in required spaces during the required time. Finally, we provide a fine-grained, comprehensive picture of occupant information, discuss its features, the modalities of information feed-in into the HVAC control, and the application of commonly utilized occupant information for OCC.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Hardware-Based Emulator with Deep Learning Model for Building Energy Control and Prediction Based on Occupancy Sensors’ Data

Heating, ventilation, and air conditioning (HVAC) is the largest source of residential energy consumption. Occupancy sensors’ data can be used for HVAC control since it indicates the number of people in the building. HVAC and sensors form a typical cyber-physical system (CPS). In this paper, we aim to build a hardware-based emulation platform to study the occupancy data’s features, which can be further extracted by using machine learning models. In particular, we propose two hardware-based emulators to investigate the use of wired/wireless communication interfaces for occupancy sensor-based building CPS control, and the use of deep learning to predict the building energy consumption with the sensor data. We hypothesize is that the building energy consumption may be predicted by using the occupancy data collected by the sensors, and question what type of prediction model should be used to accurately predict the energy load. Another hypothesis is that an in-lab hardware/software platform could be built to emulate the occupancy sensing process. The machine learning algorithms can then be used to analyze the energy load based on the sensing data. To test the emulator, the occupancy data from the sensors is used to predict energy consumption. The synchronization scheme between sensors and the HVAC server will be discussed. We have built two hardware/software emulation platforms to investigate the sensor/HVAC integration strategies, and used an enhanced deep learning model—which has sequence-to-sequence long short-term memory (Seq2Seq LSTM)—with an attention model to predict the building energy consumption with the preservation of the intrinsic patterns. Because the long-range temporal dependencies are captured, the Seq2Seq models may provide a higher accuracy by using LSTM architectures with encoder and decoder. Meanwhile, LSTMs can capture the temporal and spatial patterns of time series data. The attention model can highlight the most relevant input information in the energy prediction by allocating the attention weights. The communication overhead between the sensors and the HVAC control server can also be alleviated via the attention mechanism, which can automatically ignore the irrelevant information and amplify the relevant information during CNN training. Our experiments and performance analysis show that, compared with the traditional LSTM neural network, the performance of the proposed method has a 30% higher prediction accuracy.

Ye, Zhijing↗
DOE OSTI2021

A Flexible and Generic Functional Mock-up Unit Based Threat Injection Framework for Grid-interactive Efficient Buildings: A Case Study in Modelica

Grid-interactive efficient buildings (GEBs) have been considered as an important asset to support the power grid reliability by utilizing the demand flexibility offered by GEBs. GEBs are enabled by advances in sensors and controls, and the communication between building equipment, whole buildings, and the grid. The integration of different building technologies and network-based communication system makes GEBs vulnerable to passive threats such as equipment failure and active threats such as cyber-attacks. Modeling and simulation is an effective way to evaluate the impact of threats on the system performance. This paper proposes a generic and flexible threat injection framework for commonly-used building energy simulators such as EnergyPlus and Modelica to support threat modeling and evaluation. This framework leverages functional mock-up unit (FMU) to develop a general modeling interface for threat injection and simulation. A numerical case study using Modelica as a building energy simulator is conducted to demonstrate the capability of the framework for supporting single/multiple-order threat modeling and simulation of a GEB. Four threats and their combinations are injected on a Modelica-based threat-free building energy and control system, including operating supply fan at its full speed, remotely cycling the chiller on and off, blocking the chiller from receiving the chilled water supply temperature setpoints, and hijacking the global zone air temperature setpoint. Simulation results show that the cyber-attack that leads to short-term signal blocking has small effects on the system operation due to the "self-healing" feature of the heating, ventilation, and air-conditioning (HVAC) interactive control system. The threat that takes control of resetting the global zone air temperature setpoints has the most adverse impact on the system energy use, peak power demand, thermal comfort and the provision of demand flexibility. The combination of four threats have aggregative effects on the system but the effects are less than the additive effects of the individual threat.

Fu, Yanyang↗