Search NASA⌕ Search
Home/Archive/DOE OSTI

DOE OSTI · 1974026

Reinforcement-Learning-Based Smart Water Heater Control: An Actual Deployment

Amasyali, Kadir·Kurte, Kuldeep·Zandi, Helia·Munk, Jeffrey
View original source ↗Export metadata ↓

Also available from

Abstract

Utilizing smart control algorithms for electric water heaters (EWHs) is essential for fully harnessing the demand response (DR) potential of EWHs. For this reason, the use of reinforcement learning (RL) algorithms for EWHs has received increasing attention in recent years. However, existing RL approaches are either simulation-based or use pretrained RL agents. To this end, this paper presents the real-world deployment of a set of model-free RL approaches that aim to minimize the electricity cost of a EWH under a time-of-use electricity pricing policy using standard DR commands (e.g., shed, load up). The experiment results showed that the RL agents can help save electricity cost in the range of 11% to 14% compared to the baseline operation. This study demonstrated that RL-based EWH controllers can be deployed in real world without any prior training and can still save electricity cost.

Explore related subjects

deep learningdemand responseelectric water heatersENERGY CONSERVATION, CONSUMPTION, AND UTILIZATIONmodel-free controlreinforcement learning

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Amasyali, Kadir, Kurte, Kuldeep, Zandi, Helia, Munk, Jeffrey. 2023-03-22. Reinforcement-Learning-Based Smart Water Heater Control: An Actual Deployment. https://doi.org/10.1109/isgt51731.2023.10066373

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

DOE OSTI2026

F-Hash: Feature-Based Hash Design for Time-Varying Volume Visualization via Multi-Resolution Tesseract Encoding

Interactive time-varying volume visualization is challenging due to its complex spatiotemporal features and sheer size of the dataset. Recent works transform the original discrete time-varying volumetric data into continuous Implicit Neural Representations (INR) to address the issues of compression, rendering, and super-resolution in both spatial and temporal domains. However, training the INR takes a long time to converge, especially when handling large-scale time-varying volumetric datasets. In this work, we proposed F-Hash, a novel feature-based multi-resolution Tesseract encoding architecture to greatly enhance the convergence speed compared with existing input encoding methods for modeling time-varying volumetric data. The proposed design incorporates multi-level collision-free hash functions that map dynamic 4D multi-resolution embedding grids without bucket waste, achieving high encoding capacity with compact encoding parameters. Our encoding method is agnostic to time-varying feature detection methods, making it a unified encoding solution for feature tracking and evolution visualization. Experiments show the F-Hash achieves state-of-the-art convergence speed in training various time-varying volumetric datasets for diverse features. We also proposed an adaptive ray marching algorithm to optimize the sample streaming for faster rendering of the time-varying neural representation.

deep learning↗
DOE OSTI2025

A Mass Conservation Relaxed (MCR) LSTM Model for Streamflow Simulation Across CONUS

The recent development of the physics-aware Mass-Conserving Long Short-Term Memory network (MC-LSTM) provides an alternative to other data-driven Deep Learning (DL) models in hydrology. Mass-Conserving Long Short-Term Memory incorporates mass conservation directly into the LSTM architecture. Despite the theoretical advancements, studies have reported a surprisingly limited performance of the MC-LSTM in streamflow simulation. We hypothesize that such a limitation is due to the unrealistic mass conservation scheme in MC-LSTM, which overlooks unobserved incoming water fluxes beyond precipitation. As an attempt to verify this hypothesis, we propose a Mass Conservation Relaxed LSTM (MCR-LSTM), which incorporates a bi-directional mass relaxation (MR) component to account for potential incoming water fluxes beyond precipitation. We train and test the proposed MCR-LSTM model across 531 watersheds in the contiguous United States (CONUS) against three baseline models: the Sacramento Soil Moisture Accounting, LSTM, and MC-LSTM. Our results show that MCR-LSTM outperforms MC-LSTM despite its underperformance compared to LSTM. Specifically, MCR-LSTM's advantage over MC-LSTM is mainly seen in the Plains and Western U.S., where the newly incorporated MR component better simulates water loss and suggests the likely existence of additional incoming water fluxes beyond precipitation, respectively. The novelty and contribution of this study are twofold: firstly, it introduces an alternative physics-aware DL tool (i.e., MCR-LSTM) in hydrology with higher accuracy in specific regions compared to MC-LSTM. Secondly, it provides a diagnosis of regions where strict, precipitation-based mass conservation constraints may be unrealistic in streamflow simulation.

deep learning↗
DOE OSTI2025

Defect Complexes in CrSBr Revealed Through Electron Microscopy and Deep Learning

Atomic defects underpin the properties of van der Waals materials, and their understanding is essential for advancing quantum and energy technologies. Scanning transmission electron microscopy is a powerful tool for defect identification in atomically thin materials, and extending it to multilayer and beam-sensitive materials would accelerate their exploration. Here, we establish a comprehensive defect library in a bilayer of the magnetic quasi-1D semiconductor CrSBr by combining atomic-resolution imaging, deep learning, and calculations. We apply a custom-developed machine learning work flow to detect, classify, and average point vacancy defects. This classification enables us to uncover several distinct Cr interstitial defect complexes, combined Cr and Br vacancy defect complexes, and lines of vacancy defects that extend over many unit cells. We show that their occurrence is in agreement with our computed structures and binding energy densities, reflecting the intriguing layer interlocked crystal structure of CrSBr. Our ab initio calculations show that the interstitial defect complexes give rise to highly localized electronic states. These states are of particular interest due to the reduced electronic dimensionality and magnetic properties of CrSBr and are, furthermore, predicted to be optically active. Our results broaden the scope of defect studies in challenging materials and reveal new defect types in bilayer CrSBr that can be extrapolated to the bulk and to over 20 materials belonging to the same FeOCl structural family.

deep learning↗