Search NASA⌕ Search

SEARCH · Search NASA

Results for “Deep potential”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

AI-Enhanced Computational Tools for Entry Systems Modeling

To advance the understanding of complex atmospheric entry phenomena, NASA’s Entry Systems Modeling (ESM) team [1] has developed high-fidelity computational tools addressing multiscale challenges, from material microstructures to full-scale heatshield response. This abstract highlights a subset of ESM tools, focusing on AI integration to enhance workflows and predictive modeling. - PuMA [2] computes effective material properties from high-resolution micro-CT scans, supporting TPS analysis for NASA missions. - TomoSAM [3] automates 3D tomography dataset segmentation for PuMA using the Segment Anything Model, reducing manual effort and improving accuracy. - PATO [4] models porous reactive materials under extreme conditions, with advancements such as unified solvers, mechanical erosion, and TPS coatings for NASA missions. - arcjetCV [5] employs deep learning to analyze arc jet test footage, measuring recession rates, shape changes, and shock standoff distances, bridging simulations, and experiments to reveal TPS ablation behavior. - ARCHeS [6] simulates arc heater plasma flows, modeling turbulence, radiation, and electromagnetic interactions to optimize arc heater performance, validate TPS under extreme conditions, and serve as a foundation for developing digital twins of arc heater facilities. - SPARTA [7] simulates rarefied hypersonic flows and gas-surface interactions for planetary entry missions, leveraging GPU architectures for scalable and efficient aerothermal and ablation analyses. AI-driven solutions, such as deep learning segmentation, have streamlined workflows in ESM tools and still hold significant potential to further accelerate processes and enhance automation in entry systems modeling. [1] Haskins, J.B. (2023), [2] Ferguson, J.C. (2018), [3] Meurisse, J.B.E. (2018), [4] Semeraro, F. (2023), [5] Quintart, A. (2024) [6] Meurisse, J.B.E. (2022), [7] Plimpton, S.J. (2019)

Predictive Modeling↗
DOE OSTI2020

Metagenome-assembled genome distribution and key functionality highlight importance of aerobic metabolism in Svalbard permafrost

Permafrost underlies a large portion of the land in the Northern Hemisphere. It is proposed to be an extreme habitat and home for cold-adaptive microbial communities. Upon thaw permafrost is predicted to exacerbate increasing global temperature trend, where awakening microbes decompose millennia old carbon stocks. Yet our knowledge on composition, functional potential and variance of permafrost microbiome remains limited. In this study, we conducted a deep comparative metagenomic analysis through a 2 m permafrost core from Svalbard, Norway to determine key permafrost microbiome in this climate sensitive island ecosystem. To do so, we developed comparative metagenomics methods on metagenomic-assembled genomes (MAG). We found that community composition in Svalbard soil horizons shifted markedly with depth: the dominant phylum switched from Acidobacteria and Proteobacteria in top soils (active layer) to Actinobacteria, Bacteroidetes, Chloroflexi and Proteobacteria in permafrost layers. Key metabolic potential propagated through permafrost depths revealed aerobic respiration and soil organic matter decomposition as key metabolic traits. We also found that Svalbard MAGs were enriched in genes involved in regulation of ammonium, sulfur and phosphate. Here, we provide a new perspective on how permafrost microbiome is shaped to acquire resources in competitive and limited resource conditions of deep Svalbard soils.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Characterization of the Triassic Newark Basin of New York and New Jersey for Geologic Storage of Carbon Dioxide

The project Characterization of the Triassic Newark Basin of New York and New Jersey for Geologic Storage of Carbon Dioxide is one of 9 site characterization projects that were implemented as part of ARRA (American Recovery and Reinvestment Act). Data from this project was used to improve resolution of data in NATCARB in the area of study. Data related to this study has already been incorporated in NATCARB Atlas. Sandia Technologies, LLC, and co-investigator Conrad Geoscience Corporation, examined the potential for large-scale, permanent CO2 storage in sedimentary strata within the Newark Rift Basin. The Newark Rift Basin underlies an industrialized, developed region comprising parts of New York, New Jersey, and Pennsylvania. The project characterized and investigated the suitability of Triassic age sedimentary formations for potential geologic CO2 storage. The project team drilled and cored two test wells to define the sedimentary geologic formations underlying the basin and to document or reach basement rock. With this geologic characterization phase, an integration of seismic, geologic, borehole, and formation core results provided a higher resolution assessment of CO2 storage potential. The Stockton Formation is known to be a potentially favorable geologic storage formation in the basin. In 2011, the 1-NYSTA Tandem Lot stratigraphic test well was drilled to a depth of 6,855 feet in the northern portion of the Newark Basin in southern New York State. Approximately 9 miles south-southeast on the Lamont Doherty Campus, TW-4 was drilled and cored in 2013 to a depth of 1,802 feet and contacted apparent igneous basement at a depth of 1,712 feet. Both wells penetrated the Palisades Sill ranging from 800 feet thick in the eastern well to approximately 1,800 feet in thickness at the 1-NYSTA Tandem Lot deep drill site. A diabase sill can provide an excellent seal and dense confining layer for potential CO2 storage reservoirs and flow layers that are situated beneath it within the Stockton Sandstone. The Stockton Sandstone was encountered beneath the sill in the TW-4 well on the Lamont campus, and data integration suggests that it was likely observed near total depth in the deep 1-NYSTA Tandem Lot well. The test wells confirm and define reservoirs are present beneath the sill and offer CO2 storage potential. The integration of geologic and reservoir characterization of well logs, formation cores, and formation fluids indicated Triassic age lacustrine playa lake and mudbank shales of the Upper Passaic Group can provide an effective seal for the porous and permeable underlying sandstone reservoir layers. This project acquired seismic data, drilled borehole well logs, acquired core samples, and integrated these findings to provide a better understanding of the subsurface geologic formations in the Newark Rift Basin. These findings have contributed to a higher degree of accuracy in predicting potential geologic storage opportunities, while refining geologic storage capacity estimates for the indicated reservoirs and flow units.

.las↗
NASA NTRS1987

Electron dynamics and potential jump across slow mode shocks

In the de Hoffmann-Teller reference frame, the cross-shock electric field is simply the thermoelectric field responsible for preserving charge neutrality. As such, it gives information regarding the heating and dissipation occurring within the shock. The total cross-shock potential can be determined by integrating a weighted electron pressure gradient through the shock, but this requires knowledge of the density and temperature profiles. Here, a recently proposed alternative approach relying on particle dynamics is exploited to provide an independent estimate of this potential. Both determinations are applied to slow mode shocks which form the plasma sheet boundary in the deep geomagnetic tail as observed by ISEE 3. The two methods correlate well. There is no indication of the expected transition from resistive to viscous shocks, although the highest Mach number shocks show the highest potentials. The implications of these results for the electron dissipation mechanisms and turbulence at the shock are discussed.

Schwartz, Steven J.↗
DOE OSTI2026

The effect of hydroxyl spacing in diols on the solvation structure, dynamics, and transport properties of choline chloride-based deep eutectic solvents

Deep eutectic solvents (DESs) are a class of liquids that offer great potential in alleviating some of the challenges present in today's long-term energy storage methods because they have physical properties that are favorable for storable electrolyte solutions. In this work, a series of glycols (ethylene glycol, 1,3-propanediol, 1,4-butanediol, and 1,5-pentanediol) were studied as potential hydrogen bond donors (HBD) with a common choline chloride (ChCl) as the hydrogen bond acceptor (HBA). The solvation dynamics of the prepared systems were studied by measuring the solvent reorganization response using femtosecond transient absorption spectroscopy (fs-TA). Conductivity, viscosity, density, ET(30) polarity, and dynamics of the prepared DESs were analyzed, with a particular interest in determining the effect of HBD chain length on these parameters. Here, classical molecular dynamics simulations were employed to investigate how the local liquid structure, solvent dynamics, and bulk solvent properties vary with changes in glycol chain length.

Conductivity↗
DOE OSTI2025

Incentivizing Cooperative Merging Control: Insights from Multi-Agent Deep Reinforcement Learning

Cooperative driving automation enables connected and automated vehicles (CAVs) to devise cooperative merging control, introducing great potentials to alleviate traffic congestion, reduce energy consumption, and enhance safety for highway on-ramp operations. Although numerous CAV cooperative merging algorithms have been developed to improve energy and traffic performance, the agreement-seeking among CAV users and their local benefits have been understudied. This can lead to rejections of cooperative merging plans and jeopardizing CAV performance, as a cooperation may entail certain CAVs to sacrifice their local benefits to achieve a system optimum. To address this issue, the study first leverages multi-agent deep reinforcement learning (MADRL) factoring both local reward and regional reward to demonstrate the discrepancies between CAV users’ local benefits and system optimum. Next, the existence of a correlated equilibrium is proved to characterize the convergence of MADRL training. This further facilitates the incorporation of incentives (computed based on reward discrepancies) to compensate for CAV users’ local benefits and facilitate system-optimal agreements in cooperative merging operations.

Zhou, Anye [ORNL] (ORCID:0000000301455579)↗
NASA NTRS2016

Proving Ground Potential Mission and Flight Test Objectives and Near Term Architectures

NASA is developing a Pioneering Space Strategy to expand human and robotic presence further into the solar system, not just to explore and visit, but to stay. NASA's strategy is designed to meet technical and non-technical challenges, leverage current and near-term activities, and lead to a future where humans can work, learn, operate, and thrive safely in space for an extended, and eventually indefinite, period of time. An important aspect of this strategy is the implementation of proving ground activities needed to ensure confidence in both Mars systems and deep space operations prior to embarking on the journey to the Mars. As part of the proving ground development, NASA is assessing potential mission concepts that could validate the required capabilities needed to expand human presence into the solar system. The first step identified in the proving ground is to establish human presence in the cis-lunar vicinity to enable development and testing of systems and operations required to land humans on Mars and to reach other deep space destinations. These capabilities may also be leveraged to support potential commercial and international objectives for Lunar Surface missions. This paper will discuss a series of potential proving ground mission and flight test objectives that support NASA's journey to Mars and can be leveraged for commercial and international goals. The paper will discuss how early missions will begin to satisfy these objectives, including extensibility and applicability to Mars. The initial capability provided by the launch vehicle will be described as well as planned upgrades required to support longer and more complex missions. Potential architectures and mission concepts will be examined as options to satisfy proving ground objectives. In addition, these architectures will be assessed on commercial and international participation opportunities and on how well they develop capabilities and operations applicable to Mars vicinity missions.

Smith, R. Marshall↗
DOE OSTI2025

A generative artificial intelligence framework for long-time plasma turbulence simulations

Generative deep learning techniques are employed in a novel framework for the construction of surrogate models capturing the spatiotemporal dynamics of 2D plasma turbulence. The proposed Generative Artificial Intelligence Turbulence (GAIT) framework enables the acceleration of turbulence simulations for long-time transport studies. GAIT leverages a convolutional variational auto-encoder and a recurrent neural network to generate new turbulence data from existing simulations, extending the time horizon of transport studies with minimal computational cost. The application of the GAIT framework to plasma turbulence using the Hasegawa–Wakatani (HW) model is presented, evaluating its performance via various analyses. Very good agreement is found between the GAIT and the HW models in the spatiotemporal Fourier and Proper Orthogonal Decomposition spectra, the flow topology characterized by the Okubo–Weiss parameter, and the time autocorrelation function of turbulent fluctuations. Excellent agreement has also been obtained in the probability distribution function of particle displacements and the effective turbulent diffusivity. In-depth analyses of the latent space of turbulent states, choice of hyperparameters and alternative deep learning models for the time prediction are presented. Our results highlight the potential of Artificial Intelligence-based surrogate models to overcome the computational challenges in turbulence simulation, which can be extended to other situations such as geophysical fluid dynamics.

Artificial intelligence↗
NASA NTRS2008

Extreme Helicity and Intense Convective Towers in Hurricane Bonnie

Helicity was calculated in Hurricane Bonnie (1998) using tropospheric-deep dropsonde soundings from the NASA Convection and Moisture Experiment. Large helicity existed downshear of the storm center with respect to the ambient vertical wind shear. It was associated with veering, semicircular hodographs created by strong, vortex-scale, radial-vertical flow induced by the shear. The most extreme values of helicity, among the largest ever reported in the literature, occurred in the vicinity of deep convective cells in the downshear-left quadrant. These cells reached as high as 17.5 km and displayed the temporal and spatial scales of supercells. Convective available potential energy (CAPE) averaged 861 J/kg downshear, but only about one-third as large upshear. The soundings nearest the deep cells were evaluated using two empirical supercell parameters that make use of CAPE, helicity, and/or shear. These parameters supported the possible existence of supercells as a consequence of the exceptional helicity combined with moderate but sufficient CAPE. Ambient vertical wind shear exceeded 12 m/s for 30 h, yet the hurricane maintained 50 m/s maximum winds. It is hypothesized that the long-lived convective cells enabled the storm to resist the negative impact of the shear. Supercells in large-helicity, curved-hodograph environments appear to provide a useful conceptual model for intense convection in the hurricane core. Helicity calculations might also give some insight into the behavior of vortical hot towers, which share some characteristics with supercells.

Molinari, John↗
DOE OSTI2022

A Combined Computer Vision and Deep Learning Approach for Rapid Drone-Based Optical Characterization of Parabolic Troughs

Optical accuracy is a primary driver of parabolic trough concentrating solar power (CSP) plant performance, but can be damaged by wind loads, gravity, error during installation, and regular plant operation. Collecting and analyzing optical measurements over an entire operating parabolic trough plant is difficult, given the large scale of typical installations. Distant Observer, a software tool developed at the National Renewable Energy Laboratory, uses images of the absorber tube reflected in the collector mirror to measure both surface slope in the parabolic mirror and offset of the absorber tube from the ideal focal point. This technology has been adapted for fast data collection using low-cost commercial drones, but until recently still required substantial human labor to process large amounts of data. A new method leveraging advanced deep learning and computer vision tools can drastically reduce the time required to process images. This new method addresses the primary analysis bottleneck, identifying featureless, reflective mirror corner points to a high degree of accuracy. Recent work has shown promising results using computer vision methods. The combined deep learning and computer vision approach presented here proved highly effective and has the potential to further automate data collection and analysis, making the tool more robust. The method presented in this paper automatically identified 74.3% of mirror corners within 2 pixels of their manually marked counterparts and 91.9% within 3 pixels. This level of accuracy is sufficient for practical Distant Observer analysis within a target uncertainty. A commercial drone collected video of over 100 parabolic trough modules at an operating CSP plant to demonstrate the deep learning and computer vision method's usefulness in processing large amounts of data. These troughs were successfully analyzed using Distant Observer, paired with the new deep learning and computer vision algorithm, and can provide plant operators and trough designers with valuable insight about plant performance, operating strategies, and plant-wide optical error trends.

computer vision↗
DOE OSTI2022

Distributed deep learning training using silicon photonic switched architectures

The scaling trends of deep learning models and distributed training workloads are challenging network capacities in today’s datacenters and high-performance computing (HPC) systems. We propose a system architecture that leverages silicon photonic (SiP) switch-enabled server regrouping using bandwidth steering to tackle the challenges and accelerate distributed deep learning training. In addition, our proposed system architecture utilizes a highly integrated operating system-based SiP switch control scheme to reduce implementation complexity. To demonstrate the feasibility of our proposal, we built an experimental testbed with a SiP switch-enabled reconfigurable fat tree topology and evaluated the network performance of distributed ring all-reduce and parameter server workloads. The experimental results show up to 3.6× improvements over the static non-reconfigurable fat tree. Our large-scale simulation results show that server regrouping can deliver up to 2.3× flow throughput improvement for a 2× tapered fat tree and a further 11% improvement when higher-layer bandwidth steering is employed. The collective results show the potential of integrating SiP switches into datacenters and HPC systems to accelerate distributed deep learning training.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Technology Case Study: Economic, Sustainability, and Deployment Considerations for Sustainable Aviation Fuels Produced via Lignocellulosic Sugar Catalysis

This report presents a technology case study reflecting one exemplary representative pathway for the conversion of lignocellulosic sugars to sustainable aviation fuels (SAF) via aqueous phase reforming (APR) catalysis, considered within a broader integrated biorefinery framework based on biochemical processing operations. While far from the only option for converting sugars to SAF, this pathway was selected as a case study here based on its relatively high technology maturity and simplistic processing approach (avoiding complex separations or other equipment scalability challenges), coupled with the potential for high fuel yields and favorable costs/carbon intensities with opportunities for further near-term optimization. The report considers key process integration and engineering design considerations for a modeled hypothetical, nth-plant commercial biorefinery, reflecting a number of processing options and parameters envisioned to be achievable as future goals. Resultant outputs from Aspen Plus process simulations are evaluated through techno-economic and life cycle analyses (TEA and LCA), including implications for marginal cost of CO2 abatement and inclusion of currently-applicable policy incentives. Moving beyond base case configurations, a number of alternative scenarios are also evaluated for their ability to further improve economics, greenhouse gas (GHG) emissions, and marginal cost of abatement, highlighting a path to achieve deep decarbonization goals of more than 70% GHG reduction for SAF (with the potential to reach net-negative carbon intensities in some cases) under reasonable fuel production costs. The report also highlights future opportunities and gaps for further research on this technology pathway.

09 BIOMASS FUELS↗
NASA NTRS1979

Control considerations for CCV fighters at high angles of attack

Wind tunnel and piloted simulation studies were conducted to investigate the potential high angle of attack control problems that are introduced by the use of the CCV concept of relaxed static pitch stability (RSS) on fighter aircraft. A conventional wing/aft tail design incorporating modest levels of static instability and a close-coupled canard/wing design exhibiting very high levels of instability was investigated. Two types of high angle of attack control problems can result from the use of RSS: pitch departures caused by coupling and deep stall trim. Avoidance of these problems requires that the airplane have sufficient nose-down pitch control at high angles of attack. The effectiveness of several pitch control configurations were investigated including conventional aft-mounted stabilators, wing-mounted elevators, canard-mounted flaps, and all-moveable canards. Varying the incidence of the canards was the most effective scheme; however, very large deflections may be required on highly unstable configurations to prevent pitch departure without sacrificing roll performance and to avoid deep stall trim. For situations where the high angle of attack pitch control requirement is not met, control laws were developed to inhibit the departure and to allow deep stall recovery. However, these schemes involve limiting airplane roll capability and therefore can potentially compromise maneuverability.

Nguyen, L. T.↗
NASA NTRS2016

Carbon Nanofiber Nanoelectrodes for Neural Stimulation and Chemical Detection: The Era of Smart Deep Brain Stimulation

A sensor platform based on vertically aligned carbon nanofibers (CNFs) has been developed. Their inherent nanometer scale, high conductivity, wide potential window, good biocompatibility and well-defined surface chemistry make them ideal candidates as biosensor electrodes. Here, we report two studies using vertically aligned CNF nanoelectrodes for biomedical applications. CNF arrays are investigated as neural stimulation and neurotransmitter recording electrodes for application in deep brain stimulation (DBS). Polypyrrole coated CNF nanoelectrodes have shown great promise as stimulating electrodes due to their large surface area, low impedance, biocompatibility and capacity for highly localized stimulation. CNFs embedded in SiO2 have been used as sensing electrodes for neurotransmitter detection. Our approach combines a multiplexed CNF electrode chip, developed at NASA Ames Research Center, with the Wireless Instantaneous Neurotransmitter Concentration Sensor (WINCS) system, developed at the Mayo Clinic. Preliminary results indicate that the CNF nanoelectrode arrays are easily integrated with WINCS for neurotransmitter detection in a multiplexed array format. In the future, combining CNF based stimulating and recording electrodes with WINCS may lay the foundation for an implantable "smart" therapeutic system that utilizes neurochemical feedback control while likely resulting in increased DBS application in various neuropsychiatric disorders. In total, our goal is to take advantage of the nanostructure of CNF arrays for biosensing studies requiring ultrahigh sensitivity, high-degree of miniaturization, and selective biofunctionalization.

bioinstrumentation↗
NASA NTRS2016

Carbon Nanofiber Nanoelectrodes for Neural Stimulation and Chemical Detection: The Era of "Smart" Deep Brain Stimulation

A sensor platform based on vertically aligned carbon nanofibers (CNFs) has been developed. Their inherent nanometer scale, high conductivity, wide potential window, good biocompatibility and well-defined surface chemistry make them ideal candidates as biosensor electrodes. Here, we report two studies using vertically aligned CNF nanoelectrodes for biomedical applications. CNF arrays are investigated as neural stimulation and neurotransmitter recording electrodes for application in deep brain stimulation (DBS). Polypyrrole coated CNF nanoelectrodes have shown great promise as stimulating electrodes due to their large surface area, low impedance, biocompatibility and capacity for highly localized stimulation. CNFs embedded in SiO2 have been used as sensing electrodes for neurotransmitter detection. Our approach combines a multiplexed CNF electrode chip, developed at NASA Ames Research Center, with the Wireless Instantaneous Neurotransmitter Concentration Sensor (WINCS) system, developed at the Mayo Clinic. Preliminary results indicate that the CNF nanoelectrode arrays are easily integrated with WINCS for neurotransmitter detection in a multiplexed array format. In the future, combining CNF based stimulating and recording electrodes with WINCS may lay the foundation for an implantable smart therapeutic system that utilizes neurochemical feedback control while likely resulting in increased DBS application in various neuropsychiatric disorders. In total, our goal is to take advantage of the nanostructure of CNF arrays for biosensing studies requiring ultrahigh sensitivity, high-degree of miniaturization, and selective biofunctionalization.

human body↗
NASA NTRS2012

The Potential for a Ka-band (32 GHz) Worldwide VLBI Network

Ka-band (32 GHz, 9mm) Very Long Baseline Interferometric (VLBI) networking has now begun and has tremendous potential for expansion over the next few years. Ka-band VLBI astrometry from NASA's Deep Space Network has already developed a catalog of 470 observable sources with highly accurate positions. Now, several antennas worldwide are planning or are considering adding Ka-band VLBI capability. Thus, there is now an opportunity to create a worldwide Ka-band network with potential for high resolution imaging and astrometry. With baselines approaching a Giga-lambda, a Ka-band network would be able to probe source structure at the nano-radian (200 as) level ( 100X better than Hubble) and thus gain insight into the astrophysics of the most compact regions of emission in active galactic nuclei. We discuss the advantages of Ka-band, show the known sources and candidates, simulate projected baseline (uv) coverage, and discuss potential radio frequency feeds. The combination of these elements demonstrates the feasibility of a worldwide Ka network within the next few years!

quasar↗
NASA NTRSDate not supplied

Supporting Crew Medical Decisions on Deep Space Missions: A Real-Time Performance Monitoring

Crewed missions into deep space will require astronauts to respond autonomously to safety- and time-critical anomalies. These and other potential issues will require the continuous monitoring and recognition of potentially subtle, but complex, anomalous data patterns. NASA continues to investigate real-time metrics for Extravehicular Operations1 and for providing feedback to improve performance2. Since it is unknown how long-term exposure to deep space will affect crew health, continuous monitoring of clinical and subclinical status is warranted. NASA is characterizing crew medical decision-support needs and identifying the recommended requirements of a Clinical Decision Support System (CDSS) within the Exploration Medical Capability Element (ExMC) of the Human Research Program3. Early detection, and continuous monitoring, of performance may provide an effective prognostic capability for assessing crew health state, and the identified requirements could be provided for an in-vehicle CDSS that acts as an assistant for delivering optimal health and performance and medical care during exploration missions. There is evidence to suggest that automatic, unobtrusive collection of keystroke features, such as alphanumeric key latencies, backspaces or typing rhythm, could be a valuable performance screening tool. Keystroke features show significant differences between control subjects and patients with overt clinical illness, such as multiple sclerosis4 and Parkinson’s disease5, as well as behavioral health and performance issues, such as mild cognitive impairment6 and depression7. This work is exploring a capability for the assessment of early task performance decline in crew as they perform daily activities. Timestamped keyboard entries are continuously collected and stored. Nominal keystroke interactions are compared to those obtained after exposure to spaceflight stressors, e.g., fatigue, to determine the methods sensitivity. This work will pave the way toward an objective detection tool that may be deployed in a spaceflight setting.

clinical↗
DOE OSTI2023

Deep Reinforcement Learning Based Smart Water Heater Control for Reducing Electricity Consumption and Carbon Emission

Water heating is the third largest electricity consumer in U.S. households, after space heating and cooling. Thus, water heaters represent a significant potential for reducing electricity consumption and associated CO2 emissions of residential buildings. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity consumption and the CO2 emissions of a heat pump water heater without affecting user comfort. In this approach, a set of RL agents focusing on either electricity saving or emission reduction, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on different hot water usage and Marginal Operating Emissions Rate (MOER) profiles. The testing results showed that the RL agents that focus on electricity saving can save electricity in the range of 12–22% by operating the water heater with maximum heat pump efficiency and minimum electric element utilization. On the other hand, the RL agents that focus on emission reduction reduced emissions in the range of 18–37% by making use of the variable MOER values. These RL agents used the heat pump and/or an element when the MOER values are low due to the availability of renewable energy sources (e.g., solar and wind) and mostly avoided the periods of carbon-intensive periods. Overall, these results showed that the proposed RL approach can help minimize the electricity consumption and the CO2 emissions of a heat pump water heater without having any prior knowledge about the device.

Amasyali, Kadir↗