Learning bang-bang regulators.
Continuous time algorithm for learning stable bang-bang regulator assuming known Liapunov function
Butz, A. R.↗
SEARCH · Search NASA
Results for “continual learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Deep learning to generate in silico chemical property libraries and candidate molecules for small molecule identification in complex samples
Comprehensive and unambiguous identification of small molecules in complex samples will revolutionize our understanding of the role of metabolites in biological systems. Existing and emerging technologies have enabled measurement of chemical properties of molecules in complex mixtures and, in concert, are sensitive enough to resolve even stereoisomers. Despite these experimental advances, small molecule identification is inhibited by (i) chemical reference libraries (e.g. mass spectra, collision cross section, and other measurable property libraries) representing <1% of known molecules, limiting the number of possible identifications, and (ii) the lack of a method to generate candidate matches directly from experimental features (i.e. without a library). To this end, we developed a variational autoencoder (VAE) to learn a continuous numerical, or latent, representation of molecular structure to expand reference libraries for small molecule identification. We extended the VAE to include a chemical property decoder, trained as a multitask network, in order to shape the latent representation such that it assembles according to desired chemical properties. The approach is unique in its application to metabolomics and small molecule identification, with its focus on properties that can be obtained from experimental measurements (m/z, CCS) paired with its training paradigm, which involved a cascade of transfer learning iterations. First, molecular representation is learned from a large dataset of structures with m/z labels. Next, in silico property values are used to continue training, as experimental property data is limited. Finally, the network is further refined by being trained with the experimental data. This allows the network to learn as much as possible at each stage, enabling success with progressively smaller datasets without overfitting. Once trained, the network can be used to predict chemical properties directly from structure, as well as generate candidate structures with desired chemical properties. Our approach is orders of magnitude faster than first-principles simulation for CCS property prediction. Additionally, the ability to generate novel molecules along manifolds, defined by chemical property analogues, positions DarkChem as highly useful in a number of application areas, including metabolomics and small molecule identification, drug discovery and design, chemical forensics, and beyond.
Colby, Sean M.↗
Deep learning to generate in silico chemical property libraries and candidate molecules for small molecule identification in complex samples
Comprehensive and unambiguous identification of small molecules in complex samples will revolutionize our understanding of the role of metabolites in biological systems. Existing and emerging technologies have enabled measurement of chemical properties of molecules in complex mixtures and, in concert, are sensitive enough to resolve even stereoisomers. Despite these experimental advances, small molecule identification is inhibited by (i) chemical reference libraries (e.g. mass spectra, collision cross section, and other measurable property libraries) representing <1% of known molecules, limiting the number of possible identifications, and (ii) the lack of a method to generate candidate matches directly from experimental features (i.e. without a library). To this end, we developed a variational autoencoder (VAE) to learn a continuous numerical, or latent, representation of molecular structure to expand reference libraries for small molecule identification. We extended the VAE to include a chemical property decoder, trained as a multitask network, in order to shape the latent representation such that it assembles according to desired chemical properties. The approach is unique in its application to metabolomics and small molecule identification, with its focus on properties that can be obtained from experimental measurements (m/z, CCS) paired with its training paradigm, which involved a cascade of transfer learning iterations. First, molecular representation is learned from a large dataset of structures with m/z labels. Next, in silico property values are used to continue training, as experimental property data is limited. Finally, the network is further refined by being trained with the experimental data. This allows the network to learn as much as possible at each stage, enabling success with progressively smaller datasets without overfitting. Once trained, the network can be used to predict chemical properties directly from structure, as well as generate candidate structures with desired chemical properties. Our approach is orders of magnitude faster than first-principles simulation for CCS property prediction. Additionally, the ability to generate novel molecules along manifolds, defined by chemical property analogues, positions DarkChem as highly useful in a number of application areas, including metabolomics and small molecule identification, drug discovery and design, chemical forensics, and beyond.
Colby, Sean M↗
Grassroots Efforts To Quantify and Improve the Academic Climate of an R1 STEM Department: Using Evidence-Based Discussions To Foster Community
Women and some racial and ethnic groups remain underrepresented in chemistry departments across the United States, and generally, efforts to improve representation have resulted in minimal or no improvements in the last 10 years. Here, we present the outcomes of a graduate-student-led initiative that sought to assess the issues affecting inclusivity, diversity, and wellness within the Department of Chemistry at the University of California, Berkeley. We report how the results of a department-tailored academic climate survey were used to develop a method to foster open, productive discussion among graduate students, postdoctoral researchers, and faculty. This event format led to an improved understanding of the challenges facing our community members, as well as the identification of strategies that can be used to make the Department of Chemistry more welcoming for all members. We report the success of this student-led effort to highlight the value of assessing diversity and inclusion at the department-level, as well as the benefits of using community data to stimulate productive, evidence-based discussions. Furthermore, we envision that these methods can be implemented within any research-focused academic community to promote positive cultural change.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Uncertainty guided online ensemble for non-stationary data streams in fusion science
Machine Learning (ML) is poised to play a pivotal role in the development and operation of next-generation fusion devices. Fusion data shows non-stationary behavior with distribution drifts, resulted by both experimental evolution and machine wear-and-tear. ML models assume stationary distribution and fail to maintain performance when encountered with such non-stationary data streams. Online learning techniques have been leveraged in other domains, however it has been largely unexplored for fusion applications. In this paper, we investigate online learning for continuous adaptation to drifting data streams in the prediction of Toroidal Field (TF) coils deflection at the DIII-D fusion facility. We further address the short-term performance degradation inherent to standard online learning, which arises because ground truth is unavailable at prediction time. To mitigate this issue, we propose an uncertainty-guided online ensemble framework. The method leverages the Deep Gaussian Process Approximation (DGPA) for calibrated uncertainty estimation and uses these uncertainty measures to guide a meta-algorithm that aggregates predictions from learners trained over different historical horizons. Our results show that online learning reduces prediction error by 80% compared to a static model. The online ensemble and the proposed uncertainty-guided ensemble further reduce error by approximately 6%, and 10% respectively, relative to standard single-model online learning, while also providing calibrated uncertainty estimates to support operational decision-making.
AI↗
CRCNS22 Learning Rules in the Hippocampus and their Mapping to Neuromorphic Systems (Final Technical Report)
Large scale biologically-realistic computational models are key to investigating the interplay between structure and function in nervous systems, thus paving the way to new clinical methods and neuro-inspired computing solutions. This project focuses on the hippocampus, in particular the CA3-CA1 regions, due to their role in associative learning and memory, pattern separation and completion, and spatial navigation. Investigations into the neuronal organization and learning rule(s) of this circuit can shed light into how declarative memories are formed, stored, recalled and forgotten and inform computational, experimental and clinical neuroscience work. Our project aims at developing a novel data-driven methodology supported by a broad heterogeneous base of neuroscience experimental knowledge and inspired from advances in computer science and engineering. Specifically, this work will benchmark existing and new learning rules within a full-scale spiking neural network simulation of the CA3-CA1 region. The model will be based on an open-source repository, called the Hippocampome, which contains neuronal morphologies, firing patterns, synapse probabilities, and most other required parameters for all known neuron types in the rodent hippocampal formation. The model will be first trained in a supervised fashion for associative memory tasks using backpropagation through time traditionally used in computer science, enhanced with a new technique called the surrogate gradient method. This optimization method will be used to obtain a global loss minimization, but it is not biologically inspired as it assumes the use of data not locally available to the synapses. However, we propose its use as a benchmarking tool, to compare the training performance of local biologically plausible and hardware-mappable learning rules at scale. New rules or combinations will be proposed and tested as needed, based on the obtained results. Progress in this area will also drive the development of novel hardware-mappable algorithms for continual lifelong learning and categorization of new events from few presented examples. This project goes beyond the existing state-of-the-art by looking at large scale realistic neuronal circuits as networks trainable via global optimization methods such as surrogate gradient descent. The objective function of the brain that supports learning is largely unknown, but it is likely that it operates through local learning rules. Studying network trajectories around local minima as proposed in this work represents a useful strategy for understanding whether a network is training by using a specific (set of) learning rule(s). Starting from a completely untrained network is a challenging test since it is difficult to determine how the learning rule affects the trajectory of the network. This interdisciplinary project will help understand what rule governs learning in these regions or if multiple learning rules are involved. The work will develop a robust methodology to measure if the network is converging to the target solution, oscillating around it, or diverging away.
59 BASIC BIOLOGICAL SCIENCES↗
My Internship at NASA
My name is Isaac Lopez and I am a junior at the University of Houston majoring in Mechanical Engineering Technology. I will be completing my first tour at the NASA-Johnson Space Center ("JSC") as a Mechanical Engineer within the Human Interfaces Branch. Throughout my tour, I was given the opportunity to work on multiple projects that have expanded my knowledge and interest in acoustics and engineering design. One of the projects I worked on at JSC consisted of doing acoustic simulation of the EVA comm. cap. While working on the comm. cap headset, my main duty consisted of simulating the acoustics of the headset to find a solution to the condensing water that can accumulate and block the acoustic tube, causing attenuation or complete loss of audio in one ear for an astronaut using the EVA. For this project, I had to create a Creo model of the comm. cap so that I would be able to import it into Comsol for acoustic simulation. I also had the opportunity to design a portable and lightweight beam degrader for the EEE Parts and Radiation team. With the help of Creo, I was able to make a CAD design and put together a small working prototype for the radiation team to demonstrate the capabilities that the beam degrader had. In addition to these projects, JSC allowed me to work closely on projects with other interns. I had the opportunity to help another intern with his acoustic diverter, intended to improve the sound quality in Node 1 of the ISS. During this project, I helped with some of the acoustic testing inside the anechoic chamber as well as helping record data during testing at the ISS mock up. During the course of my first tour, I was able to learn and continually improve on my CAD drafting skills. With each project I worked on, I acquired new ways to create and improve various designs with various constraints. Furthermore, I also had the opportunity to work with electrical engineers and learn about the electronic components that would provide control of the beam degrader functions. I was able to take the knowledge, skills and designs I had acquired during my tour at JSC (such as Finite Element Analysis) and apply them to my work on the Comsol Multiphysics software for acoustics simulation. I was also able to use it to do some stress analysis on a proposed design for a proposed Habitat structure. My time at JSC has been invaluable as it helped me grow and more fully-comprehend my major as well as spark an interest in acoustics and how much it relates to mechanical engineering. This internship has taught me that engineering is much more than a profession - it is embedded in our daily lives as a puzzle that requires critical thinking as an everyday challenge that makes life more interesting.
Lopez, Isaac↗
Method for simultaneous characterization and expansion of reference libraries for small molecule identification
A variational autoencoder (VAE) has been developed to learn a continuous numerical, or latent, representation of molecular structure to expand reference libraries for small molecule identification. The VAE has been extended to include a chemical property decoder, trained as a multitask network, to shape the latent representation such that it assembles according to desired chemical properties. The approach is unique in its application to metabolomics and small molecule identification, focused on properties that are obtained from experimental measurements (m/z, CCS) paired with its training paradigm, which involves a cascade of transfer learning iterations. First, molecular representation is learned from a large dataset of structures with m/z labels. Next, in silico property values are used to continue training. Finally, the network is further refined by being trained with the experimental data. The trained network is used to predict chemical properties directly from structure and generate candidate structures with desired chemical properties. The network is extensible to other training data and molecular representations, and for use with other analytical platforms, for both chemical property and feature prediction as well as molecular structure generation.
Colby, Sean M.↗
Smart Congestion Control for Delay- and Disruption Tolerant Networks
In this paper, we propose a novel congestion control framework for delay- and disruption tolerant networks (DTNs). The proposed framework, called Smart-DTN-CC, adjusts its operation automatically as a function of the dynamics of the underlying network. It employs reinforcement learning, a machine learning technique known to be well suited to problems in which the environment, in this case the network, plays a crucial role; yet, no prior knowledge about the target environment can be assumed, i.e., the only way to acquire information about the environment is to interact with it through continuous online learning. Smart-DTN-CC nodes get input from the environment (e.g., its buffer occupancy, set of neighbors, etc), and, based on that information, choose an action to take from a set of possible actions. Depending on an action’s effectiveness in controlling congestion, it will be given a reward. Smart-DTN-CC’s goal is to maximize the overall reward which translates to minimizing congestion. To our knowledge, Smart-DTN-CC is the first DTN congestion control framework that has the ability to automatically and continuously adapt to the dynamics of the target environment which allows Smart-DTNCC to deliver adequate performance in a variety of DTN applications and scenarios. As demonstrated by our experimental evaluation, Smart-DTN-CC is able to consistently outperform existing DTN congestion control mechanisms under a wide range of network conditions and characteristics.
Hirata, Celso M.↗
Test and Evaluation of Systems with Embedded Machine Learning Components
As Machine Learning (ML) continues to advance, it is being integrated into more systems. Often, the ML component represents a significant portion of the system that reduces the burden on the end user or significantly improves task performance. However, the ML component represents an unknown complex phenomenon that is learned from collected data without the need to be explicitly programmed. Despite the improvement in task performance, the models are often black boxes. Evaluating the credibility and the vulnerabilities of ML models poses a gap in current test and evaluation practice. For high consequence applications, the lack of testing and evaluation procedures represents a significant source of uncertainty and risk. To help reduce that risk, here we present considerations to evaluate systems embedded with an ML component within a red-teaming inspired methodology. We focus on (1) cyber vulnerabilities to an ML model, (2) evaluating performance gaps, and (3) adversarial ML vulnerabilities.
97 MATHEMATICS AND COMPUTING↗
Exploring Continuous Seismic Data at an Industry Facility Using Unsupervised Machine Learning
Seismic data recorded at industrial sites contain valuable information on anthropogenic activities. With advances in machine learning and computing power, new opportunities have emerged to explore the seismic wavefield in these complex environments. We applied two unsupervised machine learning algorithms to analyze continuous seismic data collected from an industrial facility in Texas, United States. The Uniform Manifold Approximation and Projection for Dimension Reduction algorithm was used to reduce the dimensionality of the data and generate 2D embeddings. Then, the Hierarchical Density-Based Spatial Clustering of Applications with Noise method was employed to automatically group these embeddings into distinct signal clusters. Our analysis of over 1400 hr (around 59 days) of continuous seismic data revealed five and seven signal clusters at two separate stations. At both stations, we identified clusters associated with background noise and vehicle traffic, with the latter’s temporal patterns aligning closely with the facility’s work schedule. Furthermore, the algorithms detected signal clusters from unknown sources and underline the ability of unsupervised machine learning for uncovering previously unrecognized patterns. Our analysis demonstrates the effectiveness of unsupervised approaches in examining continuous seismic data without requiring prior knowledge or pre-existing labels.
58 GEOSCIENCES↗
Reinforcement Learning-Based Approach for EMT Automation of Large-Scale PV Plants
In the pursuit of efficient and precise modeling of large-scale power systems, particularly utility-scale photovoltaic (PV) plants, Electromagnetic Transient (EMT) simulations play a crucial role. As utility-scale PV plants increase in size and complexity, traditional computational methods become inadequate, necessitating more advanced techniques. This paper highlights the progressive efforts made to accelerate EMT simulations. A novel continuous reinforcement learning (RL) strategy is explored to automate the differentiation and categorization of stiff and non-stiff differential algebraic equations (DAEs). The use of stiff and non-stiff integration methods applied to relevant parts of the DAEs assists with the speed-up of the simulations. The paper details the data acquisition, development and offline training of the RL model, leading to its validation that demonstrates a high precision in optimizing simulation methods. The proposed RL promises to significantly enhance the efficacy of EMT simulations, offering a robust framework for the future of power system analysis.
Xia, Qianxue↗
Quantifying Uncertainty to Improve Decision Making in Machine Learning
Data-driven modeling, including machine learning methods, continue to play an increasing role in society. Data-driven methods impact decision making for applications ranging from everyday determinations about which news people see and control of self-driving cars to high-consequence national security situations related to cyber security and analysis of nuclear weapons reliability. Although modern machine learning methods have made great strides in model induction and show excellent performance in a broad variety of complex domains, uncertainty remains an inherent aspect of any data-driven model. In this report, we provide an update to the preliminary results on uncertainty quantification for machine learning presented in SAND2017-6776. Specifically, we improve upon the general problem definition and expand upon the experiments conducted for the earlier re- port. Most importantly, we summarize key lessons learned about how and when uncertainty quantification can inform decision making and provide valuable insights into the quality of learned models and potential improvements to them.
97 MATHEMATICS AND COMPUTING↗
The Curiosity Effect
This conference aims to improve how we learn through integrative project and discovery-based methods. My talk highlights areas in my experience as a scientist, and most recently working for our national space agency, NASA, where we work in teams with a "discovery-based" mindset. When you demonstrate broad curiosity, you become open to different viewpoints and ways to approach and manage situations. Sometimes working only from "what you have been trained to do" or "what you know" is not enough, especially when the rules may be changing. Increasing our openness in our learning, and sharing what we know, can lead to a more diverse and innovative community, solving problems in new ways, overcoming resistance to new ideas, and hopefully creating a dynamic and faring-forward society. Let us not kill curiosity, at any age, in any situation. Let us remind ourselves, at any time, in any circumstance, to continue to learn, to mentor, to stimulate, to engage and reconnect with that "open sense of possibility."
STEM↗
Additive Manufacturing, Design, Testing, and Fabrication: A Full Engineering Experience at JSC
I worked on several projects this term. While most projects involved additive manufacturing, I was also involved with two design projects, two testing projects, and a fabrication project. The primary mentor for these was Richard Hagen. Secondary mentors were Hai Nguyen, Khadijah Shariff, and fabrication training from James Brown. Overall, my experience at JSC has been successful and what I have learned will continue to help me in my engineering education and profession long after I leave. My 3D printing projects ranged from less than a 1 cubic centimeter to about 1 cubic foot and involved several printers using different printing technologies. It was exciting to become familiar with printing technologies such as industrial grade FDM (Fused Deposition Modeling), the relatively new SLA (Stereolithography), and PolyJet. My primary duty with the FDM printers was to model parts that came in from various sources to print effectively and efficiently. Using methods my mentor taught me and the Stratasys Insight software, I was able to minimize imperfections, hasten build time, improve strength for specific forces (tensile, shear, etc...), and reduce likelihood of a print-failure. Also using FDM, I learned how to repair a part after it was printed. This is done by using a special kind of glue that chemically melts the two faces of plastic parts together to form a fused interface. My first goal with SLA technology was to bring the printer back to operational readiness. In becoming familiar with the Pegasus SLA printer, I researched the leveling, laser settings, and different vats to hold liquid material. With this research, I was successfully able to bring the Pegasus back online and have successfully printed multiple sample parts as well as functional parts. My experience with PolyJet technology has been focused on an understanding of the abilities/limits, costs, and the maintenance for daily use. Still upcoming will be experience with using a composite printer that uses FDM technology to print plastic while laying an internal filament of Kevlar or carbon-fiber inside the printed material. It has been incredible being exposed to this range of technologies and I feel very fortunate to be ready for virtually any kind of printing technology I come across in the future. Design work played a part in my internship this term as well. Working with Hai Nguyen, I was able to design a set of testing tips and a test frame for use with an Arcjet. The testing tips will be made of several different materials that will possibly be used in the heat shield of the Orion space craft. These designs included technical drawings that were presented to the fabrication shop. The frame design was created from 80/20 (a popular brand of frame construction equipment) and included an order form with pricing for fabrication. An independent design was also done for the virtual reality lab. This design was to create a hand-controller based on a previous design. This final design was sent directly to a 3D printer without technical drawings. Overall, my design work has given me experience with using 80/20, helped improve my CAD (Computer Aided Design) proficiency, and increased my knowledge of how to set up technical drawings for fabrication. The final role I have played in this internship has been to assist with testing of the inflatable technology materials working Khadijah Shariff. I began the internship assisting with permeability testing with the initial plan to continue the testing independently after training. Unfortunately, the testing apparatus suffered a technical failure and had funding pulled which cancelled that portion of the project. Further testing with inflatable technology continued with tensile testing of various stitching methods. This testing took place over a two-days and concluded successfully. Final testing was to be more tensile testing but of straps used to connect various inflatable sections. Unfortunately, the needed grips for the tensile tests could not be located and put the testing on hold. It is possible this round of testing will still take place by the end of the internship if the grips can be found. Overall, this portion of the internship has helped me become familiar with one kind of permeability test as well as a popular tensile/compression testing machine. Finally, I also had the chance to be trained using a metal lathe for making very small tips for a soldering iron. These tips will be used to melt threaded brass inserts into 3D printed plastic. Working with James Brown, I was able to successfully machine a brass rod down to as little as 0.064 inches plus or minus 0.001 inches. It was very rewarding to learn how to best use the machine and become familiar with a skill that will undoubtedly be used again in the future. I have been told by several professional engineers that learning to use a lathe and mill will be invaluable skills in the field. This knocks 50 percent of that goal off and I look forward to learning the mill at some point in the future. As is apparent with this list of projects, my internship was not focused on a single over-arching goal. Instead, I was able to gain experience in a myriad of very different areas. I feel like my time here was spent bouncing from one project to the next. Though sometimes difficult to switch gears, it was very rewarding to be a part of so much in so little time. My career and education will be positively impacted by what I have learned at JSC. My experience with 3D printing has improved my ability to handle many issues that may come up in the future with multiple different technologies. The design work I took part in, especially creating technical drawings, will help me better present designs to any engineer or shop I will encounter. My testing experience has helped me become familiar with a popular kind of tensile test machine that will likely be similar to the kinds I will encounter in the future. Finally, my experience with fabrication has given me a rare opportunity, as an engineer, to take part in the fabrication of a part. This experience will help me better tailor my future designs for the manufacturing process and has given me an appreciation for detailed/delicate machining work. My experience at JSC has been successful and will continue to assist me for a long time within the engineering field.
Zusack, Steven↗
Things That Work: Roles and Services of SPDF
The current Heliophysics Science Data Management Policy (HpSDMP) defines the roles of the Space Physics Data Facility (SPDF) project as a heliophysics active Final Archive (aFA), a focus for critical data infrastructure services and a center of excellence for data and ancillary information services. This presentation will highlight (1) select current SPDF activities, (2) the lessons we are continuing to learn in how to usefully serve the the heliophysics science community and (3)SPDF's programmatic emphasis in the coming year. In cooperation with the Heliophysics Virtual discipline Observatories (VxOs), we are working closely with current, and with upcoming missions such as RBSP and MMS, to define effective approaches to ensure the long-term availability and archiving of mission data, as well as how SPDF services can complement active mission capabilities. We are working to make the Virtual Space Physics Observatory (VSPO) service comprehensive in all significant and NASA relevant heliophysics data. We will highlight a new CDAWeb interface, a faster SSCWeb, availability of our data through VxO services such as Autoplot, a new capability to easily access our data from within IDL and continuing improvements to CDF including better handling of leap seconds.
McGuire, R. E.↗
Advancements in the H-Canyon Exhaust Ventilation System Inspections at the Savannah River Site - 16030
A facility safety envelope requires management of activities to ensure no unacceptable radiological or chemical hazards to the public, site worker and environment occur. While appropriate design and construction methods ensure an initial physical plant configuration, what actions must an operating Facility take to ensure the physical plant continues to meet a safety configuration throughout Facility life? While a physical walk down of the plant is a tried and true method to confirm conditions, what other methods must be used when a walk down is not possible? What inspection methods are used when balancing budget pressures against obtaining inspection data? The periodic inspections of the H-Canyon Exhaust Tunnel are performed remotely. The 2015 remote Tunnel inspection has been the most successful inspection to date. This presentation highlights the continued lessons learned with the most recent 2015 remote Tunnel inspection. These lessons learned are applicable to Facilities considering or performing remote inspections. The Recovery Crawler Project was tasked with determining if the 2014 Inspection Crawler could be retrieved, perform the retrieval if feasible and complete the inspection of the remaining sections of the exhaust tunnel that were not previously performed. The objectives for the project were met, the Inspection Crawler was deemed unusable and left in place and the Recovery Crawler performed the complete 143 meters (470 feet) of tunnel inspection and an additional 55 meters (180 feet) of exhaust tunnel inspection not previously completed. The remote inspection of a 60+ year operating Canyon Exhaust tunnel is a complex endeavor requiring close coordination of multiple organizations, adherence to Conduct of Operations, Radiological Safety, Worker Safety, etc. principles as well as alignment with Facility Safety Basis requirements. This paper discusses the challenges, actions required getting ready for the 2015 recovery and continued inspection and the hazards encountered performing the actual recovery/inspection. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning
Residential heating, ventilation, and air conditioning (HVAC) has been considered as an important demand response resource. However, the optimization of residential HVAC control is no trivial task due to the complexity of the thermal dynamic models of buildings and uncertainty associated with both occupant-driven heat loads and weather forecasts. In this paper, we apply a novel model-free deep reinforcement learning (RL) method, known as the deep deterministic policy gradient (DDPG), to generate an optimal control strategy for a multi-zone residential HVAC system with the goal of minimizing energy consumption cost while maintaining the users’ comfort. Here, the applied deep RL-based method learns through continuous interaction with a simulated building environment and without referring to any prior model knowledge. Simulation results show that compared with the state-of-art deep Q network (DQN), the DDPG-based HVAC control strategy can reduce the energy consumption cost by 15% and reduce the comfort violation by 79%; and when compared with a rule-based HVAC control strategy, the comfort violation can be reduced by 98%. In addition, experiments with different building models and retail price models demonstrate that the well-trained DDPG-based HVAC control strategy has high generalization and adaptability to unseen environments, which indicates its practicability for real-world implementation.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗