Search NASA⌕ Search

SEARCH · Search NASA

Results for “goal based control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37

Update on Subsonic Single Aft Engine (SUSAN) Electrofan Trade Space Exploration

NASA is conducting an ongoing trade study analysis of the SUSAN Electrofan aircraft concept, which utilizes 20-MW-class electrified aircraft propulsion to enable propulsive, aerodynamic, and control benefits while retaining the range, speed, and size of typical narrow-body regional aircraft. The study is constrained by the ground rules of operating within the current airport and airspace infrastructure. This ongoing study seeks to find a configuration and combination of technologies that yield significant fuel burn and emissions benefits. Another key goal is to reduce cost per passenger mile. Currently, the study is focused on a configuration that utilizes jet A or sustainable aviation fuels, however, we plan to consider other fuel alternatives in the future. This paper describes the progress in defining the architecture of the aircraft, engine, power system, control system, and initial understandings of the sensitivity of the potential configurations to technology assumptions based on key performance parameters. Additionally, progress towards definition and refinement of driving operational, economic, infrastructure, certification, and technical requirements is discussed.

aircraft concept↗

Update on Subsonic Single Aft Engine (SUSAN) Electrofan Trade Space Exploration

NASA is conducting an ongoing trade study analysis of the SUSAN Electrofan aircraft concept, which utilizes 20-MW-class electrified aircraft propulsion to enable propulsive, aerodynamic, and control benefits while retaining the range, speed, and size of typical narrow-body regional aircraft. The study is constrained by the ground rules of operating within the current airport and airspace infrastructure. This ongoing study seeks to find a configuration and combination of technologies that yield significant fuel burn and emissions benefits. Another key goal is to reduce cost per passenger mile. Currently, the study is focused on a configuration that utilizes jet A or sustainable aviation fuels, however, we plan to consider other fuel alternatives in the future. This presentation describes the progress in defining the architecture of the aircraft, engine, power system, control system, and initial understandings of the sensitivity of the potential configurations to technology assumptions based on key performance parameters. Additionally, progress towards definition and refinement of driving operational, economic, infrastructure, certification, and technical requirements is discussed.

Ralph H. Jansen↗

NASA LUNABOTICS ROBOTIC MINING COMPETITION 10TH ANNIVERSARY (2010 – 2019): TAXONOMY AND TECHNOLOGY REVIEW

Space Mining for resources such as water ice, and regolith, which contain many elements in the form of metals, minerals, volatiles and other compounds, is a necessary step for In-Situ Space Resource Utilization (ISRU). One of the primary goals is to extract propellants from the regolith and water ice, such as oxygen and hydrogen which could then be used for in-space transportation. In addition, the space mining system can be used for various construction tasks that can benefit human and robotic exploration as well as scientific investigations based on excavated exposed topography, such as the side walls of trenches. The National Aeronautics & Space Ad-ministration (NASA) Lunabotics Robotic Mining Competition (RMC) is a university-level competition designed to engage and retain students in science, technology, engineering and mathematics (STEM). NASA has directly benefited from the competition by encouraging the development of innovative lunar excavation concepts from universities which has resulted in clever ideas and solutions which could be applied to an actual lunar excavation device or payload. The challenge is for students to design and build a remote controlled or autonomous excavator, called a lunabot, that can collect and deposit a minimum of 10 kilograms of lunar simulant within 15 minutes. In recent years the goal has been changed to excavate a minimum of 1 kg of simulated icy regolith which is found under an over-burden of regolith simulant. The complexities of the challenge include the abrasive characteristics of the lunar simulant, the weight and size limitations of the lunabot, and the ability to control the lunabot from a remote control center. This paper will present the results of the 10 Lunabotics Robotic Mining Competitions held between May 2010 and May 2019. Each year over 50 university teams have attended, resulting in over 500 lunabot designs and subsequent prototypes. Over 5,000 university students have been part of the competition on-site at KSC. More students and the public were en-gaged via internet broadcasting and social net-working media. The various designs will be cataloged and categorized to provide information to future Lunabotics mining robot designers and competitors. It is also expected to be of value for actual future space missions, as knowledge is gained from testing many innovative prototypes in simulated lunar regoith. A taxonomy of robotic excavator designs will be presented.

Competition↗

Plasma shape and position control development for NSTX-U using the GSEvolve plasma simulator

NSTX-U discharges have been reproduced using the GSEvolve plasma simulator in order to improve their control and reduce oscillations during the plasma current (Ip) ramp-up phase, and with the ultimate goal of enabling reliable H-mode access with a double-null shape at high elongation, low internal inductance, and high plasma performance. The control objective is to eliminate undesired oscillations in the plasma shape and vertical position that start when the plasma diverts during the Ip ramp-up phase, and that happened recurrently during the first NSTX-U experimental campaign. Model-based control solutions have been developed, including alternative H-mode access recipes and a Multi-Input Multi-Output (MIMO) decoupling controller. These solutions have demonstrated their capability to reduce or eliminate the aforementioned oscillations in GSEvolve simulations, enabling reproducible H-mode access. The results of this work aim to inform NSTX-U experimental operations on potential control developments to maximize its scientific output, and highlight the powerful capabilities of high-fidelity control-simulation frameworks like GSEvolve.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Wind tunnel investigation of vortex flows on F/A-18 configuration at subsonic through transonic speed

A wind tunnel experiment was conducted in the David Taylor Research Center 7- by 10-Foot Transonic Tunnel of the wing leading-edge extension (LEX) and forebody vortex flows at subsonic and transonic speeds about a 0.06-scale model of the F/A-18. The primary goal was to improve the understanding and control of the vortical flows, including the phenomena of vortex breakdown and vortex interactions with the vertical tails. Laser vapor screen flow visualizations, LEX, and forebody surface static pressures, and six-component forces and moments were obtained at angles of attack of 10 to 50 degrees, free-stream Mach numbers of 0.20 to 0.90, and Reynolds numbers based on the wing mean aerodynamic chord of 0.96 x 10(exp 6) to 1.75 x 10(exp 6). The wind tunnel results were correlated with in-flight flow visualizations and handling qualities trends obtained by NASA using an F-18 High-Alpha Research Vehicle (HARV) and by the Navy and McDonnell Douglas on F-18 aircraft with LEX fences added to improve the vertical tail buffet environment. Key issues that were addressed include the sensitivity of the vortical flows to the Reynolds number and Mach number; the reduced vertical tail excitation, and the corresponding flow mechanism, in the presence of the LEX fence; the repeatability of data obtained during high angle-of-attack wind tunnel testing of F-18 models; the effects of particle seeding for flow visualization on the quantitative model measurements; and the interpretation of off-body flow visualizations obtained using different illumination and particle seeding techniques.

Erickson, Gary E.↗

Expert System for UNIX System Reliability and Availability Enhancement

Highly reliable and available systems are critical to the airline industry. However, most off-the-shelf computer operating systems and hardware do not have built-in fault tolerant mechanisms, the UNIX workstation is one example. In this research effort, we have developed a rule-based Expert System (ES) to monitor, command, and control a UNIX workstation system with hot-standby redundancy. The ES on each workstation acts as an on-line system administrator to diagnose, report, correct, and prevent certain types of hardware and software failures. If a primary station is approaching failure, the ES coordinates the switch-over to a hot-standby secondary workstation. The goal is to discover and solve certain fatal problems early enough to prevent complete system failure from occurring and therefore to enhance system reliability and availability. Test results show that the ES can diagnose all targeted faulty scenarios and take desired actions in a consistent manner regardless of the sequence of the faults. The ES can perform designated system administration tasks about ten times faster than an experienced human operator. Compared with a single workstation system, our hot-standby redundancy system downtime is predicted to be reduced by more than 50 percent by using the ES to command and control the system.

Xu, Catherine Q.↗

Code validation for the simulation of supersonic viscous flow about the F-16XL

The viewgraphs and discussion on code validation for the simulation of supersonic viscous flow about the F-16XL are provided. Because of the large potential gains related to laminar flow on the swept wings of supersonic aircraft, interest in the applications of laminar flow control (LFC) techniques in the supersonic regime has increased. A supersonic laminar flow control (SLFC) technology program is currently underway within NASA. The objective of this program is to develop the data base and design methods that are critical to the development of laminar flow control technology for application to supersonic transport aircraft design. Towards this end, the program integrates computational investigations underway at NASA Ames-Moffett and NASA Langley with flight-test investigations being conducted on the F-16XL at the NASA Ames-Dryden Research Facility in cooperation with Rockwell International. The computational goal at NASA Ames-Moffett is to integrate a thin-layer Reynolds averaged Navier-Stokes flow solver with a stability analysis code. The flow solver would provide boundary layer profiles to the stability analysis code which in turn would predict transition on the F-16XL wing. To utilize the stability analysis codes, reliable boundary layer data is necessary at off-design cases. Previously, much of the prediction of boundary layer transition has been accomplished through the coupling of boundary layer codes with stability theory. However, boundary layer codes may have difficulties at high Reynolds numbers, of the order of 100 million, and with the current complex geometry in question. Therefore, a reliable code which solves the thin-layer Reynolds averaged Navier-Stokes equations is needed. Two objectives are discussed, the first in greater depth. The first objective is method verification, via comparisons of computations with experiment, of the reliability and robustness of the code. To successfully implement LFC techniques to the F-16XL wing, the flow about the leading edge must be maintained as laminar flow. Therefore, the second objective is to focus on a series of numerical simulations with different values of angle of attack, alpha, and Reynolds numbers. The purpose of the simulations is to study their effects on the two main factors which precipitate transition to turbulence at leading edges of highly swept wings (e.g., 'spanwise contamination' and 'crossflow instability').

Flores, Jolen↗

Cockpit Adaptive Automation and Pilot Performance

The introduction of high-level automated systems in the aircraft cockpit has provided several benefits, e.g., new capabilities, enhanced operational efficiency, and reduced crew workload. At the same time, conventional 'static' automation has sometimes degraded human operator monitoring performance, increased workload, and reduced situation awareness. Adaptive automation represents an alternative to static automation. In this approach, task allocation between human operators and computer systems is flexible and context-dependent rather than static. Adaptive automation, or adaptive task allocation, is thought to provide for regulation of operator workload and performance, while preserving the benefits of static automation. In previous research we have reported beneficial effects of adaptive automation on the performance of both pilots and non-pilots of flight-related tasks. For adaptive systems to be viable, however, such benefits need to be examined jointly in the context of a single set of tasks. The studies carried out under this project evaluated a systematic method for combining different forms of adaptive automation. A model for effective combination of different forms of adaptive automation, based on matching adaptation to operator workload was proposed and tested. The model was evaluated in studies using IFR-rated pilots flying a general-aviation simulator. Performance, subjective, and physiological (heart rate variability, eye scan-paths) measures of workload were recorded. The studies compared workload-based adaptation to to non-adaptive control conditions and found evidence for systematic benefits of adaptive automation. The research provides an empirical basis for evaluating the effectiveness of adaptive automation in the cockpit. The results contribute to the development of design principles and guidelines for the implementation of adaptive automation in the cockpit, particularly in general aviation, and in other human-machine systems. Project goals were met or exceeded. The results of the research extended knowledge of automation-related performance decrements in pilots and demonstrated the positive effects of adaptive task allocation. In addition, several practical implications for cockpit automation design were drawn from the research conducted. A total of 12 articles deriving from the project were published.

Parasuraman, Raja↗

Shortcomings with Tree-Structured Edge Encodings for Neural Networks

In evolutionary algorithms a common method for encoding neural networks is to use a tree structured assembly procedure for constructing them. Since node operators have difficulties in specifying edge weights and these operators are execution-order dependent, an alternative is to use edge operators. Here we identify three problems with edge operators: in the initialization phase most randomly created genotypes produce an incorrect number of inputs and outputs; variation operators can easily change the number of input/output (I/O) units; and units have a connectivity bias based on their order of creation. Instead of creating I/O nodes as part of the construction process we propose using parameterized operators to connect to preexisting I/O units. Results from experiments show that these parameterized operators greatly improve the probability of creating and maintaining networks with the correct number of I/O units, remove the connectivity bias with I/O units and produce better controllers for a goal-scoring task.

Hornby, Gregory S.↗

Generation of Electrical Power from Stimulated Muscle Contractions Evaluated

This project is a collaborative effort between NASA Glenn Research Center's Revolutionary Aeropropulsion Concepts (RAC) Project, part of the NASA Aerospace Propulsion and Power Program of the Aerospace Technology Enterprise, and Case Western Reserve University's Cleveland Functional Electrical Stimulation (FES) Center. The RAC Project foresees implantable power requirements for future applications such as organically based sensor platforms and robotics that can interface with the human senses. One of the goals of the FES Center is to develop a totally implantable neural prosthesis. This goal is based on feedback from patients who would prefer a system with an internal power source over the currently used system with an external power source. The conversion system under investigation would transform the energy produced from a stimulated muscle contraction into electrical energy. We hypothesize that the output power of the system will be greater than the input power necessary to initiate, sustain, and control the electrical conversion system because of the stored potential energy of the muscle. If the system can be made biocompatible, durable, and with the potential for sustained use, then the biological power source will be a viable solution.

Lewandowski, Beth↗

Acceptability of Flight Deck-Based Interval Management Crew Procedures

The Interval Management for Near-term Operations Validation of Acceptability (IM-NOVA) experiment was conducted at the National Aeronautics and Space Administration (NASA) Langley Research Center (LaRC) in support of the NASA Next Generation Air Transportation System (NextGen) Airspace Systems Program's Air Traffic Management Technology Demonstration - 1 (ATD-1). ATD-1 is intended to showcase an integrated set of technologies that provide an efficient arrival solution for managing aircraft using NextGen surveillance, navigation, procedures, and automation for both airborne and ground-based systems. The goal of the IM-NOVA experiment was to assess if procedures outlined by the ATD-1 Concept of Operations, when used with a minimum set of Flight deck-based Interval Management (FIM) equipment and a prototype crew interface, were acceptable to and feasible for use by flight crews in a voice communications environment. To investigate an integrated arrival solution using ground-based air traffic control tools and aircraft automatic dependent surveillance broadcast (ADS-B) tools, the LaRC FIM system and the Traffic Management Advisor with Terminal Metering and Controller Managed Spacing tools developed at the NASA Ames Research Center (ARC) were integrated in LaRC's Air Traffic Operations Laboratory. Data were collected from 10 crews of current, qualified 757/767 pilots asked to fly a high-fidelity, fixed based simulator during scenarios conducted within an airspace environment modeled on the Dallas-Fort Worth (DFW) Terminal Radar Approach Control area. The aircraft simulator was equipped with the Airborne Spacing for Terminal Area Routes algorithm and a FIM crew interface consisting of electronic flight bags and ADS-B guidance displays. Researchers used "pseudo-pilot" stations to control 24 simulated aircraft that provided multiple air traffic flows into DFW, and recently retired DFW air traffic controllers served as confederate Center, Feeder, Final, and Tower controllers. Pilot participant feedback indicated that the procedures used by flight crews to receive and execute interval management (IM) clearances in a voice communications environment were logical, easy to follow, did not contain any missing or extraneous steps, and required the use of an acceptable level of workload. The majority of the pilot participants found the IM concept, in addition to the proposed FIM crew procedures, to be acceptable and indicated that the ATD-1 procedures can be successfully executed in a near-term NextGen environment.

Murdock, Jennifer L.↗

A Psychoacoustic Evaluation of Noise Signatures from Advanced Civil Transport Aircraft

The NASA Environmentally Responsible Aviation project has been successful in developing and demonstrating technologies for integrated aircraft systems that can simultaneously meet aggressive goals for fuel burn, noise and emissions. Some of the resulting systems substantially differ from the familiar tube and wing designs constituting the current civil transport fleet. This study attempts to explore whether or not the effective perceived noise level metric used in the NASA noise goal accurately reflects human subject response across the range of vehicles considered. Further, it seeks to determine, in a quantitative manner, if the sounds associated with the advanced aircraft are more or less preferable to the reference vehicles beyond any differences revealed by the metric. These explorations are made through psychoacoustic tests in a controlled laboratory environment using simulated stimuli developed from auralizations of selected vehicles based on systems noise assessments.

Rizzi, Stephen A.↗

NASA JSC’s Simulant Development Lab Capabilities and Artemis Testing

The Simulant Development Lab (SDL) is a multifunctional collaborative workspace that supports the development, curation, analysis, testing, and distribution of planetary regolith simulants – including lunar, Martian, asteroidal, and other granular materials. The lab provides a multidisciplinary setting for scientific characterization of simulant physical properties and for engineering evaluations conducted with simulant test beds. To enable this work, the SDL curates and maintains a stock of more than 35 metric tons of simulant material. To evaluate these materials and support testing goals, the lab is equipped with a comprehensive suite of processing tools and analytical instruments. These capabilities enable the SDL’s mission at NASA’s Johnson Space Center to distribute, develop, process, characterize, and test regolith simulants for mission relevant applications. Through controlled and repeatable testing environments that replicate the physical and compositional properties of lunar regolith, the SDL supports Artemis hardware maturation, providing safe, Earth‑based analogs for evaluating systems that must withstand regolith dust interactions, physical wear and abrasion, and operational loads. The facility’s extensive simulant inventory and integrated geological and engineering test infrastructure accelerate technology readiness for Artemis and future exploration campaigns (e.g., future crewed or robotic missions to Mars).

Simulant Development Lab↗

The NASA Continuous Risk Management Process

As an intern this summer in the GRC Risk Management Office, I have become familiar with the NASA Continuous Risk Management Process. In this process, risk is considered in terms of the probability that an undesired event will occur and the impact of the event, should it occur (ref., NASA-NPG: 7120.5). Risk management belongs in every part of every project and should be ongoing from start to finish. Another key point is that a risk is not a problem until it has happened. With that in mind, there is a six step cycle for continuous risk management that prevents risks from becoming problems. The steps are: identify, analyze, plan, track, control, and communicate & document. Incorporated in the first step are several methods to identify risks such as brainstorming and using lessons learned. Once a risk is identified, a risk statement is made on a risk information sheet consisting of a single condition and one or more consequences. There can also be a context section where the risk is explained in more detail. Additionally there are three main goals of analyzing a risk, which are evaluate, classify, and prioritize. Here is where a value is given to the attributes of a risk &e., probability, impact, and timeframe) based on a multi-level classification system (e.g., low, medium, high). It is important to keep in mind that the definitions of these levels are probably different for each project. Furthermore the risks can be combined into groups. Then, the risks are prioritized to see what risk is necessary to mitigate first. After the risks are analyzed, a plan is made to mitigate as many risks as feasible. Each risk should be assigned to someone in the project with knowledge in the area of the risk. Then the possible approaches to choose from are: research, accept, watch, or mitigate. Next, all risks, mitigated or not, are tracked either individually or in groups. As the plan is executed, risks are re-evaluated, and the attribute values are adjusted as necessary. Metrics are established and monitored as tools for risk tracking. Also a trigger or threshold should be set on the metric data that indicates when an action is needed. Results of this tracking are usually evaluated and reported in a relevant format at weekly or monthly meetings. Choosing controls is the subsequent step, which involves the effects of the tracking. The three basic controls are: close, continue tracking, and re- plan. Finally communicate & document is the last step, but occurs throughout the process. It is vital that main risks, plans, changes, and progress are known by everyone in the project. A good way to keep everyone updated and inform other projects of common issues is by thoroughly documenting project risks. NASA sees value in risk management and believes that projects have greater probability or success by using the NASA Continuous Risk Management Process.

Pokorny, Frank M.↗

Lunabotics Robotic Mining Competition for Universities

Space Mining for resources such as water ice, and regolith, which contain many elements in the form of metals, minerals, volatiles and other compounds, is a necessary step for In-Situ Space Resource Utilization (ISRU). One of the primary goals is to extract propellants from the regolith and water ice, such as oxygen and hydrogen which could then be used for in-space transportation. In addition, the space mining system can be used for various construction tasks that can benefit human and robotic exploration as well as scientific investigations based on excavated exposed topography, such as the side walls of trenches. The National Aeronautics & Space Administration (NASA) “Lunabotics” Robotic Mining Competition (RMC) is a university-level competition designed to engage and retain students in science, technology, engineering and mathematics (STEM). NASA has directly benefited from the competition by encouraging the development of innovative lunar excavation concepts from universities which has resulted in clever ideas and solutions which could be applied to an actual lunar excavation device or payload. The challenge is for students to design and build a remote controlled or autonomous 80 kg mass lightweight excavator, called a “lunabot”, which can collect and deposit a minimum of 10 kilograms of lunar simulant within 15 minutes. In recent years the goal has been changed to excavate a minimum of 1 kg of simulated icy regolith which is found under an overburden of regolith simulant. The complexities of the challenge include the abrasive characteristics of the lunar regolith simulant, the weight and size limitations of the lunabot, and the ability to control the lunabot from a remote mission control center or operate it autonomously. This presentation will summarize the results of the Lunabotics Robotic Mining Competitions held in May 2022. Each year, since 2010, over 50 university teams have attended, resulting in over 500 lunabot designs and subsequent prototypes. Over 6,000 university students have been part of the on-site competition at KSC, and many are employed in the space sector today with thriving careers. Even more students and the public were engaged via internet broadcasting and social networking media. Examples of how this translates to hiring opportunities for commercial sponsors will also be discussed. Overall, this competition has proven to be popular with multi-disciplinary university senior design classes and a valuable tool for NASA and academia in educating and inspiring the next generation of space engineers.

Lunabotics↗

Comparing variability in diagnosis of upper respiratory tract infections in patients using syndromic, next generation sequencing, and PCR-based methods

Early and accurate diagnosis of respiratory pathogens and associated outbreaks can allow for the control of spread, epidemiological modeling, targeted treatment, and decision making–as is evident with the current COVID-19 pandemic. Many respiratory infections share common symptoms, making them difficult to diagnose using only syndromic presentation. Yet, with delays in getting reference laboratory tests and limited availability and poor sensitivity of point-of-care tests, syndromic diagnosis is the most-relied upon method in clinical practice today. Here, we examine the variability in diagnostic identification of respiratory infections during the annual infection cycle in northern New Mexico, by comparing syndromic diagnostics with polymerase chain reaction (PCR) and sequencing-based methods, with the goal of assessing gaps in our current ability to identify respiratory pathogens. Of 97 individuals that presented with symptoms of respiratory infection, only 23 were positive for at least one RNA virus, as confirmed by sequencing. Whereas influenza virus (n = 7) was expected during this infection cycle, we also observed coronavirus (n = 7), respiratory syncytial virus (n = 8), parainfluenza virus (n = 4), and human metapneumovirus (n = 1) in individuals with respiratory infection symptoms. Four patients were coinfected with two viruses. In 21 individuals that tested positive using PCR, RNA sequencing completely matched in only 12 (57%) of these individuals. Few individuals (37.1%) were diagnosed to have an upper respiratory tract infection or viral syndrome by syndromic diagnostics, and the type of virus could only be distinguished in one patient. Thus, current syndromic diagnostic approaches fail to accurately identify respiratory pathogens associated with infection and are not suited to capture emerging threats in an accurate fashion. We conclude there is a critical and urgent need for layered agnostic diagnostics to track known and unknown pathogens at the point of care to control future outbreaks.

60 APPLIED LIFE SCIENCES↗

Optimizing Control for Efficient Load Shifting with Thermal Energy Storage in Existing HVAC Systems

This project developed the integration of a direct-contact heat exchanger (DCHX) based thermal energy storage (TES) system with a chiller–air handling unit (AHU) plant to evaluate its potential for reducing building energy costs. Detailed physical models of the TES unit, building envelope, and HVAC components were developed alongside simplified control-oriented models to support both high-fidelity simulation and real-time optimization. Two control strategies were implemented and compared: a rule-based control (RBC) aligned with utility time-of-use (ToU) rates, and a model predictive control (MPC) framework leveraging forecasts of building load, weather, and internal gains.Simulation results show that the RBC strategy reduced daily electricity costs by around 30% by shifting cooling production from on-peak to off-peak hours. In contrast, the MPC strategy achieved significantly greater performance, reducing daily operating costs by up to 44% and peak-hour costs by more than 60%. Both strategies maintained indoor thermal comfort within acceptable limits, with MPC further improving load distribution and reducing equipment cycling.The outcomes confirm that TES integration, particularly when coordinated with advanced predictive control, can provide substantial cost savings and on-peak demand reduction. . These findings directly support the U.S. Department of Energy’s goals for grid-interactive efficient buildings and demonstrate the potential of TES-enabled HVAC systems for scalable deployment across the commercial building.

99 GENERAL AND MISCELLANEOUS↗

Reinforcement learning building control approach harnessing imitation learning

Reinforcement learning (RL) has shown significant success in sequential decision making in fields like autonomous vehicles, robotics, marketing and gaming industries. This success has attracted the attention to the RL control approach for building energy systems which are becoming complicated due to the need to optimize for multiple, potentially conflicting, goals like occupant comfort, energy use and grid interactivity. However, for real world applications, RL has several drawbacks like requiring large training data and time, and unstable control behavior during the early exploration process making it infeasible for an application directly to building control tasks. To address these issues, an imitation learning approach is utilized herein where the RL agents starts with a policy transferred from accepted rule based policies and heuristic policies. This approach is successful in reducing the training time, preventing the unstable early exploration behavior and improving upon an accepted rule-based policy - all of these make RL a more practical control approach for real world applications in the domain of building controls.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗