Search NASA⌕ Search

SEARCH · Search NASA

Results for “Real Time Optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

A Secure and Adaptive Hierarchical Multi-Timescale Framework for Resilient Load Restoration Using a Community Microgrid

Distribution system integrated community microgrids (CMGs) can partake in restoring loads during extended duration outages. At such times, the CMGs are challenged with limited resource availability, absence of robust grid support, and heightened demand-supply uncertainty. Here, this paper proposes a secure and adaptive three-stage hierarchical multi-timescale framework for scheduling and real-time (RT) dispatch of CMGs with hybrid PV systems to address these challenges. The framework enables the CMG to dynamically expand its boundary to support the neighboring grid sections and is adaptive to the changing forecast error impacts. The first stage solves a stochastic extended duration scheduling (EDS) problem to obtain referral plans for optimal resource rationing. The intermediate near-real-time (NRT) scheduling stage updates the EDS schedule closer to the dispatch time using new obtained forecasts, followed by the RT dispatch stage. To make the decisions more secure and robust against forecast errors, a novel concept called delayed recourse is designed. The approach is evaluated via numerical simulations on a modified IEEE 123-bus system and validated using OpenDSS and hardware-in-loop simulations. The results show superior performance in maximizing load supply and continuous secure distribution network operation under different operating scenarios.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Optimal Power Flow Pursuit

Past works that focused on addressing power-quality and reliability concerns related to renewable energy resources (RESs) operating with business-as-usual practices have looked at the design of Volt/VAr and Volt/Watt strategies to regulate real or reactive powers based on local voltage measurements, so that terminal voltages are within acceptable levels. These control strategies have the potential of operating at the same time scale of distribution-system dynamics, and can therefore mitigate disturbances precipitated fast time-varying loads and ambient conditions; however, they do not necessarily guarantee system-level optimality, and stability claims are mainly based on empirical evidences. On a different time scale, centralized and distributed optimal power flow (OPF) algorithms have been proposed to compute optimal steady-state inverter setpoints, so that power losses and voltage deviations are minimized and economic benefits to end-users providing ancillary services are maximized. However, traditional OPF schemes may offer decision making capabilities that do not match the dynamics of distribution systems. Particularly, during the time required to collect data from all the nodes of the network (e.g., loads), solve the OPF, and subsequently dispatch setpoints, the underlying load, ambient, and network conditions may have already changed; in this case, the DER output powers would be consistently regulated around outdated setpoints, leading to suboptimal system operation and violation of relevant electrical limits. The present work focuses on the synthesis of distributed RES-inverter controllers that leverage the opportunities for fast feedback offered by power-electronics interfaced RESs. The overarching objective is to bridge the temporal gap between long-term system optimization and real-time control, to enable seamless RES integration in large scale with stability and efficiency guarantees, while congruently pursuing system-level optimization objectives. The design of the control framework is based on suitable linear approximations of the AC power-flow equations as well as Lagrangian regularization methods. The proposed controllers enable an update of the power outputs at a time scale that is compatible with the underlying dynamics of loads and ambient conditions, and continuously drive the system operation towards OPF-based solutions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

MO-DE-BRA-04: Hands-On Fluoroscopy Safety Training with Real-Time Patient and Staff Dosimetry

Purpose: Fluoroscopically guided interventions (FGI) are routinely performed across many different hospital departments. However, many involved staff members have minimal training regarding safe and optimal use of fluoroscopy systems. We developed and taught a hands-on fluoroscopy safety class incorporating real-time patient and staff dosimetry in order to promote safer and more optimal use of fluoroscopy during FGI. Methods: The hands-on fluoroscopy safety class is taught in an FGI suite, unique to each department. A patient equivalent phantom is set on the patient table with an ion chamber positioned at the x-ray beam entrance to the phantom. This provides a surrogate measure of patient entrance dose. Multiple solid state dosimeters (RaySafe i2 dosimetry systemTM) are deployed at different distances from the phantom (0.1, 1, 3 meters), which provide surrogate measures of staff dose. Instructors direct participating clinical staff to operate the fluoroscopy system as they view live fluoroscopic images, patient entrance dose, and staff doses in real-time. During class, instructors work with clinical staff to investigate how patient entrance dose, staff doses, and image quality are affected by different parameters, including pulse rate, magnification, collimation, beam angulation, imaging mode, system geometry, distance, and shielding. Results: Real-time dose visualization enables clinical staff to directly see and learn how to optimize their use of their own fluoroscopy system to minimize patient and staff dose, yet maintain sufficient image quality for FGI. As a direct result of the class, multiple hospital departments have implemented changes to their imaging protocols, including reduction of the default fluoroscopy pulse rate and increased use of collimation and lower dose fluoroscopy modes. Conclusion: Hands-on fluoroscopy safety training substantially benefits from real-time patient and staff dosimetry incorporated into the class. Real-time dose display helps clinical staff visualize, internalize, and ultimately utilize the safety techniques learned during the training. RaySafe/Unfors/Fluke lent us a portable version of their RaySafe i2 Dosimetry System for 6 months.

60 APPLIED LIFE SCIENCES↗

HURON (HUman and Robotic Optimization Network) Multi-Agent Temporal Activity Planner/Scheduler

HURON solves the problem of how to optimize a plan and schedule for assigning multiple agents to a temporal sequence of actions (e.g., science tasks). Developed as a generic planning and scheduling tool, HURON has been used to optimize space mission surface operations. The tool has also been used to analyze lunar architectures for a variety of surface operational scenarios in order to maximize return on investment and productivity. These scenarios include numerous science activities performed by a diverse set of agents: humans, teleoperated rovers, and autonomous rovers. Once given a set of agents, activities, resources, resource constraints, temporal constraints, and de pendencies, HURON computes an optimal schedule that meets a specified goal (e.g., maximum productivity or minimum time), subject to the constraints. HURON performs planning and scheduling optimization as a graph search in state-space with forward progression. Each node in the graph contains a state instance. Starting with the initial node, a graph is automatically constructed with new successive nodes of each new state to explore. The optimization uses a set of pre-conditions and post-conditions to create the children states. The Python language was adopted to not only enable more agile development, but to also allow the domain experts to easily define their optimization models. A graphical user interface was also developed to facilitate real-time search information feedback and interaction by the operator in the search optimization process. The HURON package has many potential uses in the fields of Operations Research and Management Science where this technology applies to many commercial domains requiring optimization to reduce costs. For example, optimizing a fleet of transportation truck routes, aircraft flight scheduling, and other route-planning scenarios involving multiple agent task optimization would all benefit by using HURON.

Hua, Hook↗

Molecular Surface Sampling and Chemical Imaging using Proximal Probe Thermal Desorption/Secondary Ionization Mass Spectrometry

Proximal probe thermal desorption/secondary ionization mass spectrometry was studied and applied to molecular surface sampling and chemical imaging using printed patterns on photopaper as test substrates. With the use of a circular cross section proximal probe with a tip diameter of 50 m and fixed temperature (350 C), the influence of probe-to-surface distance, lane scan spacing, and surface scan speed on signal quality and spatial resolution were studied and optimized. As a compromise between signal amplitude, signal reproducibility, and data acquisition time, a surface scan speed of 100 m/s, probe-to-paper surface distance of 5 m, and lane spacing of 10 m were used for imaging. Under those conditions the proximal probe thermal desorption/secondary ionization mass spectrometry method was able to achieve a spatial resolution of about 50 m as determined by the ability to distinguish surface patterns of known dimensions that were printed on the paper substrate. It is expected that spatial resolution and chemical image quality could be further improved by using probes of smaller cross section size and by incorporating a means to maintain a fixed optimal probe-to-surface distance real time, continuously adapting to the changing topography of the surface during a lane scan.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

The first clinical implementation of electromagnetic transponder-guided MLC tracking

Purpose: We report on the clinical process, quality assurance, and geometric and dosimetric results of the first clinical implementation of electromagnetic transponder-guided MLC tracking which occurred on 28 November 2013 at the Northern Sydney Cancer Centre. Methods: An electromagnetic transponder-based positioning system (Calypso) was modified to send the target position output to in-house-developed MLC tracking code, which adjusts the leaf positions to optimally align the treatment beam with the real-time target position. Clinical process and quality assurance procedures were developed and performed. The first clinical implementation of electromagnetic transponder-guided MLC tracking was for a prostate cancer patient being treated with dual-arc VMAT (RapidArc). For the first fraction of the first patient treatment of electromagnetic transponder-guided MLC tracking we recorded the in-room time and transponder positions, and performed dose reconstruction to estimate the delivered dose and also the dose received had MLC tracking not been used. Results: The total in-room time was 21 min with 2 min of beam delivery. No additional time was needed for MLC tracking and there were no beam holds. The average prostate position from the initial setup was 1.2 mm, mostly an anterior shift. Dose reconstruction analysis of the delivered dose with MLC tracking showed similar isodose and target dose volume histograms to the planned treatment and a 4.6% increase in the fractional rectal V{sub 60}. Dose reconstruction without motion compensation showed a 30% increase in the fractional rectal V{sub 60} from that planned, even for the small motion. Conclusions: The real-time beam-target correction method, electromagnetic transponder-guided MLC tracking, has been translated to the clinic. This achievement represents a milestone in improving geometric and dosimetric accuracy, and by inference treatment outcomes, in cancer radiotherapy.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

A Real-Time Big Data Control-Theoretical Framework for Cyber-Physical-Human Systems [Book Chapter]

It is known that cyber-physical-human systems naturally arise from interdependent infrastructure systems and smart connected communities. Such applications require ubiquitous information sensing and processing, intelligent machine-to-machine communication for a seamless coordination, as well as intelligent interactions between humans and machines. This chapter presents a control-theoretical framework to model heterogeneous physical dynamic systems, information and communication, as well as cooperative controls and/or distributed optimization of such interconnected systems. It is shown that efficient analytical and computational algorithms can be modularly designed and hierarchically implemented to operate and optimize cyber-physical-human systems, first to quantify individually the input–output relationship of nonlinear dynamic behaviors of every physical subsystem, then to coordinate locally both cyber-physical interactions of neighboring agents as well as physical-human interactions, and finally to dynamically model and optimize the overall networked system. The hierarchical structure makes the overall optimization and control problem scalable and solvable. Moreover, the three levels integrate individual designs and optimization, distributed cooperative optimization, and decision-making through real-time, data-driven, model-based learning and control. Specifically, one of the contributions of the chapter is to demonstrate how the combination of dissipativity theory and cooperative control serves as a natural framework and promising tools to analyze, optimize, and control such large-scale system. Application to digital grid is investigated as an illustrative example.

97 MATHEMATICS AND COMPUTING↗

OPAD-EDIFIS Real-Time Processing

The Optical Plume Anomaly Detection (OPAD) detects engine hardware degradation of flight vehicles through identification and quantification of elemental species found in the plume by analyzing the plume emission spectra in a real-time mode. Real-time performance of OPAD relies on extensive software which must report metal amounts in the plume faster than once every 0.5 sec. OPAD software previously written by NASA scientists performed most necessary functions at speeds which were far below what is needed for real-time operation. The research presented in this report improved the execution speed of the software by optimizing the code without changing the algorithms and converting it into a parallelized form which is executed in a shared-memory multiprocessor system. The resulting code was subjected to extensive timing analysis. The report also provides suggestions for further performance improvement by (1) identifying areas of algorithm optimization, (2) recommending commercially available multiprocessor architectures and operating systems to support real-time execution and (3) presenting an initial study of fault-tolerance requirements.

Katsinis, Constantine↗

Advancing Additive Manufacturing Through Artificial Intelligence–Powered, High-Throughput, Nondestructive Characterization and Process Optimization

This Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and ZEISS Industrial Metrology has demonstrated the transformative potential of artificial intelligence (AI)-enabled x-ray computed tomography (XCT) to accelerate the qualification and certification of additively manufactured (AM) parts. At the core of this effort is Simurgh, an AI-powered XCT reconstruction framework jointly advanced by ORNL and ZEISS that integrates computer-aided design (CAD) models, physics-based simulations, and deep learning to overcome the long-standing challenges of metal artifact correction, long scan durations, and limited flaw detectability in dense and geometrically complex components. Simurgh enables high-throughput, high-quality 3D reconstruction from sparse and fast scans, which reduces XCT acquisition times by more than an order of magnitude and simultaneously improves defect detection limits by up to fourfold compared with industry-standard approaches. This capability reduces scan costs by more than 50%, lowers labor overhead, and makes XCT characterization economically viable for routine industrial use. By enabling reliable flaw detection in minutes rather than hours, Simurgh facilitates real-time feedback loops for process parameter optimization, which was highlighted in a recent npj Computational Materials (a Nature journal) issue. In the published study, more than 100 alloy coupons were characterized within a single day. This work represents a tenfold acceleration in the development of novel AM alloys and processes compared with conventional workflows. The ZEISS collaboration has also demonstrated the scalability of Simurgh to diverse application domains, including aerospace, nuclear, automotive, and biomedical components; in these applications, ensuring structural integrity is paramount. By drastically reducing barriers to XCT adoption, this partnership has laid the foundation for digital twins and data-driven certification pipelines and directly addressed bottlenecks in qualifying new materials and designs. Together, ORNL and ZEISS have shown that Simurgh advances the state of the art in nondestructive evaluation and aligns with the broader mission of enabling Industry 4.0 manufacturing ecosystems, in which intelligent, cost-effective, rapid quality assurance is integral to accelerating innovation and ensuring safety in critical applications.

36 MATERIALS SCIENCE↗

Advancing Additive Manufacturing Through Artificial Intelligence–Powered, High-Throughput, Nondestructive Characterization and Process Optimization

This Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and ZEISS Industrial Metrology has demonstrated the transformative potential of artificial intelligence (AI)-enabled x-ray computed tomography (XCT) to accelerate the qualification and certification of additively manufactured (AM) parts. At the core of this effort is Simurgh, an AI-powered XCT reconstruction framework jointly advanced by ORNL and ZEISS that integrates computer-aided design (CAD) models, physics-based simulations, and deep learning to overcome the long-standing challenges of metal artifact correction, long scan durations, and limited flaw detectability in dense and geometrically complex components. Simurgh enables high-throughput, high-quality 3D reconstruction from sparse and fast scans, which reduces XCT acquisition times by more than an order of magnitude and simultaneously improves defect detection limits by up to fourfold compared with industry-standard approaches. This capability reduces scan costs by more than 50%, lowers labor overhead, and makes XCT characterization economically viable for routine industrial use. By enabling reliable flaw detection in minutes rather than hours, Simurgh facilitates real-time feedback loops for process parameter optimization, which was highlighted in a recent npj Computational Materials (a Nature journal) issue. In the published study, more than 100 alloy coupons were characterized within a single day. This work represents a tenfold acceleration in the development of novel AM alloys and processes compared with conventional workflows. The ZEISS collaboration has also demonstrated the scalability of Simurgh to diverse application domains, including aerospace, nuclear, automotive, and biomedical components; in these applications, ensuring structural integrity is paramount. By drastically reducing barriers to XCT adoption, this partnership has laid the foundation for digital twins and data-driven certification pipelines and directly addressed bottlenecks in qualifying new materials and designs. Together, ORNL and ZEISS have shown that Simurgh advances the state of the art in nondestructive evaluation and aligns with the broader mission of enabling Industry 4.0 manufacturing ecosystems, in which intelligent, cost-effective, rapid quality assurance is integral to accelerating innovation and ensuring safety in critical applications.

36 MATERIALS SCIENCE↗

Machine Learning for Fairness-Aware Load Shedding: A Real-Time Solution via Identifying Binding Constraints: Preprint

Timely and effective load shedding in power systems is critical for maintaining supply-demand balance and preventing cascading blackouts. To eliminate load shedding bias against specific regions in the system, optimization-based methods are uniquely positioned to help balance between economic and fairness considerations. However, the resulting optimization problem involves complex constraints, which can be time-consuming to solve and thus cannot meet the real-time requirements of load shedding. To tackle this challenge, in this paper we present an efficient machine learning algorithm to enable millisecond-level computation for the optimization-based load shedding problem. Numerical studies on both a 3-bus toy example and a realistic RTS-GMLC system have demonstrated the validity and efficiency of the proposed algorithm for delivering fairness-aware and real-time load shedding decisions.

97 MATHEMATICS AND COMPUTING↗

Digital twin framework for PIP-II linac: AI-driven multi-scale modeling from ion source to 800 MeV

The PIP-II superconducting linac at Fermilab is designed to deliver multi-megawatt proton beams for neutrino physics and other high-intensity applications. To expedite commissioning and enhance operational reliability, we have developed an EPICS-based data flow framework that seamlessly integrates digital twins (DT) with physical twins (PT). These digital twins comprise high-fidelity beam dynamics models or data-driven surrogate models connected to their physical counterparts through real-time diagnostics and advanced machine-learning algorithms.Central to this framework is Linac_Gen, an accelerated simulation tool that incorporates convolutional neural networks, random forests, and genetic algorithms to provide up to a tenfold speedup in optimizing the accelerator geometry model. An EPICS translator layer ensures interoperability by efficiently mapping lattice parameters across diverse simulation platforms.Our EPICS-based framework supports multiple operational modes—monitoring, passive learning, closed-loop control, and online learning—covering the entire machine lifecycle. By leveraging HPC resources and multi-objective optimization techniques, the digital twin enables adaptive trajectory correction, real-time fault detection, and predictive modeling of beam stability. This comprehensive approach paves the way for robust, high-intensity operation and data-driven accelerator R&D at Fermilab.

Pathak, Abhishek [Fermilab]↗

SU-F-J-105: Towards a Novel Treatment Planning Pipeline Delivering Pareto- Optimal Plans While Enabling Inter- and Intrafraction Plan Adaptation

Purpose: To develop a new IMRT treatment planning methodology suitable for the new generation of MR-linear accelerator machines. The pipeline is able to deliver Pareto-optimal plans and can be utilized for conventional treatments as well as for inter- and intrafraction plan adaptation based on real-time MR-data. Methods: A Pareto-optimal plan is generated using the automated multicriterial optimization approach Erasmus-iCycle. The resulting dose distribution is used as input to the second part of the pipeline, an iterative process which generates deliverable segments that target the latest anatomical state and gradually converges to the prescribed dose. This process continues until a certain percentage of the dose has been delivered. Under a conventional treatment, a Segment Weight Optimization (SWO) is then performed to ensure convergence to the prescribed dose. In the case of inter- and intrafraction adaptation, post-processing steps like SWO cannot be employed due to the changing anatomy. This is instead addressed by transferring the missing/excess dose to the input of the subsequent fraction. In this work, the resulting plans were delivered on a Delta4 phantom as a final Quality Assurance test. Results: A conventional static SWO IMRT plan was generated for two prostate cases. The sequencer faithfully reproduced the input dose for all volumes of interest. For the two cases the mean relative dose difference of the PTV between the ideal input and sequenced dose was 0.1% and −0.02% respectively. Both plans were delivered on a Delta4 phantom and passed the clinical Quality Assurance procedures by achieving 100% pass rate at a 3%/3mm gamma analysis. Conclusion: We have developed a new sequencing methodology capable of online plan adaptation. In this work, we extended the pipeline to support Pareto-optimal input and clinically validated that it can accurately achieve these ideal distributions, while its flexible design enables inter- and intrafraction plan adaptation. This research is financially supported by Elekta AB, Stockholm, Sweden.

60 APPLIED LIFE SCIENCES↗

Innovative Open-Source Energy Software to Drive and Enable Energy Savings and Process Optimization

U.S. Department of Energy (DOE) energy system-analysis software tools have been used for years helping to identify over 170 TBTU/year of energy savings through the Save Energy Now “Energy Savings Assessment” and Better Plants “In-Plant Training” Programs. While most of the fundamental energy savings opportunities do not evolve much with time, computer technology never stops, leaving most of these software tools inoperable on modern computers.This has sparked an initiative to revitalize the DOE energy system analysis tools into a new, open-source, integrated software tool suite - MEASUR. Designed for industrial energy coordinators, engineers, utility-program implementers and individuals who are interested in improving system efficiency, MEASUR can aide users in identifying, assessing, and quantifying energy savings opportunities within pump, fan, process heating, steam, motors, and compressed air systems. In addition, the software contains over 40 equipment and property calculators for simple energy-related calculations and analyses.This paper will present a results comparison for each energy-consuming system using data from past DOE-sponsored energy assessments (to establish the accuracy of the savings algorithms) to validate and verify the accuracy of the energy and cost savings calculated by MEASUR. Additionally, an example highlighting 3rd-party development of the tool’s opensource framework into real-time sensor systems and efficiency-optimization will be presented.

Armstrong, Kristina↗

Optimizing Travel Time to Outpatient Interventional Radiology Procedures in a Multi-Site Hospital System Using a Google Maps Application

The purpose of this study is to determine whether a custom Google Maps application can optimize site selection when scheduling outpatient interventional radiology (IR) procedures within a multi-site hospital system. The Google Maps for Business Application Programming Interface (API) was used to develop an internal web application that uses real-time traffic data to determine estimated travel time (ETT; minutes) and estimated travel distance (ETD; miles) from a patient’s home to each a nearby IR facility in our hospital system. Hypothetical patient home addresses based on the 33 cities comprising our institution’s catchment area were used to determine the optimal IR site for hypothetical patients traveling from each city based on real-time traffic conditions. For 10/33 (30%) cities, there was discordance between the optimal IR site based on ETT and the optimal IR site based on ETD at non-rush hour time or rush hour time. By choosing to travel to an IR site based on ETT rather than ETD, patients from discordant cities were predicted to save an average of 7.29 min during non-rush hour (p = 0.03), and 28.80 min during rush hour (p < 0.001). Using a custom Google Maps application to schedule outpatients for IR procedures can effectively reduce patient travel time when more than one location providing IR procedures is available within the same hospital system.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Optimized anion exchange column isolation of zirconium-89 (89Zr) from yttrium cyclotron target: Method development and implementation on an automated fluidic platform

Zirconium-89 (89Zr), produced by the (p,n) reaction from naturally monoisotopic yttrium (natY), is a promising positron emitting isotope for immunoPET imaging. Its long half-life of 78.4 h is sufficient for evaluating slow physiological processes. A prototype automated fluidic system, coupled to on-line and in-line detectors, has been constructed to facilitate development of new 89Zr purification methodologies. The highly reproducible reagent delivery platform and near-real time monitoring of column effluents allows for efficient method optimization. The separation of Zr from dissolved Y metal targets was evaluated using several anion exchange resins. Each resin was evaluated against its ability to quantitatively capture Zr from a load solution that is high in dissolved Y. The most appropriate anion exchange resin for this application was identified, and the separation method was optimized. The method is capable of a high Y decontamination factor (>105) and has been shown to separate Fe, an abundant contaminant in Y foils, from the 89Zr elution fraction. Finally, the performance of the method was evaluated using cyclotron bombarded Y foil targets. The separation method was shown to achieve >95% recovery of the 89Zr present in the foils. The 89Zr eluent, however, was in a chemical matrix not immediately conducive to labeling onto proteins. The main intent of this study was to develop a tandem column 89Zr purification process, wherein the anion exchange column method described here is the first separation in a dual-column purification process.

Zirconium-89, ImmunoPET, medical isotope, automati↗

Enhancements on the Convex Programming Based Powered Descent Guidance Algorithm for Mars Landing

In this paper, we present enhancements on the powered descent guidance algorithm developed for Mars pinpoint landing. The guidance algorithm solves the powered descent minimum fuel trajectory optimization problem via a direct numerical method. Our main contribution is to formulate the trajectory optimization problem, which has nonconvex control constraints, as a finite dimensional convex optimization problem, specifically as a finite dimensional second order cone programming (SOCP) problem. SOCP is a subclass of convex programming, and there are efficient SOCP solvers with deterministic convergence properties. Hence, the resulting guidance algorithm can potentially be implemented onboard a spacecraft for real-time applications. Particularly, this paper discusses the algorithmic improvements obtained by: (i) Using an efficient approach to choose the optimal time-of-flight; (ii) Using a computationally inexpensive way to detect the feasibility/ infeasibility of the problem due to the thrust-to-weight constraint; (iii) Incorporating the rotation rate of the planet into the problem formulation; (iv) Developing additional constraints on the position and velocity to guarantee no-subsurface flight between the time samples of the temporal discretization; (v) Developing a fuel-limited targeting algorithm; (vi) Initial result on developing an onboard table lookup method to obtain almost fuel optimal solutions in real-time.

Guidance↗

Two-Stage Reinforcement Learning Policy Search for Grid-Interactive Building Control

This paper develops an intelligent grid-interactive building controller, which optimizes building operation during both normal hours and demand response (DR) events. To avoid costly on-demand computation and to adapt to non-linear building models, the controller utilizes reinforcement learning (RL) and makes real-time decisions based on a near-optimal control policy. Learning such a policy typically amounts to solving a hard non-convex optimization problem. We propose to address this problem with a novel global-local policy search method. In the first stage, an RL algorithm based on zero-order gradient estimation is leveraged to search for the optimal policy globally, due to its scalability and the potential to escape some poor performing local optima. The obtained policy is then fine-tuned locally to bring the first-stage solution closer to that of the original unsmoothed problem. Experiments on a simulated five-zone commercial building demonstrate the advantages of the proposed method over existing learning approaches. They also show that the learned control policy outperforms a pragmatic linear model predictive controller (MPC) and approaches the performance of an oracle MPC in testing scenarios. Using a state-of-the-art advanced computing system, we demonstrate that the controller can be learned and deployed within hours of training.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗