Search NASA⌕ Search

DOE OSTI · 1817364

Accelerating Optimization and Reinforcement Learning with Quasi Stochastic Approximation

Abstract

The paper sets out to obtain precise convergence rates for quasi-stochastic approximation (QSA), with applications to optimization and reinforcement learning.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Chen, Shuhang, Devraj, Adithya, Bernstein, Andrey, Meyn, Sean. 2021-07-28. Accelerating Optimization and Reinforcement Learning with Quasi Stochastic Approximation. https://doi.org/10.23919/acc50511.2021.9482825. https://searchnasa.com/record/osti%3A1817364/accelerating-optimization-and-reinforcement-learning-with-quasi-stochastic-approximation

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Search NASA by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related reports

A Hierarchical OPF Algorithm with Improved Gradient Evaluation in Three-Phase Networks

Linear approximation commonly used in solving alternating-current optimal power flow (AC-OPF) simplifies the system models but incurs accumulated voltage errors in large power networks. Such errors will make the primal-dual type gradient algorithms converge to solutions with voltage violation. In this paper, we improve a recent hierarchical OPF algorithm that rested on primal-dual gradients evaluated with a linearized distribution power flow model. Specifically, we propose a more accurate gradient evaluation method based on an unbalanced three-phase nonlinear distribution power flow model to mitigate the errors arising from linearization. The resultant gradients feature a blocked structure that enables our development of an improved hierarchical primal-dual algorithm to solve the OPF problem. Numerical results on the IEEE 123-bus test feeder and a 4,518-node test feeder show that the proposed method can enhance voltage safety at comparable computational efficiency with the linearized algorithm.

approximation algorithms↗

Towards Robustness Guarantees for Feedback-Based Optimization

Feedback-based online optimization algorithms have gained traction in recent years because of their simple implementation, their ability to reject disturbances in real time, and their increased robustness to model mismatch. While the robustness properties have been observed both in simulation and experimental results, the theoretical analysis in the literature is mostly limited to nominal conditions. In this work, we propose a framework to systematically assess the robust stability of feedback-based online optimization algorithms. We leverage tools from monotone operator theory, variational inequalities and classical robust control to obtain tractable numerical tests that guarantee robust convergence properties of online algorithms in feedback with a physical system, even in the presence of disturbances and model uncertainty. The results are illustrated via an academic example and a case study of a power distribution system.

approximation algorithms↗