Search NASA⌕ Search

SEARCH · Search NASA

Results for “online distributed optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Microgrid energy scheduling under uncertain extreme weather: Adaptation from parallelized reinforcement learning agents

Microgrids are useful solutions for integrating renewable energy resources and providing seamless green electricity to minimize carbon footprint. In recent years, extreme weather events happened often worldwide and caused significant economic and societal losses. Such events bring uncertainties to the microgrid energy scheduling problems and increase the challenges of microgrid operation. Traditional optimization approaches suffer from the inaccuracy of the uncertain microgrid model and the unseen events. Existing reinforcement learning (RL) - based approaches are also hampered by the limited generalization and the increasing computational burden when stochastic formulations are required to accommodate the uncertainties. This paper proposes a new parallelized reinforcement learning (PRL) method based on the probabilistic events to handle the microgrid energy uncertainties. Specifically, several local learning agents are employed to interact with pertinent microgrid environments in a distributed manner and report outcomes to the global agent, which will optimize microgrid energy resources online during extreme events. The stochastic microgrid energy optimization problem is reformulated to include all possible scenarios with probabilities. The advantage estimate functions of learning agents are designed with a backward sweep to transfer the outcomes to the value function updating process. Two simulation studies, stochastic optimization and online testing, are performed to compare with several existing RL approaches. Results substantiate that the proposed PRL method can achieve up to 20% optimization performance improvement with 4 and 28 times less computation cost than Q-learning with experience replay and multi-agent Q-learning approaches, respectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Combining On-Line Characterization Tools with Modern Software Environments for Optimal Operation of Polymerization Processes

Here, this paper discusses the initial steps towards the formulation and implementation of a generic and flexible model centric framework for integrated simulation, estimation, optimization and feedback control of polymerization processes. For the first time it combines the powerful capabilities of the automatic continuous on-line monitoring of polymerization system (ACOMP), with a modern simulation, estimation and optimization software environment towards an integrated scheme for the optimal operation of polymeric processes. An initial validation of the framework was performed for modelling and optimization using literature data, illustrating the flexibility of the method to apply under different systems and conditions. Subsequently, off-line capabilities of the system were fully tested experimentally for model validations, parameter estimation and process optimization using ACOMP data. Experimental results are provided for free radical solution polymerization of methyl methacrylate.

42 ENGINEERING↗

A physics-based model for industrial steam-methane reformer optimization with non-uniform temperature field

In an industrial hydrogen production facility, steam-methane reforming reactions take place inside hundreds of catalyst-filled tubes placed in a large scale, high temperature furnace. Process efficiency depends strongly on the wall temperature distribution of the ensemble of reformer tubes; a narrower distribution has a process intensification effect, by providing similar processing experience to every feedstock molecule. Such process intensification efforts require a furnace model that can predict the temperature distribution as a function of operating conditions. Currently available furnace modeling solutions are either computationally intensive, making them unsuitable for (online) optimization calculations, or empirical, having limited accuracy when wide changes in operating conditions are required. Here in this work, a physics-based furnace model is presented that overcomes these limitations. Empirical perturbations in a Hottel zone radiation model are proposed to capture the spatially non-symmetrical temperature distribution. The low computational time makes the model suitable for operational intensification based on reduction of temperature distribution non-uniformity.

42 ENGINEERING↗

Reinforcement Learning for Load-balanced Parallel Particle Tracing

We explore an online reinforcement learning (RL) paradigm to dynamically optimize parallel particle tracing performance in distributed-memory systems. Our method combines three novel components: (1) a work donation algorithm, (2) a high-order workload estimation model, and (3) a communication cost model. First, we design an RL-based work donation algorithm. Our algorithm monitors workloads of processes and creates RL agents to donate data blocks and particles from high-workload processes to low-workload processes to minimize program execution time. The agents learn the donation strategy on the fly based on reward and cost functions designed to consider processes' workload changes and data transfer costs of donation actions. Second, we propose a workload estimation model, helping RL agents estimate the workload distribution of processes in future computations. Third, we design a communication cost model that considers both block and particle data exchange costs, helping RL agents make effective decisions with minimized communication costs. We demonstrate that our algorithm adapts to different flow behaviors in large-scale fluid dynamics, ocean, and weather simulation data. Our algorithm improves parallel particle tracing performance in terms of parallel efficiency, load balance, and costs of I/O and communication for evaluations with up to 16,384 processors.

Distributed and parallel particle tracing↗

Restoring Critical Loads In Resilient Distribution Systems using A Curriculum Learned Controller

In this paper, we propose a curriculum learned reinforcement learning (RL) controller to facilitate distribution system critical load restoration (CLR), leveraging RL's fast online response and its outstanding optimal sequential control capability. Like many grid control problems, CLR is complicated due to the large control action space and renewable uncertainty in a heavily constrained non-linear environment with strong intertemporal dependency. The nature of the problem oftentimes causes the RL policy to converge to a poor-performing local optimum if learned directly. To overcome this, we design a two-stage curriculum in which the RL agent will learn generation control and load restoration decision under different scenarios progressively. Via curriculum learning, the trained RL controller is expected to achieve a better control performance, with critical loads restored as rapidly and reliably as possible. Using the IEEE 13-bus test system, we illustrate the performance of the RL controller trained by the proposed curriculum-based method.

curriculum learning↗

Primal-Dual Differentiable Programming for Distribution System Critical Load Restoration: Preprint

Swift and reliable critical load restoration (CLR) can help make a distribution system resilient towards extreme events. To optimally achieve that, alongside practical concerns such as limiting online computational burden, some studies leverage model-free reinforcement learning (RL) to train control policies. Despite the advantages provided by RL algorithms, these approaches suffer from two issues: 1) the lack of a proper mechanism for constraint enforcement, and 2) poor sample efficiency. Therefore, in this paper, a primal-dual differentiable programming (PDDP) method is developed for guiding the training leading to a constraint-satisfying policy. Additionally, the model-based nature of the proposed method aims at improving sample efficiency. The experiment on a CLR problem demonstrates that PDDP can effectively train a control policy that both achieves desirable performance and satisfies required constraints.

differentiable programming↗

Restoring Critical Loads in Resilient Distribution Systems Using a Curriculum Learned Controller: Preprint

In this paper, we propose a curriculum learned reinforcement learning (RL) controller to facilitate distribution system critical load restoration (CLR), leveraging RL's fast online response and its outstanding optimal sequential control capability. Like many grid control problems, CLR is complicated due to the large control action space and renewable uncertainty in a heavily constrained non-linear environment with strong intertemporal dependency. The nature of the problem oftentimes causes the RL policy to converge to a poor-performing local optimum if learned directly. To overcome this, we design a two-stage curriculum in which the RL agent will learn generation control and load restoration decision under different scenarios progressively. Via curriculum learning, the trained RL controller is expected to achieve a better control performance, with critical loads restored as rapidly and reliably as possible. Using the IEEE 13-bus test system, we illustrate the performance of the RL controller trained by the proposed curriculum-based method.

61 RADIATION PROTECTION AND DOSIMETRY↗

Measurement of the D + -meson production cross section at low transverse momentum in p p ¯ collisions at s = 1.96 TeV

We report on a measurement of the D + -meson production cross section as a function of transverse momentum (p T ) in proton-antiproton (p$\bar{p}$) collisions at 1.96 TeV center-of-mass energy, using the full data set collected by the Collider Detector at Fermilab in Tevatron Run II and corresponding to 10 fb -1 of integrated luminosity. We use D + →K - π + π + decays fully reconstructed in the central rapidity region |y|<1 with transverse momentum down to 1.5 GeV/c, a range previously unexplored in p$\bar{p}$ collisions. Inelastic p$\bar{p}$ scattering events are selected online using minimally biasing requirements followed by an optimized offline selection. The K - π + π + mass distribution is used to identify the D + signal, and the D + transverse impact-parameter distribution is used to separate prompt production, occurring directly in the hard-scattering process, from secondary production from b-hadron decays. We obtain a prompt D + signal of 2950 candidates corresponding to a total cross section σ(D + ,1.5 T <14.5 GeV/c,|y|<1)=71.9±6.8(stat)±9.3(syst) μb. While the measured cross sections are consistent with theoretical estimates in each p T bin, the shape of the observed p T spectrum is softer than the expectation from quantum chromodynamics. The results are unique in p$\bar{p}$ collisions and can improve the shape and uncertainties of future predictions.

(K-↗

Attention Enabled Multi-Agent DRL for Decentralized Volt-VAR Control of Active Distribution System Using PV Inverters and SVCs

This paper proposes attention enabled multi-agent deep reinforcement learning (MADRL) framework for active distribution network decentralized Volt-VAR control. Using the unsupervised clustering, the whole distribution system can be decomposed into several sub-networks according to the voltage and reactive power sensitivity relationships. Then, the distributed control problem of each sub-network is modeled as Markov games and solved by the improved MADRL algorithm, where each sub-network is modeled as an adaptive agent. An attention mechanism is developed to help each agent focus on specific information that is mostly related to the reward. All agents are centrally trained offline to learn the optimal coordinated Volt-VAR control strategy and executed in a decentralized manner to make online decisions with only local information. Compared with other distributed control approaches, the proposed method can effectively deal with uncertainties, achieve fast decision makings, and significantly reduce the communication requirements. Comparison results with model-based and other data-driven methods on IEEE 33-bus and 123-bus systems demonstrate the benefits of the proposed approach.

distribution network↗

Online Primal-Dual Methods With Measurement Feedback for Time-Varying Convex Optimization

This paper addresses the design and analysis of feedback-based online algorithms to control systems or networked systems based on performance objectives and engineering constraints that may evolve over time. The emerging timevarying convex optimization formalism is leveraged to model optimal operational trajectories of the systems, as well as explicit local and network-level operational constraints. Departing from existing batch and feed-forward optimization approaches, the design of the algorithms capitalizes on an online implementation of primal-dual projected-gradient methods; the gradient steps are, however, suitably modified to accommodate feedback from the system in the form of measurements - hence, the term 'online optimization with feedback.' By virtue of this approach, the resultant algorithms can cope with model mismatches in the algebraic representation of the system states and outputs, they avoid pervasive measurements of exogenous inputs, and they naturally lend themselves to a distributed implementation. Under suitable assumptions, analytical convergence claims are established in terms of dynamic regret. Furthermore, when the synthesis of the feedback-based online algorithms is based on a regularized Lagrangian function, Q-linear convergence to solutions of the time-varying optimization problem is shown.

97 MATHEMATICS AND COMPUTING↗

Program for Online Network Inversion

PONI determines the source location of a contamination incident in a water distribution network. PONI uses large scale optimization methods to predict likely source locations by reconciling the differences between observations and numerical predictions of possible contamination incidents.

Bloemen Waanders, Bart van↗

Distribution Feeder-Scale Fast Frequency Response via Optimal Coordination of Net-load Resources Part I: Solution Design

This work is the first of a two-part series that develops and experimentally demonstrates a first-of-its-kind hierarchical control solution for optimally dispatching thousands of deferrable loads and distributed energy resources (DERs) across a distribution feeder to provide fast frequency response (FFR) within 500 ms to the bulk power system. This approach rapidly coordinates resources online after a frequency event occurs, allowing fast-changing, behind-the-meter (BTM) resources to be incorporated and aggregate FFR power set points to be achieved more quickly and accurately than existing approaches. We also present a solution for determining the optimal amount of headroom to operate solar inverters with to minimize opportunity cost while ensuring the FFR response viability of a building with the inverter and deferrable loads. In Part I, we develop practical algorithms for fast, cost-based optimal dispatch at multiple aggregation scales (single building, multiple buildings, and full distribution feeder), establish their optimality, and demonstrate via simulation that they are faster than state-of-the-art, coordinated frequency response approaches. In Part II, the entire platform is implemented and experimentally verified using a unique power hardware-in-the-loop demonstration, including more than 100 powered loads and DERs connected to a real-world distribution network model and over 10,000 net-load resources dispatched.

24 POWER TRANSMISSION AND DISTRIBUTION↗

TU-C-BRE-11: 3D EPID-Based in Vivo Dosimetry: A Major Step Forward Towards Optimal Quality and Safety in Radiation Oncology Practice

Purpose: To develop a 3D in vivo dosimetry method that is able to substitute pre-treatment verification in an efficient way, and to terminate treatment delivery if the online measured 3D dose distribution deviates too much from the predicted dose distribution. Methods: A back-projection algorithm has been further developed and implemented to enable automatic 3D in vivo dose verification of IMRT/VMAT treatments using a-Si EPIDs. New software tools were clinically introduced to allow automated image acquisition, to periodically inspect the record-and-verify database, and to automatically run the EPID dosimetry software. The comparison of the EPID-reconstructed and planned dose distribution is done offline to raise automatically alerts and to schedule actions when deviations are detected. Furthermore, a software package for online dose reconstruction was also developed. The RMS of the difference between the cumulative planned and reconstructed 3D dose distributions was used for triggering a halt of a linac. Results: The implementation of fully automated 3D EPID-based in vivo dosimetry was able to replace pre-treatment verification for more than 90% of the patient treatments. The process has been fully automated and integrated in our clinical workflow where over 3,500 IMRT/VMAT treatments are verified each year. By optimizing the dose reconstruction algorithm and the I/O performance, the delivered 3D dose distribution is verified in less than 200 ms per portal image, which includes the comparison between the reconstructed and planned dose distribution. In this way it was possible to generate a trigger that can stop the irradiation at less than 20 cGy after introducing large delivery errors. Conclusion: The automatic offline solution facilitated the large scale clinical implementation of 3D EPID-based in vivo dose verification of IMRT/VMAT treatments; the online approach has been successfully tested for various severe delivery errors.

60 APPLIED LIFE SCIENCES↗

Distribution-Agnostic Stochastic Optimal Power Flow for Distribution Grids

This paper outlines a data-driven, distributionally robust approach to solve chance-constrained AC optimal power flow problems in distribution networks. Uncertain forecasts for loads and power generated by photovoltaic (PV) systems are considered, with the goal of minimizing PV curtailment while meeting power flow and voltage regulation constraints. A data- driven approach is utilized to develop a distributionally robust conservative convex approximation of the chance-constraints; particularly, the mean and covariance matrix of the forecast errors are updated online, and leveraged to enforce voltage regulation with predetermined probability via Chebyshev-based bounds. By combining an accurate linear approximation of the AC power flow equations with the distributionally robust chance constraint reformulation, the resulting optimization problem becomes convex and computationally tractable.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Online Robot Dead Reckoning Localization Using Maximum Relative Entropy Optimization With Model Constraints

The principle of Maximum relative Entropy optimization was analyzed for dead reckoning localization of a rigid body when observation data of two attached accelerometers was collected. Model constraints were derived from the relationships between the sensors. The experiment's results confirmed that accelerometers each axis' noise can be successfully filtered utilizing dependency between channels and the dependency between time series data. Dependency between channels was used for a priori calculation, and a posteriori distribution was derived utilizing dependency between time series data. There was revisited data of autocalibration experiment by removing the initial assumption that instantaneous rotation axis of a rigid body was known. Performance results confirmed that such an approach could be used for online dead reckoning localization.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A Hybrid Optimization and Deep Learning Algorithm for Cyber-Resilient DER Control

With the proliferation of distributed energy resources (DERs) in the distribution grid, it is a challenge to effectively control a large number of DERs resilient to the communication and security disruptions, as well as to provide the online grid services, such as voltage regulation and virtual power plant (VPP) dispatch. To this end, a hybrid feedback-based optimization algorithm along with deep learning forecasting technique is proposed to specifically address the cyber-related issues. The online decentralized feedback-based DER optimization control requires timely, accurate voltage measurement from the grid. However, in practice such information may not be received by the control center or even be corrupted. Therefore, the long short-term memory (LSTM) deep learning algorithm is employed to forecast delayed/missed/attacked messages with high accuracy. The IEEE 37-node feeder with high penetration of PV systems is used to validate the efficiency of the proposed hybrid algorithm. The results show that 1) the LSTM-forecasted lost voltage can effectively improve the performance of the DER control algorithm in the practical cyber-physical architecture; and 2) the LSTM forecasting strategy outperforms other strategies of using previous message and skipping dual parameter update.

cyber-resilient algorithm↗

Data-Driven Robust RVFLNs Modeling of a Blast Furnace Iron-Making Process Using Cauchy Distribution Weighted M-Estimation

Optimal operation of a practical blast furnace (BF) ironmaking process depends largely on a good measurement of molten iron quality (MIQ) indices. However, measuring the MIQ online is not feasible using the available techniques. In this paper, a novel data-driven robust modeling is proposed for online estimation of MIQ using improved random vector functional-link networks (RVFLNs). Since the output weights of traditional RVFLNs are obtained by the least squares approach, a robustness problem may occur when the training dataset is contaminated with outliers. This affects the modeling accuracy of RVFLNs. To solve this problem, a Cauchy distribution weighted M-estimation based robust RFVLNs is proposed. Since the weights of different outlier data are properly determined by the Cauchy distribution, their corresponding contribution on modeling can be properly distinguished. Thus robust and better modeling results can be achieved. Moreover, given that the BF is a complex nonlinear system with numerous coupling variables, the data-driven canonical correlation analysis is employed to identify the most influential components from multitudinous factors that affect the MIQ indices to reduce the model dimension. Finally, experiments using industrial data and comparative studies have demonstrated that the obtained model produces a better modeling and estimating accuracy and stronger robustness than other modeling methods.

42 ENGINEERING↗

Data Driven Optimization Framework for Volt-age Regulation in Distribution Systems

This letter proposes a data-driven optimization framework for voltage regulation problems to address the challenge of model inaccuracy and parameter varying. To achieve online voltage optimization, the recursive kernel regression and interior point methods are integrated. The IEEE 123-Bus system and EPRI Ckt5 feeder are selected to validate the effectiveness of the proposed data-driven optimization framework. The proposed method is also compared with a linear function based method.

Hong, Tianqi↗