SEARCH · Search NASA
Results for “Decentralized optimization”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems
Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system throughput. Traditional heuristic-based scheduling algorithms increasingly struggle and lack the efficiency needed to meet the demands and address the complexity and scale of modern HPC systems. Consequently, recent research efforts have focused on leveraging advancements in Artificial Intelligence (AI) and Deep Learning (DL), particularly Reinforcement Learning (RL), to develop more adaptable and intelligent scheduling strategies. Previous RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks (GNNs) with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, with few methods being evaluated using large-scale, multi-million-job trace datasets representative of real-world HPC workloads. Moreover, existing RL schedulers face scalability issues due to centralized policy updates, which hinder training efficiency and performance when applied to large datasets. This study introduces a novel RL-based scheduler utilizing Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. Experimental validation using a large real-world dataset containing over 11.5 million job traces collected from petascale HPC systems over six years assesses the influence of dataset scale on training effectiveness and compares DD-PPO performance to traditional and advanced scheduling approaches. The experimental results demonstrate improved scheduling performance in comparison to both heuristic-based schedulers and existing RL-based scheduling algorithms.
Evaluating the Impact of Managed EV Charging for Reliable Operation of Bulk Power Systems with High Non-Dispatchable Generation
The growth of electric vehicles (EVs) and variable-generation (VG) sources introduces new challenges for power-system operations. This study introduces a modeling framework and evaluates five EV charging strategies under projected 2040 grid conditions in the Evergy service territory with high non-dispatchable generation. Using realistic EV behavior and generation models, their impacts on system peak demand, ramp rate, and reserve capacity are evaluated. Results show that only the peak-avoidance strategy effectively reduces system peak demand, while decentralized strategies-particularly cost based dynamic charging-can exacerbate peaks due to synchronized user behavior. However, ramp-rate minimization strategy significantly reduce the stress on dispatchable generation achieving the lowest maximum absolute ramp rate (MARR) (56.81 MW) and lowest reserve requirement (2.39 GW). In contrast, unmanaged and TOU random strategies increase the stress on dispatchable generation sources with increased MARR and reserve requirements. These findings highlight the importance of coordinated, system-aware managed charging strategies to ensure reliable and affordable grid operation in the presence of EVs and VG sources.
Encrypted Decentralized Multi-Agent Optimization for Privacy Preservation in Cyber-Physical Systems
Not provided.
Optimal design and integration of decentralized electrochemical energy storage with renewables and fossil plants
Increasing renewable energy requires improving the electricity grid flexibility. Existing measures include power plant cycling and grid-level energy storage, but they incur high operational and investment costs. Using a systems modeling and optimization framework, we study the integration of electrochemical energy storage with individual power plants at various renewable penetration levels. Furthermore, our techno-economic analysis includes both Li-ion and NaS batteries to encompass different technology maturity levels. A California case-study indicates localized integration to be cost-effective for greater grid flexibility. Li-ion batteries can mitigate the residual demand fluctuations of small to medium-sized plants, while NaS batteries would be best-suited for larger storage with higher renewable penetration. Overall, the battery-enabled renewable integration could reduce the unmet grid demand by 75%, the renewable curtailment by 58%, and the CO 2 emission intensity by 16% while including the life cycle emissions of the battery and the renewable farm. Our scenario-based analysis also indicates that rather than replacing all fossil power plants, it is more economical to combine batteries and renewables with individual fossil plants to achieve a clean energy grid.
Decentralized Carrier Phase Shifting for Optimal Harmonic Minimization in Asymmetric Parallel-Connected Inverters
This paper presents a carrier phase shifting technique for minimizing the aggregate harmonics in networks of asymmetric parallel-connected inverters for distributed power generation system applications. The proposed technique is: 1) implemented in a decentralized manner, relying only on local voltage and current measurements, and 2) optimal in the sense that it minimizes a cost function representing the carrier-frequency current harmonics. The analysis indicates that the proposed optimal carrier phase shifting technique can enable order-of-magnitude reductions in harmonic power, and also universal improvements compared to symmetric carrier interleaving for asymmetric inverter networks. Moreover, compared to existing methods that require either centralized communication or information exchange between inverters to coordinate carriers, the proposed technique is completely decentralized, which provides important practical benefits for implementation, including improved robustness and reduced cost. The technique is experimentally validated on a network of three single-phase 2-kW inverters and demonstrates a 36.5% reduction in the weighted total harmonic distortion factor of the aggregate inverter current, and the ability to converge to the optimal carrier phase spacing dynamically in less than one line frequency cycle (16.7 ms) in steady state and transient operating conditions.
Decentralized regulation of dynamic systems
A special class of decentralized control problem is discussed in which the objectives of the control agents are to steer the state of the system to desired levels. Each agent is concerned about certain aspects of the state of the entire system. The state and control equations are given for linear time-invariant systems. Stability and coordination, and the optimization of decentralized control are analyzed, and the information structure design is presented.
Regulation in the Air: Price and Frequency Cap
Despite deregulation on the air-transportation markets, many connections are still operated by a single operator. Regulation is thus a central issue in this industry. There is however a great concern for the (possibly negative) consequences of price regulation on the quality of services. We argue that both aspects should be considered jointly and propose a mechanism that allow to decentralize the optimal structure of services in this industry. The regulatory procedure is robust to the introduction of heterogeneity in the travellers' valuation of the connections' frequency.
Algorithms for output feedback, multiple-model, and decentralized control problems
The optimal stochastic output feedback, multiple-model, and decentralized control problems with dynamic compensation are formulated and discussed. Algorithms for each problem are presented, and their relationship to a basic output feedback algorithm is discussed. An aircraft control design problem is posed as a combined decentralized, multiple-model, output feedback problem. A control design is obtained using the combined algorithm. An analysis of the design is presented.
Decentralized Schemes with Overlap for Solving Graph-Structured Optimization Problems
We present a new algorithmic paradigm for the decentralized solution of graph-structured optimization problems that arise in the estimation and control of network systems. A key and novel design concept of the proposed approach is that it uses overlapping subdomains to promote and accelerate convergence. We show that the algorithm converges if the size of the overlap is sufficiently large and that the convergence rate improves exponentially with the size of the overlap. The proposed approach provides a bridge between fully decentralized and centralized architectures and is flexible in that it enables the implementation of asynchronous schemes, handling of constraints, and balancing of computing, communication, and data privacy needs. The proposed scheme is tested in an estimation problem for a 9241-node power network and we show that it outperforms the alternating direction method of multipliers.
Multi-Agent Control of Lane-Switching Automated Vehicles for Energy Efficiency
The proliferation of automatic control systems and their connectivity accents the performance of their interactions. In particular, connected and automated vehicles could become high-impact examples thanks to their energy use and productivity effects. Ideally, a controller might collectively optimize all agents' control moves for a given objective. However, limits on computational complexity, incomplete knowledge of the central planner, and risks of a single point of failure make distributed control attractive as well. This paper proposes a collaborative heuristic to approach the performance of centralized control with decentralized-like computational effort. The related centralized controller is also described in detail and evaluated as a baseline. A collaboration-intensive obstacle avoidance scenario involving electric vehicles is simulated to demonstrate benefits over fully decentralized control. While centralized optimization performed best with an 8.6 % energy reduction, the collaborative decentralized scheme reached a favorable computation-performance tradeoff with a 6.7 % energy reduction.
Randomized Federated Learning Methods for Nonsmooth, Nonconvex, and Hierarchical Optimization (Final Technical Report)
This final technical report summarizes the outcomes of a DOE-funded project on federated scientific machine learning (FL) under nonsmooth, nonconvex, and hierarchical optimization settings. The project develops new mathematical models, algorithms, and theoretical guarantees for decentralized stochastic, bilevel, and minimax optimization problems arising in DOE mission-relevant applications. A unified framework of randomized and zeroth-order federated optimization methods is introduced, providing provable convergence, communication efficiency, and sample-complexity guarantees. The report documents algorithmic design, theoretical analysis, and empirical validation of the proposed federated learning methods. The project also contributes to workforce development through graduate training and dissemination of results via publications and seminars.
Transition-Independent Decentralized Markov Decision Processes
There has been substantial progress with formal models for sequential decision making by individual agents using the Markov decision process (MDP). However, similar treatment of multi-agent systems is lacking. A recent complexity result, showing that solving decentralized MDPs is NEXP-hard, provides a partial explanation. To overcome this complexity barrier, we identify a general class of transition-independent decentralized MDPs that is widely applicable. The class consists of independent collaborating agents that are tied up by a global reward function that depends on both of their histories. We present a novel algorithm for solving this class of problems and examine its properties. The result is the first effective technique to solve optimally a class of decentralized MDPs. This lays the foundation for further work in this area on both exact and approximate solutions.
Distributionally Robust Decentralized Volt-Var Control With Network Reconfiguration
Here, this paper presents a decentralized volt-var optimization (VVO) and network reconfiguration strategy to address the challenges arising from the growing integration of distributed energy resources, particularly photovoltaic (PV) generation units, in active distribution networks. To reconcile control measures with different time resolutions and empower local control centers to handle intermittency locally, the proposed approach leverages a two-stage distributionally robust optimization; decisions on slow-responding control measures and set points that link neighboring subnetworks are made in advance while considering all plausible distributions of uncertain PV outputs. We present a decomposition algorithm with an acceleration scheme for solving the proposed model. Numerical experiments on the IEEE 123 bus distribution system are given to demonstrate its outstanding out-of-sample performance and computational efficiency, which suggests that the proposed method can effectively localize uncertainty via risk-informed proactive timely decisions.
A Hybrid Optimization and Deep Learning Algorithm for Cyber-Resilient DER Control
With the proliferation of distributed energy resources (DERs) in the distribution grid, it is a challenge to effectively control a large number of DERs resilient to the communication and security disruptions, as well as to provide the online grid services, such as voltage regulation and virtual power plant (VPP) dispatch. To this end, a hybrid feedback-based optimization algorithm along with deep learning forecasting technique is proposed to specifically address the cyber-related issues. The online decentralized feedback-based DER optimization control requires timely, accurate voltage measurement from the grid. However, in practice such information may not be received by the control center or even be corrupted. Therefore, the long short-term memory (LSTM) deep learning algorithm is employed to forecast delayed/missed/attacked messages with high accuracy. The IEEE 37-node feeder with high penetration of PV systems is used to validate the efficiency of the proposed hybrid algorithm. The results show that 1) the LSTM-forecasted lost voltage can effectively improve the performance of the DER control algorithm in the practical cyber-physical architecture; and 2) the LSTM forecasting strategy outperforms other strategies of using previous message and skipping dual parameter update.
Federated Deep Reinforcement Learning for Decentralized VVO of BTM DERs
The future of grid control requires a hybrid approach combining centralized and decentralized methods to fully utilize the potential of smart edge devices with artificial intelligence (AI) capabilities. This paper aims to develop and evaluate a federated deep reinforcement learning (FDRL) framework for decentralized adaptive volt-var optimization (VVO) of behind-the-meter (BTM) distributed energy resources (DERs). First, this paper models a single deep reinforcement learning (DRL) agent using the Markov Decision Process (MDP) framework for decentralized adaptive VVO of BTM DERs. Two DRL algorithms, soft actor-critic (SAC) and twin-delayed deep deterministic policy gradient (TD3), are compared for their effectiveness in optimizing VVO. Results show that TD3 outperforms SAC, achieving a 71.3% improvement in mean reward. Finally, the DRL agent is deployed within the FDRL framework, using the Flower platform, to enhance learning, provide adaptive control, and ensure data privacy for BTM DERs.
A Decentralized Primal-Dual Framework for Non-Convex Smooth Consensus Optimization
Not provided.