Search NASA⌕ Search

SEARCH · Search NASA

Results for “Multi-agent control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

101 records · Page 6

Buyers Collusion in Incentivized Forwarding Networks: A Multi-Agent Reinforcement Learning Study

We present the issue of monetarily incentivized forwarding in a multi-hop mesh network architecture from an economic perspective. It is anticipated that credit-incentivized forwarding and relaying will be a simple method of exchanging transmission power and spectrum for connectivity. However, gateways and forwarding nodes, like any other free market, may create an oligopolistic market for the users they serve. In this study, a coalition scheme between buyers aims to address price control by gateways or nodes closer to gateways. In a Stackelberg competition game, buyer agents (users) and sellers (gateways) make decisions using reinforcement learning (RL), with decentralized Deep Q-Networks to buy and sell forwarding resources. We allow communication links between the buyers with a limited messaging space, without defining a collusion mechanism. The idea is to demonstrate that through messaging, and RL tacit collusion can emerge between agents in a decentralized setup. The multi-agent reinforcement learning (MARL) system is presented and analyzed from a machine-learning perspective. Moreover, MARL dynamics are discussed via mean field analysis to better understand divergence causes and make implementation recommendations for such systems. Finally, the simulation results show the results of coordination among the users.

42 ENGINEERING↗

Resilient Design of Continuous-time Distributed Optimization Algorithm in the Presence of Cyber-attacks

This paper presents a continuous-time resilient distributed optimization algorithm based on competitive interaction design method on connected graphs in the presence of adversaries. Here, the competitive interaction method allows us to design a network that protects the multi-agent systems from adversaries without requiring high network connectivity. In addition, the proposed algorithm does not require the global information about the number of adversaries. First, we show that the proposed distributed algorithm solves the resilient distributed optimization problem with no attack on the communication links. Second, we show that the proposed continuous-time distributed optimization algorithm on connected graphs converges to the small neighborhood of the optimal solution in the presence of cyber-attacks onto the communication channel. Simulations are presented to illustrate our theoretical results.

97 MATHEMATICS AND COMPUTING↗

Blockchain Enabled Intelligence of Federated Systems (BELIEFS): An attack-tolerant trustable distributed intelligence paradigm

In this article, a Blockchain Enabled Intelligence of Federated Systems (BELIEFS) is proposed to conduct cooperative control for the multi-regional large-scale power system with a multi-agents system (MAS). By establishing a two levels blockchain, each regional AI agent can simultaneously manage intra-regional controllers and cooperate with other AI agents. Under the consensus mechanism, the agents, which respectively conducted distributed deep reinforcement learning (DDRL) algorithm in multi-regions, can have the tolerant capability of malicious attacks in their training process. The demonstration of the proposed approach is within a multi-regional large-scale interconnected power system. Under the mode of "centralized dispatching and hierarchical management", this article aims to definite a mathematical model to deal with the control problem of the power systems. With the comparison experiments, the effectiveness and efficiency of our proposed method in the training process are verified. In addition, malicious attacks are set on the main chain and shard chains to verify the attack-tolerant capability. We expect that such approach and results can suggest a new paradigm of attack-tolerant trustable distributed AI deployment.

97 MATHEMATICS AND COMPUTING↗

Data-driven cyber attack detection and mitigation for decentralized wide-area protection and control in smart grids

Modern power systems have already evolved into complicated cyber physical systems (CPS), often referred to as smart grids, due to the continuous expansion of the electrical infrastructure, the augmentation of the number of heterogeneous system components and players, and the consequential application of a diversity of information and telecommunication technologies to facilitate the Wide Area Monitoring, Protection and Control (WAMPAC) of the day-to-day power system operation. Because of the reliance on cyber technologies, WAMPAC, among other critical functions, is prone to various malicious cyber attacks. Successful cyber attacks, especially those sabotage the operation of Bulk Electric System (BES), can cause great financial losses and social panics. Application of conventional IT security solutions is indispensable, but it often turns out to be insufficient to mitigate sophisticated attacks that deploy zero-day vulnerabilities or social engineering tactics. To further improve the resilience of the operation of smart grids when facing cyber attacks, it is desirable to make the WAMPAC functions per se capable of detecting various anomalies automatically, carrying out adaptive activity adjustments in time and thus staying unimpaired even under attack. Most of the existing research efforts attempt to achieve this by adding novel functional modules, such as model-based anomaly detectors, to the legacy centralized WAMPAC functions. In contrast, this dissertation investigates the application of data-driven algorithms in cyber attack detection and mitigation within a decentralized architecture aiming at improving the situational awareness and self-adaptiveness of WAMPAC. First part of the research focuses on the decentralization of System Integrity Protection Scheme (SIPS) with Multi-Agent System (MAS), within which the data-driven anomaly detection and optimal adaptive load shedding are further explored. An algorithm named as Support Vector Machine embedded Layered Decision Tree (SVMLDT) is proposed for the anomaly detection, which provides satisfactory detection accuracy as well as decision-making interpretability. The adaptive load shedding is carried out by every agent individually with dynamic programming. The load shedding relies on the load profile propagation among peer agents and the attack adaptiveness is accomplished by maintaining the historical mean of load shedding proportion. Load shedding only takes place after the consensus pertaining to the anomaly detection is achieved among all interconnected agents and it serves the purpose of mitigating certain cyber attacks. The attack resilience of the decentralized SIPS is evaluated using IEEE 39 bus model. It is shown that, unlike the traditional centralized SIPS, the proposed solution is able to carry out the remedial actions under most Denial of Service (DoS) attacks. The second part investigates the clustering based anomalous behavior detection and peer-assisted mitigation for power system generation control. To reduce the dimensionality of the data, three metrics are designed to interpret the behavior conformity of generator within the same balancing area. Semi-supervised K-means clustering and a density sensitive clustering algorithm based on Hierarchical DBSCAN (HDBSCAN) are both applied in clustering in the 3D feature space. Aiming to mitigate the cyber attacks targeting the generation control commands, a peer-assisted strategy is proposed. When the control commands from control center is detected as anomalous, i.e. either missing or the payload of which have been manipulated, the generating unit utilizes the peer data to infer and estimate a new generation adjustment value as replacement. Linear regression is utilized to obtain the relation of control values received by different generating units, Moving Target Defense (MTD) is adopted during the peer selection and 1-dimensional clustering is performed with the inferred control values, which are followed by the final control value estimation. The mitigation strategy proposed requires that generating units can communicate with each other in a peer-to-peer manner. Evaluation results suggest the efficacy of the proposed solution in counteracting data availability and data integrity attacks targeting the generation controls. However, the strategy stays effective only if less than half of the generating units are compromised and it is not able to mitigate cyber attacks targeting the measurements involved in the generation control.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Enhancing Cyber Resilience of Networked Microgrids using Vertical Federated Reinforcement Learning

This paper presents a novel federated reinforcement learning (Fed-RL) methodology to inject sufficient resiliency into the operations of the network of microgrids. We consider adversarial actions to the voltage and power control loop reference signals at the grid forming (GFM) inverters in the microgrids which are essential to integrate renewable resources. Therefore, we formulate a resilient reinforcement learning training setup that uses these adversarial injections to generate episodic trajectories and train the RL agents to alleviate their impact on performance. To circumvent the concerns about data-sharing and privacy for different owners of the microgrids in the networked setting, we bring in the aspects of the federated operation to propose novel Fed-RL algorithms. As the dynamics of each microgrid are coupled due to electrical interlinks, the conventional federated RL approaches using decoupled independent environments are not applicable, which leads us to propose a multi-agent vertically federated variation of actor-critic algorithms, namely federated soft actor-critic (FedSAC). We have performed numerical simulations on an IEEE 123-bus benchmark test feeder with three microgrids by creating a customized simulation setup by encapsulating the microgrid dynamic simulations in GridLAB-D/HELICS co-simulation platform with the OpenAI Gym environment and validated the proposed resilient and secured learning methodology.

Artificial Intelligence (AI), reinforcement learni↗

Learning Sequential Distribution System Restoration via Graph-Reinforcement Learning

We report a distribution service restoration algorithm as a fundamental resilient paradigm for system operators provides an optimally coordinated, resilient solution to enhance the restoration performance. The restoration problem is formulated to coordinate distribution generators and controllable switches optimally. A model-based control scheme is usually designed to solve this problem, relying on a precise model and resulting in low scalability. To tackle these limitations, this work proposes a graph-reinforcement learning framework for the restoration problem. We link the power system topology with a graph convolutional network, which captures the complex mechanism of network restoration in power networks and understands the mutual interactions among controllable devices. Latent features over graphical power networks produced by graph convolutional layers are exploited to learn the control policy for network restoration using deep reinforcement learning. The solution scalability is guaranteed by modeling distributed generators as agents in a multi-agent environment and a proper pre-training paradigm. Comparative studies on IEEE 123-node and 8500-node test systems demonstrate the performance of the proposed solution.

24 POWER TRANSMISSION AND DISTRIBUTION↗

ReLIC: Full-Scale Realization of Reinforcement Learning for Infrastructure Control

Prior efforts have shown that deep reinforcement learning (DRL) may provide a new method for controlling networked power systems. Though successful, prior approaches have not yet demonstrated their behavior on systems of realistic scale. This effort examined multiple theoretical and technical approaches to allow a DRL model to operate over a system of 2,000 buses or more. We find that allowing the DRL models to run training episodes in parallel provides near limitless efficiency gains, allowing us to train successful agents to behave on our Kuramoto transmission model of up to 4,000 buses. We further show that we can expand our PowerWorld DRL implementation to systems of up to 25 buses but struggle to go beyond this limit due to PowerWorld’s inability to run multiple instances at once. Finally, we examine a multi-agent approach and find that it performs as well if not better than our existing centralized approach.

97 MATHEMATICS AND COMPUTING↗

Autonomous Intelligent Charging/Discharging of Electric Vehicles using Distributed Multi-Agent ADMM Framework for Grid Ancillary Services

The increasing popularity of Electric Vehicles (EVs) in the distribution grid along with technological advancement in EV electronics such as vehicle to grid (V2G) technique has enabled them to participate in grid ancillary services. To achieve this, the EVs need to establish a contract with third-party aggregators and connect to a charging unit, either residential or commercial. At any time they are connected, the EVs can decide to take part in the ancillary services program offered to them by the aggregators. If agreed, the aggregators will use the EVs as a power source capable of charging/discharging power according to the input signal, and in return, they will be compensated. This inter-temporal nature of charging/discharging is also transforming the traditional optimal power flow (OPF) problem into a dynamic OPF problem. This chapter aims at developing a multi-layer time-dependent optimization algorithm to utilize EV potential and provide ancillary services while maximizing its utilization function. Specifically, in the upper layer, an autonomous distributed ADMM algorithm is developed to optimize the cost for charging/discharging EVs while using them to regulate the voltage at each bus in the distribution grid. The distributed ADMM algorithm is also expanded to the lower layer where the individual EVs active and reactive power is controlled for voltage regulation while maintaining the desired state of the charge of the vehicle at the end of the charging period. Here, the effectiveness and performance improvement of the proposed multi-layer algorithm is illustrated through analytical analysis and simulation results.

Rahman, Towfiq↗

Autonomous Intelligent Charging/Discharging of Electric Vehicles using Distributed Multi-Agent ADMM Framework for Grid Ancillary Services

The increasing popularity of Electric Vehicles (EVs) in the distribution grid along with technological advancement in EV electronics such as vehicle to grid (V2G) technique has enabled them to participate in grid ancillary services. To achieve this, the EVs need to establish a contract with third-party aggregators and connect to a charging unit, either residential or commercial. At any time they are connected, the EVs can decide to take part in the ancillary services program offered to them by the aggregators. If agreed, the aggregators will use the EVs as a power source capable of charging/discharging power according to the input signal, and in return, they will be compensated. This inter-temporal nature of charging/discharging is also transforming the traditional optimal power flow (OPF) problem into a dynamic OPF problem. This chapter aims at developing a multi-layer time-dependent optimization algorithm to utilize EV potential and provide ancillary services while maximizing its utilization function. Specifically, in the upper layer, an autonomous distributed ADMM algorithm is developed to optimize the cost for charging/discharging EVs while using them to regulate the voltage at each bus in the distribution grid. The distributed ADMM algorithm is also expanded to the lower layer where the individual EVs active and reactive power is controlled for voltage regulation while maintaining the desired state of the charge of the vehicle at the end of the charging period. The effectiveness and performance improvement of the proposed multi-layer algorithm is illustrated through analytical analysis and simulation results.

Rahman, Towfiq↗

A Distributed Model Identification Algorithm for Multi-Agent Systems: Preprint

In this study, we investigate agent-based approach for system model identification with emphasis on power distribution system applications. Departing from conventional practices of relying on historical data for offline model identification, we adopt online update approach utilizing real-time data by employing the latest data points for gradient computation. This methodology offers advantages including a large reduction in the communication network's bandwidth requirements by minimizing the data exchanged at each iteration and enabling the model to adapt in real-time to disturbances. Furthermore, we extend our model identification process from linear frameworks to more complex non-linear convex models. This extension is validated through numerical studies demonstrating improved control performance for a synthetic IEEE test case.

data-driven control↗

Enhancing Operational Safety via Agentic Dialogue Hazard Identification Analysis

Operational safety in high-stakes domains such as industrial process control, autonomous, and safety-critical systems demand reliable hazard identification. While large language models (LLMs) have shown promise in automating safety analysis tasks, single-turn, monolithic inference is brittle: it lacks the self-correction, deliberation, and contextual refinement that safety engineers apply iteratively. In this paper, we introduce HAZDIAL, a framework that investigates whether structured agentic dialogue (multi-agent, multi-turn interactions) improves the quality of NLP-based hazard identification over single-pass baselines. We systematically compare two dialogue modalities: adversarial debate and constructive discussion, and propose an genetic algorithm-based agentic interaction optimization. We evaluate all configurations against a curated golden dataset using standard classification metrics (accuracy, precision, recall, F1) and a novel dialogue metrics. This work advances the intersection of dialogue systems, multi-agent reasoning, and AI safety, providing empirical evidence for dialogue-driven hazard analysis.

Das, Sanjay [ORNL] (ORCID:0009000542591915)↗