Search NASA⌕ Search

SEARCH · Search NASA

Results for “resilient system, agent, control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2012

Resilient Control System Execution Agent (ReCoSEA)

In an increasingly networked world, critical infrastructure systems suffer from two types of vulnerability. The first is the traditionally recognized problem of monitoring the systems for faults and failures, recognizing and analyzing data, and responding with real understanding to the problems of the system. Increasingly complex systems create the opportunity for single points of failure to cascade when inaccurate assessment of system health increases response time or leads to faulty analysis of the problems involved. A second problem involves vulnerability to cyber intrusion, in which bad actors can mask system deterioration or present false data about system status. A resilient system will protect stability, efficiency, and security. To ensure these three states, the system must react to changing conditions within the system with coordination: no one component of the system can be allowed to react to problems without real consideration of the effects of that action on other components within the system. Systems with multi-agent design typically have three layers of action, a management layer, a coordination layer, and an execution layer. A resilient multi-agent system will emphasize functions of the execution layer, which has the responsibility of initiating actions, monitoring, analyzing, and controlling its own processes, while feeding information back to the higher levels of management and coordination. The design concept of a resilient control system execution agent (ReCoSEA) grows out of these underpinnings, and through the use of computational intelligence techniques, this paper suggests an associated design methodology.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Systems and methods for control system security

A resilient security agent determines a cyber and/or physical health of a control system by, inter alia, communicating cyber-physical key data through cyber-physical control paths of the system, and determining error introduced by the communication. The resilient security agent may be further configured to verify the integrity of acquired cyber-physical state information. The cyber health of the control system may be evaluated by comparing the acquired cyber state information to one or more cyber state profiles. The physical health of the control system may be evaluated by comparing the acquired physical state information to one or more physical state profiles.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

Agent-based Cyber Control Strategy Design for Resilient Control Systems: Concepts, Architecture and Methodologies

The implementation of automated regulatory control has been around since the middle of the last century through analog means. It has allowed engineers to operate the plant more consistently by focusing on overall operations and settings instead of individual monitoring of local instruments (inside and outside of a control room). A similar approach is proposed for cyber security, where current border-protection designs have been inherited from information technology developments that lack consideration of the high-reliability, high consequence nature of industrial control systems. Instead of an independent development, however, an integrated approach is taken to develop a holistic understanding of performance. This performance takes shape inside a multiagent design, which provides a notional context to model highly decentralized and complex industrial process control systems, the nervous system of critical infrastructure. The resulting strategy will provide a framework for researching solutions to security and unrecognized interdependency concerns with industrial control systems.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Ripple-Type Control for Enhancing Resilience of Networked Physical Systems

Distributed control agents have been advocated as an effective means for improving the resiliency of our physical infrastructures under unexpected events. Purely local control has been shown to be insufficient, centralized optimal resource allocation approaches can be slow. In this context, we put forth a hybrid low-communication saturation-driven protocol for the coordination of control agents that are distributed over a physical system and are allowed to communicate with peers over a “hotline” communication network. According to this protocol, agents act on local readings unless their control resources have been depleted, in which case they send a beacon for assistance to peer agents. Our ripple-type scheme triggers communication locally only for the agents with saturated resources and it is proved to converge. Moreover, under a monotonicity assumption on the underlying physical law coupling control outputs to inputs, the devised control is proved to converge to a configuration satisfying safe operational constraints. The assumption is shown to hold for voltage control in electric power systems and pressure control in water distribution networks. Numerical tests corroborate the efficacy of the novel scheme.

distributed control↗
DOE OSTI2022

Introduction to the special section on learning and security for multi-agent systems

This report features six articles that provide an overview of the emerging research topics at the intersection learning, security, and multi-agent systems. Recent years have witnessed a surge in the number of works at their intersections, and they have appeared in system and control communities as well as many other communities in artificial intelligence, cyber–physical systems, and economics. The articles in this special section give accessible and comprehensive tutorials and surveys for a broad systems and control audience, covering topics including adversarial machine learning, multi-agent reinforcement learning, cyber resilience, resilient control systems, and game design. It is hopeful that this special section will spawn future interest and cross-disciplinary collaborations in this emerging transdisciplinary research area.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Ripple-Type Control for Enhancing Resilience of Networked Physical Systems: Preprint

Distributed control agents have been advocated as an effective means for improving the resiliency of our physical in-frastructures under unexpected events. While purely local control has been shown to be insufficient, centralized optimal resource allocation approaches can be slow. In this context, we put forth a hybrid low-communication saturation-driven protocol for the coordination of control agents that are distributed over a physicalsystem and are allowed to communicate with peers over a 'hot-line' communication network. According to this protocol, agents act upon on local readings unless their control resources have been depleted, in which case they send a beacon for assistance to peer agents. Our ripple-type scheme triggers communication locally only for the agents with saturated resources, and is proved to converge. Moreover, under a monotonicity assumption on the underlying physical law coupling control outputs to inputs, the devised control is proved to converge to a configuration satisfying safe operational constraints. The assumption is shown to hold for voltage control in electric power systems and pressure control in water distribution networks. Numerical tests on both networks corroborate the efficacy of the novel scheme.

distributed control↗
DOE OSTI2024

Reinforcement Learning to Enhance Optimal Operation of Resilient Community Energy Systems

This paper presents a novel model-free multi-agent Reinforcement Learning (RL) control method to enhance the resilience of community energy systems in island mode, which coordinates multiple objectives without the necessity of identifying system models that require expert knowledge. Specifically, a community-level coordinator agent is designed to allocate renewable energy resources among different buildings, and multiple building-level agents are developed to optimize load schedules based on limited energy resources and requirements of building loads and occupants’ comfort. In a two-day evaluation, our RL approach demonstrated a similar performance against MPC without requiring system models and formulation of optimization problems as required in MPC.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗
DOE OSTI2019

Implementation and Evaluation of IEC 61850-based Distributed Control System for Microgrid Applications

This paper presents a resilient, modular and scalable distributed control architecture for microgrid systems which uses distributed agents controlling individual loads, network switches, generators or storage devices to provide intelligent power management and efficient microgrid operation. The communication between the distributed agents is realized by implementing IEC 61850 data models and GOOSE messages. The data models include state representation, measurements at various points, and dispatch requests for reliable operation of microgrid control system. System resiliency is enhanced through implementing distributed agent architecture with no single point of failure.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Multi-Agent Control Planes for Quantum Networks: A Scalable Architecture for Autonomous Quantum Internet Management

Quantum networks are expected to enable distributed quantum computing, secure communication, and global entanglement distribution. However, operating such networks presents significant challenges, including stochastic quantum processes, fragile entanglement resources, dynamic topology, and cross-layer control requirements. Current quantum network control architectures largely rely on centralized or hierarchical controllers inspired by classical software-defined networking (SDN). While effective for small testbeds, these approaches face scalability, latency, and reliability limitations as quantum networks grow. This paper proposes a multi-agent control plane architecture for quantum networks. In this design, intelligent software agents operate at quantum nodes, repeaters, and orchestration layers, collectively managing entanglement generation, routing, purification, and scheduling. The distributed intelligence of the agent system allows the network to adapt dynamically to quantum hardware variability and environmental noise. We argue that multi-agent systems provide significant advantages over centralized control approaches, including scalability, resilience, local autonomy, and real-time adaptation. The paper discusses architectural design principles, agent coordination mechanisms, and research challenges in deploying multi-agent control planes for the emerging quantum Internet.

Alnajjar, Anees [ORNL] (ORCID:0000000237101601)↗
DOE OSTI2025

Virtual Agents-Based Attack-Resilient Distributed Control for Islanded AC Microgrid

Due to its dependence on a communication network, distributed secondary control of microgrids is susceptible to denial-of-service (DoS) attacks in channel shutdown mode, which may negatively impact the network connectivity and thus deteriorate the coordination and power sharing among distributed generators (DGs). Honeypot is a common method for cyber deception by introducing fake targets. However, in the context of microgrid, the misleading information spread by honeypots will also impact the system performance. This paper proposes an attack-resilient distributed control for AC microgrids utilizing virtual agents (VAs) to counteract both DoS edge and node attacks. The VAs are designed to not impact the system’s steady state during normal operation but to share information among neighboring real agents and serve as dummy targets for DoS attacks. The control with VAs is implemented by a primal-dual gradient based distributed algorithm to efficiently obtain a practical solution for voltage/frequency regulation and power sharing. The simulation results on a 4-DG test system and a modified IEEE 34-bus system show that 1) VAs do not impact the normal functionality of the test system, and 2) deploying VAs can enhance the resilience of the microgrid control against DoS edge and node attacks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Towards Resilient Design of Leader-following Consensus with Attack Identification and Privacy Preservation Capabilities

This paper considers a leader-following consensus in the presence of unknown but bounded cyber-attacks. Specifically, we consider the following cyber-attack scenarios: (i) an attacker aims to destabilize the consensus dynamics by injecting exogenous signals to both the actuators of the followers and/or the communication network, (ii) an eavesdropper adversary aims to obtain information on the physical state of the agents. To this end, a novel resilient leader-following consensus algorithm based on a competitive interaction method is proposed. In addition, it is demonstrated that by appropriately choosing the information exchanged between the agents, the proposed control framework also enables the cooperative system to either distributively identify the compromised communication links in real-time or to protect the privacy of the physical state of the agents from the eavesdropper. Here, a numerical example is provided to illustrate the proposed resilient control algorithms.

Gusrialdi, Azwirman↗
DOE OSTI2022

Network Reconfiguration for Enhanced Operational Resilience Using Reinforcement Learning

This paper proposes a reinforcement learning-based approach for distribution network reconfiguration(DNR) to enhance the resilience of the electric power supply. Resilience enhancements usually require solving large-scale stochastic optimization problems that are computationally expensive and sometimes infeasible. The exceptional performance of reinforcement learning techniques has encouraged their adoption in various power system control studies, specifically resilience-based real-time applications. In this paper, a single agent framework is developed using an Actor-Critic algorithm (ACA) to determine statuses of tie-switches in a distribution feeder impacted by an extreme weather event. The proposed approach provides a fast-acting control algorithm that reconfigures the feeder topology to reduce or even avoid load shedding. The problem is formulated as a discrete Markov decision process in such a way that a system state captures the system topology and its operational characteristics. An action is made to open or close a specific set of tie-switches after which a reward is calculated to evaluate the practicality and advantage of that action. The iterative Markov process is used to train the proposed ACA under diverse failure scenarios and is demonstrated on the 33-node distribution feeder system. Results show the capability of the proposed ACA to determine proper switching action of tie-switches with accuracy exceeding 93%.

actor critic↗
DOE OSTI2025

CyRRL (Cyber Resilient Reinforcement Learning for grid voltage control) [SWR-24-115]

This codebase contains a multi-agent, actor-critic reinforcement learning implementation for cyber-resilient grid voltage control. It uses a 123-bus OpenDSS system as the environment, with three-phase power flow translating nodal power injections into solved nodal voltages. The reward function penalizes deviations from nominal voltage as well as reactive power dispatch, while encouraging agents to take actions that result in fast convergence to nominal conditions. The codebase models false data injection attacks and includes functionality for training, testing, hyper-parameter tuning, and visualization.

Murphy, Sinnott [National Renewable Energy Laborat↗
DOE OSTI2023

Utilizing grid–supportive load response to shape resilient frequency control of the power grid

The increasing penetration of renewable energy sources and the retirement of conventional generation units have decreased system inertia, making power systems more vulnerable to resilience and stability issues. To address this problem, this paper proposes a novel approach using grid-supportive loads (GSLs) to provide a fast and concise primary frequency response and a deep deterministic policy gradient agent-based secondary controller to restore the system frequency to the nominal value. The proposed method is evaluated on the single-area and multi-area test systems. The simulation results demonstrate that using GSLs enhances the power system's stability and resilience. Compared to conventional controllers, the frequency nadir is improved with GSLs. Additionally, the proposed method effectively enhances resilience even with high penetration. These findings indicate that the proposed approach can improve the resilience and stability of power systems and provide a promising solution for future power systems. The results of this study underscore the importance of utilizing innovative approaches to enhance the stability and resilience of power systems in the context of high penetration of renewable energy sources and the retirement of conventional generation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

DSS-SimPy-RL (Open-DSS and SimPy based Cyber-Physical RL environment) [SWR-23-29]

Recently, numerous data-driven approaches to control an electric grid using machine learning techniques have been investigated. With the advancement of reinforcement learning (RL) based techniques, gradually the conventional optimization based solvers are being replaced with RL approach where there is uncertainty in the environment such as renewable generation or cyber system emulation. However, to train an agent efficiently, it requires numerous interactions with an environment to learn the best policies. There are numerous RL environments for the power systems based on some well-known simulators, similarly there are environment for communication domains. While majority of the cyber emulators are based in an UNIX environment, the power simulators are based in the Windows-based operating system, the generation of cyber-physical mixed domain RL environment has been challenging. Existing co-simulation methods are efficient but resource and time intensive to generate large scale data set for training RL agents. Hence, this software focuses on development and validation of a mixed domain RL environment using Open DSS for the physical side and leverages a discrete event simulator python package, SimPy, for cyber-side emulation which is Operating Systems agnostic. Further utilizing this software co-simulation and training RL agents for re-routing based resilient control for network reconfiguration and volt-var control in power distribution feeder are performed.

Sahu, Abhijeet↗
DOE OSTI2021

Impact of Blockchain Delay on Grid-Tied Solar Inverter Performance

This paper investigates the impact of the delay resulting from a blockchain, a promising security measure, for a hierarchical control system of inverters connected to the grid. The blockchain communication network is designed at the secondary control layer for resilience against cyberattacks. To represent the latency in the communication channel, a model is developed based on the complexity of the blockchain framework. Taking this model into account, this work evaluates the plant’s performance subject to communication delays, introduced by the blockchain, among the hierarchical control agents. In addition, this article considers an optimal model-based control strategy that performs the system’s internal control loop. The work shows that the blockchain’s delay size influences the convergence of the power supplied by the inverter to the reference at the point of common coupling. In the results section, real-time simulations on OPAL-RT are performed to test the resilience of two parallel inverters with increasing blockchain complexity.

41 EE - Solar Energy Technologies Office (EE-4S)↗