SEARCH · Search NASA

Results for “multi-agent framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

NASA NTRS2026

Hydrology Copilot: A Cloud-Native Ai System for Hydrological Data Analysis

The emergence of AI-driven Earth observation systems promises to broaden access to petabyte-scale geospatial data beyond domain specialists. However, translating this vision into operational scientific infrastructure requires addressing fundamental challenges in data virtualization, code transparency, and domain-specific reasoning. We present Hydrology Copilot, a cloud-native AI framework for natural-language-driven analysis of Earth observation data. To demonstrate operational capabilities at scale, we implement the system using NASA's North American Land Data Assimilation System version 3 (NLDAS-3), which provides surface meteorological forcing and land-surface model output across North and Central America at 1-km resolution, from which drought diagnostics are derived. The system integrates five core contributions: (1) scalable data virtualization using Kerchunk-based cloud optimized access, achieving a 1.5 to 4.6 times improvement in I/O latency across benchmark queries spanning regional single-day extractions (4.6 times speedup) to continental monthly aggregations (1.5 times speedup); (2) transparent code generation through Microsoft Azure AI Foundry agents that expose executable Python workflows for scientific verification; (3) persistent conversational memory enabling multi-turn analytical discourse across sessions; (4) intelligent query validation that enforces dataset boundaries and resolves ambiguous requests before execution; and (5) a multi-agent architecture coordinating query parsing, code generation, and visualization. We evaluate the system through drought-monitoring workflows, demonstrating reliable code generation, accurate results validated against reference computations and the operational U.S. Drought Monitor, and efficient operation across increasingly complex tasks. By bridging natural-language interfaces with rigorous hydrological analysis, Hydrology Copilot advances beyond proof-of-concept demonstrations to provide a deployable framework for operational Earth science applications.

Data virtualization↗

DOE OSTI2026

Optimal CO 2 storage management considering safety constraints in multi-stakeholder multi-site GCS projects: A Markov game perspective

Geological carbon storage (GCS) projects could involve a diverse array of stakeholders or players from public, private, and regulatory sectors, each with different objectives and responsibilities. Given the complexity, scale, and long-term nature of GCS operations, determining whether individual stakeholders can independently optimize their interests — or whether collaborative coalition agreements are needed — remains a central question for effective GCS project planning and management. To access large, high-quality storage resources, future GCS deployment may increasingly occur in geologically connected sites, where shared geological features such as pressure space and reservoir pore capacity can lead to competitive behavior among stakeholders. In this work, we propose a paradigm based on Markov games to quantitatively investigate how different coalition structures affect the goals of stakeholders. We frame this multi-stakeholder multi-site problem as a multi-agent reinforcement learning problem with safety constraints. Our approach enables agents to learn optimal strategies while complying with safety regulations. We present an example where multiple operators are injecting CO 2 into their respective project areas in a geologically connected basin. To address the high computational cost of repeated simulations of high fidelity models, a previously developed surrogate model based on the Embed-to-Control (E2C) framework is employed. Our results demonstrate the effectiveness of the proposed framework in addressing optimal management of CO 2 storage when multiple stakeholders with different objectives and goals are involved.

58 GEOSCIENCES↗

NASA NTRS2020

Optimal Multi-Agent Search and Rescue Using Potential Field Theory

This paper presents an algorithm for efficient search and rescue using a multi-agent system of vehicles. The algorithm uses an artificial potential field combined with a time-varying reward function for visiting various points within the search area. The reward function is used to weight the attractiveness of these points in the potential field, and collision avoidance terms are used to repel vehicles from each other, which has the additional effect of reducing duplication of searching efforts. The algorithm generates velocity commands in real-time based on communication with the other vehicles. This framework allows vehicles to react in a dynamic environment, which is a significant advantage to simply following a-priori defined trajectories. Simulation results are presented to demonstrate the ability of the algorithm to cover the search area effectively. The algorithm is also compared to an exhaustive lawn-mower search pattern. This comparison is done via a Monte Carlo simulation with randomized target initial conditions and trajectories. The time to find the target improved by 16 and 30% in the mean and median, respectively. Additionally, this paper presents a method for analyzing the upper bound for time to find a target under the potential field guidance algorithm assuming a radially expanding search area.

John R Cooper↗

DOE OSTI2026

Optimization-based approaches to control of connected and automated vehicles: Principles, complexities, applications, challenges, and outlook

Safe and optimal motion control for connected and automated vehicles (CAVs) poses a fundamental optimization challenge at the intersection of system complexity, environmental uncertainty, and stringent real-time constraints. Existing surveys address this challenge in isolation – focusing either on specific control techniques or individual uncertainty sources – without providing a unified framework that characterizes the trade-offs among computational tractability, performance verifiability, and adaptive generalization across paradigms. This review addresses that gap by presenting a cohesive analytical framework concentrated on the decision-making and trajectory optimization layers of the CAV autonomy stack. We systematically analyze three major optimization paradigms – first-principles model-based optimization, data-driven methods, and hybrid synergistic architectures – evaluating each against four core complexity axes: problem formulation, constraint handling, optimality guarantees, and robustness. Key applications including platooning, trajectory planning, collision avoidance, and cooperative control are examined to reveal recurring methodological patterns and critical operational constraints that limit real-world performance. Our synthesis identifies verifiable hybrid architectures, incentive-aligned multi-agent cooperation, and hardware-algorithm co-design as the defining research frontiers, and distills a targeted agenda for developing CAV control systems that are simultaneously safe, computationally efficient, and deployable in the full complexity of real-world traffic environments.

Muzahid, Abu Jafar Md [University of Tennessee, Kn↗

DOE OSTI2023

Reinforcement Learning of Structured Stabilizing Control for Linear Systems With Unknown State Matrix

This paper delves into designing feedback control gains for a continuous-time linear quadratic regulator (LQR) problem that is constrained to certain predefined structure with unknown state matrix. We bring forth the ideas from reinforcement learning (RL) in conjunction with sufficient stability and performance guarantees in order to design these structured gains using the trajectory measurements of states and controls. Here we first formulate a model-based framework using dynamic programming (DP) to embed the structural constraint to the LQR gain computation in the continuous-time setting, and then subsequently, formulate a policy iteration RL algorithm that can alleviate the requirement of known state matrix in conjunction with maintaining the feedback gain structure. The design enables a distributed learning control design which is necessary for many large-scale cyber-physical systems. Theoretical guarantees are provided for stability and convergence of the structured reinforcement learning (SRL) algorithm. We validate our theoretical results with numerical simulations on a multi-agent networked linear time-invariant (LTI) dynamic system.

42 ENGINEERING↗

DOE OSTI2026

AutoLabs: cognitive multi-agent systems with self-correction for autonomous chemical experimentation

The automation of chemical research through self-driving laboratories (SDLs) promises to accelerate scientific discovery, yet the reliability and granular performance of the underlying AI agents remain critical, under-examined challenges. In this work, we introduce AutoLabs, a self-correcting, multi-agent architecture designed to autonomously translate natural-language instructions into executable protocols for a high-throughput liquid handler. The system engages users in dialogue, decomposes experimental goals into discrete tasks for specialized agents, performs tool-assisted stoichiometric calculations, and iteratively self-corrects its output before generating a hardware-ready file. We present a comprehensive evaluation framework featuring five benchmark experiments of increasing complexity, from simple sample preparation to multi-plate timed syntheses. Through a systematic ablation study of 20 agent configurations, we assess the impact of reasoning capacity, architectural design (single- vs. multi-agent), tool use, and self-correction mechanisms. Our results demonstrate that agent reasoning capacity is the most critical factor for success, reducing quantitative errors in chemical amounts (nRMSE) by over 85% in complex tasks. When combined with a multi-agent architecture and iterative self-correction, AutoLabs approaches expert-authored reference procedures on the benchmark (F1-score > 0.89) on challenging multi-plate syntheses. These findings establish a clear blueprint for developing robust and trustworthy AI partners for autonomous laboratories, highlighting the synergistic effects of modular design, advanced reasoning, and self-correction to ensure both performance and reliability in high-stakes scientific applications. Code: https://github.com/pnnl/autolabs

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRS2013

Work Practice Simulation of Complex Human-Automation Systems in Safety Critical Situations: The Brahms Generalized berlingen Model

The transition from the current air traffic system to the next generation air traffic system will require the introduction of new automated systems, including transferring some functions from air traffic controllers to on-board automation. This report describes a new design verification and validation (V&V) methodology for assessing aviation safety. The approach involves a detailed computer simulation of work practices that includes people interacting with flight-critical systems. The research is part of an effort to develop new modeling and verification methodologies that can assess the safety of flight-critical systems, system configurations, and operational concepts. The 2002 Ueberlingen mid-air collision was chosen for analysis and modeling because one of the main causes of the accident was one crew's response to a conflict between the instructions of the air traffic controller and the instructions of TCAS, an automated Traffic Alert and Collision Avoidance System on-board warning system. It thus furnishes an example of the problem of authority versus autonomy. It provides a starting point for exploring authority/autonomy conflict in the larger system of organization, tools, and practices in which the participants' moment-by-moment actions take place. We have developed a general air traffic system model (not a specific simulation of Überlingen events), called the Brahms Generalized Ueberlingen Model (Brahms-GUeM). Brahms is a multi-agent simulation system that models people, tools, facilities/vehicles, and geography to simulate the current air transportation system as a collection of distributed, interactive subsystems (e.g., airports, air-traffic control towers and personnel, aircraft, automated flight systems and air-traffic tools, instruments, crew). Brahms-GUeM can be configured in different ways, called scenarios, such that anomalous events that contributed to the Überlingen accident can be modeled as functioning according to requirements or in an anomalous condition, as occurred during the accident. Brahms-GUeM thus implicitly defines a class of scenarios, which include as an instance what occurred at Überlingen. Brahms-GUeM is a modeling framework enabling "what if" analysis of alternative work system configurations and thus facilitating design of alternative operations concepts. It enables subsequent adaption (reusing simulation components) for modeling and simulating NextGen scenarios. This project demonstrates that BRAHMS provides the capacity to model the complexity of air transportation systems, going beyond idealized and simple flights to include for example the interaction of pilots and ATCOs. The research shows clearly that verification and validation must include the entire work system, on the one hand to check that mechanisms exist to handle failures of communication and alerting subsystems and/or failures of people to notice, comprehend, or communicate problematic (unsafe) situations; but also to understand how people must use their own judgment in relating fallible systems like TCAS to other sources of information and thus to evaluate how the unreliability of automation affects system safety. The simulation shows in particular that distributed agents (people and automated systems) acting without knowledge of each others' actions can create a complex, dynamic system whose interactive behavior is unexpected and is changing too quickly to comprehend and control.

complex systems↗

DOE OSTI2024

A tri-level optimization model for interdependent infrastructure network resilience against compound hazard events

Resilient operation of interdependent infrastructures against compound hazard events is essential for maintaining societal well-being. To address consequence assessment challenges in this problem space, we propose a novel policy-guided tri-level optimization model applied to a proof-of-concept case study with fuel distribution and transportation networks – encompassing one realistic network; one fictitious, yet realistic network; as well as networks drawn from three synthetic distributions. Mathematically, our approach takes the form of a defender-attacker-defender (DAD) model—a multi-agent tri-level optimization, comprised of a defender, attacker, and an operator acting in sequence. Here, in this study, our notional operator may choose proxy actions to operate an interdependent system comprised of fuel terminals and gas stations (functioning as supplies) and a transportation network with traffic flow (functioning as demand) to minimize unmet demand at gas stations. A notional attacker aims to hypothetically disrupt normal operations by reducing supply at the supply terminals, and the notional defender aims to identify best proxy defense policy options which include hardening supply terminals or allowing alternative distribution methods such as trucking reserve supplies. We solve our DAD formulation at a metropolitan scale and present practical defense policy insights against hypothetical compound hazards. We demonstrate the generalizability of our framework by presenting results for a realistic network; a fictitious, yet realistic network; as well as for three networks drawn from synthetic distributions. Additionally, we demonstrate the scalability of the framework by investigating runtime performance as a function of the network size. Steps for future research are also discussed.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

NASA NTRS2008

Observability and Estimation of Distributed Space Systems via Local Information-Exchange Networks

In this work, we develop an approach to formation estimation by explicitly characterizing formation's system-theoretic attributes in terms of the underlying inter-spacecraft information-exchange network. In particular, we approach the formation observer/estimator design by relaxing the accessibility to the global state information by a centralized observer/estimator- and in turn- providing an analysis and synthesis framework for formation observers/estimators that rely on local measurements. The noveltyof our approach hinges upon the explicit examination of the underlying distributed spacecraft network in the realm of guidance, navigation, and control algorithmic analysis and design. The overarching goal of our general research program, some of whose results are reported in this paper, is the development of distributed spacecraft estimation algorithms that are scalable, modular, and robust to variations inthe topology and link characteristics of the formation information exchange network. In this work, we consider the observability of a spacecraft formation from a single observation node and utilize the agreement protocol as a mechanism for observing formation states from local measurements. Specifically, we show how the symmetry structure of the network, characterized in terms of its automorphism group, directly relates to the observability of the corresponding multi-agent system The ramification of this notion of observability over networks is then explored in the context of distributed formation estimation.

observability↗

DOE OSTI2025

Multi-agent AI collaboration for digital twin development and assessment

Developing a digital twin (DT) model involves different steps that encompass formulating requirements, model development, implementation, and assessment with respect to real applications. Human expertise is required to coordinate and implement different steps in the DT development and assessment process. However, certain parts of this process can be automated using artificial intelligence (AI) agents for efficient workflow development. In this work, we test and analyze a multiagent AI collaboration with humans in the loop to automate different elements of the DT development and assessment process. To implement the workflow for multiagent AI DT development and assessment, we use Autogen, a multiagent framework developed by Microsoft. Autogen offers a modular and flexible framework for configuring and designing task-specific multiagent workflows. In this framework, large language models (LLMs) form the core intelligence of the AI agents where the quality and performance of the automated element is governed by the inherent capabilities and knowledge base of the LLM. We use retrieval augmented generation to supplement the LLM with relevant domain-specific information for DT requirement formulation. We illustrate this multiagent workflow using a case study on a thermal energy storage system, focusing on how AI agents can collaborate with humans to expedite and optimize different elements of DT development and assessment process.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

Decentralized Voltage Control of Large-Scale Distribution System with PVs Based on MADRL

This paper proposes a model-free decentralized control framework for the voltage regulation of large-scale distribution systems through the coordinated control of PV inverters. This is achieved by developing a novel interaction mechanism between the surrogate model and the centralized training and decentralized execution multiagent deep reinforcement learning framework. Specifically, the sparse Gaussian processes regression method is first utilized to develop the surrogate model of the original distribution system for reward calculation during the training stage, where each agent represents a sub-region in the centralized fashion for coordination strategy learning. After that, the learned control rules are used to inform controllers within each sub-region for real-time decisions with only local measurements. Comparative tests among various methods on the EPRI Ckt5 test system demonstrate the effectiveness of the proposed method.

distribution system↗

DOE OSTI2022

Reinforcement Learning for Volt- Var Control: A Novel Two-stage Progressive Training Strategy

This paper develops a reinforcement learning (RL) approach to solve a cooperative, multi-agent Volt-Var Control (VVC) problem for high solar penetration distribution systems. The ingenuity of our RL method lies in a novel two-stage progressive training strategy that can effectively improve training speed and convergence of the machine learning algorithm. In Stage 1 (individual training), while holding all the other agents inactive, we separately train each agent to obtain its own optimal VVC actions in the action space: fconsume, generate, do-nothingg. In Stage 2 (cooperative training), all agents are trained again coordinatively to share VVC responsibility. Rewards and costs in our RL scheme include (i) a system-level reward (for taking an action), (ii) an agent-level reward (for doing-nothing), and (iii) an agent-level action cost function. This new framework allows rewards to be dynamically allocated to each agent based on their contribution while accounting for the trade-off between control effectiveness and action cost. The proposed methodology is tested and validated in a modified IEEE 123-bus system using realistic PV and load profiles. Simulation results confirm that the proposed approach is robust and computationally efficient; and it achieves desirable volt-var control performance under a wide range of operation conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Behavior, Energy, Autonomy, Mobility Modeling Framework (BEAM) v1.0

The Behavior, Energy, Autonomy, and Mobility (BEAM) model is an integrated, agent-based travel demand simulation framework. Individual agents express preferences through a utility- maximizing evolutionary algorithm that minimizes each individual’s cost and time spent traveling via diverse modal options, including the competition for scarce supply resources such as parking spaces and charging infrastructure. BEAM simulates the essential elements that compose a dynamic transportation system. From the road network, parking and charging infrastructure, to the transit system and a synthetic population with plans and preferences, the virtual system is an amalgamation of multiple spatially resolved layers that together represent an integrated transportation system. BEAM is an extension to the MATSim (Multi-Agent Transportation Simulation) model, where agents employ reinforcement learning across successive simulated days to maximize their personal utility through plan mutation (exploration) and selecting between previously executed plans (exploitation). The BEAM model shifts some of the behavioral emphasis in MATSim from across-day planning to within- day planning, where agents dynamically respond to the state of the system during the mobility simulation. In BEAM, agents can plan across all major modes of travel including driving, walking, biking, transit, and demand-responsive ride hailing. It is designed to integrate with other open source transportation models, such as ActivitySim.

Lazarus, Jessica↗

NASA NTRS2000

Optimal Reward Functions in Distributed Reinforcement Learning

We consider the design of multi-agent systems so as to optimize an overall world utility function when (1) those systems lack centralized communication and control, and (2) each agents runs a distinct Reinforcement Learning (RL) algorithm. A crucial issue in such design problems is to initialize/update each agent's private utility function, so as to induce best possible world utility. Traditional 'team game' solutions to this problem sidestep this issue and simply assign to each agent the world utility as its private utility function. In previous work we used the 'Collective Intelligence' framework to derive a better choice of private utility functions, one that results in world utility performance up to orders of magnitude superior to that ensuing from use of the team game utility. In this paper we extend these results. We derive the general class of private utility functions that both are easy for the individual agents to learn and that, if learned well, result in high world utility. We demonstrate experimentally that using these new utility functions can result in significantly improved performance over that of our previously proposed utility, over and above that previous utility's superiority to the conventional team game utility.

Wolpert, David H.↗

NASA NTRS2000

Optimal Wonderful Life Utility Functions in Multi-Agent Systems

The mathematics of Collective Intelligence (COINs) is concerned with the design of multi-agent systems so as to optimize an overall global utility function when those systems lack centralized communication and control. Typically in COINs each agent runs a distinct Reinforcement Learning (RL) algorithm, so that much of the design problem reduces to how best to initialize/update each agent's private utility function, as far as the ensuing value of the global utility is concerned. Traditional team game solutions to this problem assign to each agent the global utility as its private utility function. In previous work we used the COIN framework to derive the alternative Wonderful Life Utility (WLU), and experimentally established that having the agents use it induces global utility performance up to orders of magnitude superior to that induced by use of the team game utility. The WLU has a free parameter (the clamping parameter) which we simply set to zero in that previous work. Here we derive the optimal value of the clamping parameter, and demonstrate experimentally that using that optimal value can result in significantly improved performance over that of clamping to zero, over and above the improvement beyond traditional approaches.

Wolpert, David H.↗

NASA NTRS2005

Distributed Optimization

We demonstrate a new framework for analyzing and controlling distributed systems, by solving constrained optimization problems with an algorithm based on that framework. The framework is ar. information-theoretic extension of conventional full-rationality game theory to allow bounded rational agents. The associated optimization algorithm is a game in which agents control the variables of the optimization problem. They do this by jointly minimizing a Lagrangian of (the probability distribution of) their joint state. The updating of the Lagrange parameters in that Lagrangian is a form of automated annealing, one that focuses the multi-agent system on the optimal pure strategy. We present computer experiments for the k-sat constraint satisfaction problem and for unconstrained minimization of NK functions.

Macready, William↗

DOE OSTI2023

A Novel Multi-Agent Deep Reinforcement Learning-enabled Distributed Power Allocation Scheme for mmWave Cellular Networks

We consider the power allocation problem over shared spectrum for millimeter-Wave (mmWave) cellular downlink. Existing approaches usually find sub-optimal solutions by solving a non-convex optimization which leads to scalability issues due to centralized control. Therefore, distributed and adaptive approaches are desirable. Recently, model-free Deep Reinforcement Learning (DRL) has achieved success in such wireless resource management tasks. By modeling the radio environment as a Markov Decision Process (MDP) with the base stations (BSs) being the agents, power allocation can be automated at the agent level with comparable throughput performance to conventional centralized schemes. The multi-agent setting presents new challenges as the radio environment is impacted by the joint actions of the agents and is no longer stationary from any individual agent’s perspective. Existing literature bypasses this non-stationarity violation by ignoring it which may cause performance degradation. To tackle this issue, we propose a distributed continuous power allocation scheme based on a modified version of multi-agent Deep Deterministic Policy Gradient (MADDPG) that is tailored for the distributed multiple-agent setting. The proposed scheme employs a centralized-training distributed-execution framework where Q-functions are trained over subsets of BSs while each BS determines its transmit power based only on its own local observation. It admits constant per-BS communication and computation complexity and is thus scalable to large networks. Numerical evaluation shows that the proposed scheme adapts well to a wide range of interference conditions and can achieve comparable or better performance than several state-of-the-art non-learning approaches.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2024

A Distributed Model Identification Algorithm for Multi-Agent Systems: Preprint

In this study, we investigate agent-based approach for system model identification with emphasis on power distribution system applications. Departing from conventional practices of relying on historical data for offline model identification, we adopt online update approach utilizing real-time data by employing the latest data points for gradient computation. This methodology offers advantages including a large reduction in the communication network's bandwidth requirements by minimizing the data exchanged at each iteration and enabling the model to adapt in real-time to disturbances. Furthermore, we extend our model identification process from linear frameworks to more complex non-linear convex models. This extension is validated through numerical studies demonstrating improved control performance for a synthetic IEEE test case.

data-driven control↗