SEARCH · Search NASA

Results for “multiple agents”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

NASA NTRS2014

Swarmie User Manual: A Rover Used for Multi-agent Swarm Research

The ability to create multiple functional yet cost effective robots is crucial for conducting swarming robotics research. The Center Innovation Fund (CIF) swarming robotics project is a collaboration among the KSC Granular Mechanics and Regolith Operations (GMRO) group, the University of New Mexico Biological Computation Lab, and the NASA Ames Intelligent Robotics Group (IRG) that uses rovers, dubbed "Swarmies", as test platforms for genetic search algorithms. This fall, I assisted in the development of the software modules used on the Swarmies and created this guide to provide thorough instructions on how to configure your workspace to operate a Swarmie both in simulation and out in the field.

Robotics↗

NASA NTRS1998

Preliminary Work for Examining the Scalability of Reinforcement Learning

Researchers began studying automated agents that learn to perform multiple-step tasks early in the history of artificial intelligence (Samuel, 1963; Samuel, 1967; Waterman, 1970; Fikes, Hart & Nilsonn, 1972). Multiple-step tasks are tasks that can only be solved via a sequence of decisions, such as control problems, robotics problems, classic problem-solving, and game-playing. The objective of agents attempting to learn such tasks is to use the resources they have available in order to become more proficient at the tasks. In particular, each agent attempts to develop a good policy, a mapping from states to actions, that allows it to select actions that optimize a measure of its performance on the task; for example, reducing the number of steps necessary to complete the task successfully. Our study focuses on reinforcement learning, a set of learning techniques where the learner performs trial-and-error experiments in the task and adapts its policy based on the outcome of those experiments. Much of the work in reinforcement learning has focused on a particular, simple representation, where every problem state is represented explicitly in a table, and associated with each state are the actions that can be chosen in that state. A major advantage of this table lookup representation is that one can prove that certain reinforcement learning techniques will develop an optimal policy for the current task. The drawback is that the representation limits the application of reinforcement learning to multiple-step tasks with relatively small state-spaces. There has been a little theoretical work that proves that convergence to optimal solutions can be obtained when using generalization structures, but the structures are quite simple. The theory says little about complex structures, such as multi-layer, feedforward artificial neural networks (Rumelhart & McClelland, 1986), but empirical results indicate that the use of reinforcement learning with such structures is promising. These empirical results make no theoretical claims, nor compare the policies produced to optimal policies. A goal of our work is to be able to make the comparison between an optimal policy and one stored in an artificial neural network. A difficulty of performing such a study is finding a multiple-step task that is small enough that one can find an optimal policy using table lookup, yet large enough that, for practical purposes, an artificial neural network is really required. We have identified a limited form of the game OTHELLO as satisfying these requirements. The work we report here is in the very preliminary stages of research, but this paper provides background for the problem being studied and a description of our initial approach to examining the problem. In the remainder of this paper, we first describe reinforcement learning in more detail. Next, we present the game OTHELLO. Finally we argue that a restricted form of the game meets the requirements of our study, and describe our preliminary approach to finding an optimal solution to the problem.

Clouse, Jeff↗

DOE OSTI2024

Reinforcement Learning to Enhance Optimal Operation of Resilient Community Energy Systems

This paper presents a novel model-free multi-agent Reinforcement Learning (RL) control method to enhance the resilience of community energy systems in island mode, which coordinates multiple objectives without the necessity of identifying system models that require expert knowledge. Specifically, a community-level coordinator agent is designed to allocate renewable energy resources among different buildings, and multiple building-level agents are developed to optimize load schedules based on limited energy resources and requirements of building loads and occupants’ comfort. In a two-day evaluation, our RL approach demonstrated a similar performance against MPC without requiring system models and formulation of optimization problems as required in MPC.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗

NASA NTRS2013

The Regolith Biters: A Divide-And-Conquer Architecture for Sample-Return Missions

The collective interaction of simple systems can be leveraged to attain complex goals. Based on this principle, we envision space system architectures where the core functional components are decoupled, autonomous, and cooperative. We aim to pursue this vision in the context of small-body sample-return missions. After all, no experimental study sheds more light into our understanding of the origin and evolution of the Solar System than the analysis of samples from asteroids and comets. We also believe that their study is important from a strategic perspective: meteorite impacts pose a direct and credible threat to life on Earth, and the development of contingency small-body deflection missions presupposes some knowledge of the target body. The current architectural paradigm for sample-return missions is centered around a design where spacecraft and sampling device are merged into a single, complex system. We argue that this monolithic approach couples the navigation and sample-collection problems, making both more difficult. In contrast, we propose a decoupled system based on the coordinated interaction between a spacecraft and a collective of small, simple devices - the Regolith Biters (RBs). A spacecraft carrying a number of RBs would travel to the vicinity of a small body. From a favorable vantage point, and while remaining within a safe distance in a non-colliding trajectory, it would release the RBs towards the target body. Upon encountering the body, they would bite the regolith (thus retaining a sample), and eject back to orbit. The spacecraft, being endowed with appropriate navigation and tracking capabilities, would rendezvous with and collect those RBs within its reach, and bring them back to Earth. Separating the navigation and sampling concerns removes the need for proximity operations with the small body-the stage in current architectures that carries the most challenges and risks. Eliminating the need for proximity operations brings back to the discussion the exploration of exciting prospects, like highly active comets, fast-rotating bodies, and binary systems. Distributing the sampling problem among a collective of agents provides the opportunity to sample multiple regions in a single mission. It also provides robustness to various environmental conditions, and may enable the distributed, in situ characterization of the body. In the search for reliability, current architectures rely on complexity: an elaborate system should succeed at once. We rely on numbers: a given agent may fail at any stage, but success is attained by the collective.

Asteroids↗

NASA NTRS2003

Mobile Agents: A Distributed Voice-Commanded Sensory and Robotic System for Surface EVA Assistance

A model-based, distributed architecture integrates diverse components in a system designed for lunar and planetary surface operations: spacesuit biosensors, cameras, GPS, and a robotic assistant. The system transmits data and assists communication between the extra-vehicular activity (EVA) astronauts, the crew in a local habitat, and a remote mission support team. Software processes ("agents"), implemented in a system called Brahms, run on multiple, mobile platforms, including the spacesuit backpacks, all-terrain vehicles, and robot. These "mobile agents" interpret and transform available data to help people and robotic systems coordinate their actions to make operations more safe and efficient. Different types of agents relate platforms to each other ("proxy agents"), devices to software ("comm agents"), and people to the system ("personal agents"). A state-of-the-art spoken dialogue interface enables people to communicate with their personal agents, supporting a speech-driven navigation and scheduling tool, field observation record, and rover command system. An important aspect of the engineering methodology involves first simulating the entire hardware and software system in Brahms, and then configuring the agents into a runtime system. Design of mobile agent functionality has been based on ethnographic observation of scientists working in Mars analog settings in the High Canadian Arctic on Devon Island and the southeast Utah desert. The Mobile Agents system is developed iteratively in the context of use, with people doing authentic work. This paper provides a brief introduction to the architecture and emphasizes the method of empirical requirements analysis, through which observation, modeling, design, and testing are integrated in simulated EVA operations.

Clancey, William J.↗

DOE OSTI2024

Wholesale Electricity Market Design to Support Resource Adequacy

Wholesale electricity markets are intended to incentivize system generation investments and operations outcomes that meet evolving system needs. In this work, we evaluate the effectiveness of wholesale market structures, rules and policies in achieving system resource adequacy (RA) and clean energy targets in the presence of self-interested generation investors using the Electricity Markets and Investment Suite Agent-based Simulation (EMIS-AS) model. Results highlight that both capacity markets and operating reserve demand curves (ORDCs) can help achieve a reliable system but with different RA compliance timelines and distribution of generation technologies. Structures with capacity markets tend to favor more capital-intensive peaking technologies while reducing wind and solar build-outs due to suppressed energy and clean energy market prices, particularly in the absence of strong clean energy targets. Conversely, ORDCs improve the commitment of available generation units, but this comes at the expense of higher system costs and renewable generation curtailment. We also find that well-calibrated static capacity demand curves can yield similar reliability and total cost compared to capacity market demand curves informed dynamically by resource adequacy while also yielding stable annual capacity prices. Different approaches to formulating ORDC curves can also yield key trade-offs, namely that a more efficient treatment of storage chronology results in lower ORDC curves and prices, yielding less investment and cost but at the expense of reliability. Finally, the effectiveness of wholesale electricity markets in practically achieving very high clean energy generation targets highly depends on the cost-competitiveness of clean energy technologies that can support critical balancing needs across multiple timescales.

agent based modeling↗

DOE OSTI2025

Entanglement engineering of optomechanical systems by reinforcement learning

Entanglement is fundamental to quantum information science and technology, yet controlling and manipulating entanglement—so-called entanglement engineering—for arbitrary quantum systems remains a formidable challenge. There are two difficulties: the fragility of quantum entanglement and its experimental characterization. We develop a model-free deep reinforcement-learning (RL) approach to entanglement engineering, in which feedback control together with weak continuous measurement and partial state observation is exploited to generate and maintain desired entanglement. We employ quantum optomechanical systems with linear or nonlinear photon–phonon interactions to demonstrate the workings of our machine-learning-based entanglement engineering protocol. In particular, the RL agent sequentially interacts with one or multiple parallel quantum optomechanical environments, collects trajectories, and updates the policy to maximize the accumulated reward to create and stabilize quantum entanglement over an arbitrary amount of time. The machine-learning-based model-free control principle is applicable to the entanglement engineering of experimental quantum systems in general.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2005

Quicker Q-Learning in Multi-Agent Systems

Multi-agent learning in Markov Decisions Problems is challenging because of the presence ot two credit assignment problems: 1) How to credit an action taken at time step t for rewards received at t' greater than t; and 2) How to credit an action taken by agent i considering the system reward is a function of the actions of all the agents. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning OK TD(lambda) The second credit assi,onment problem is typically addressed either by hand-crafting reward functions that assign proper credit to an agent, or by making certain independence assumptions about an agent's state-space and reward function. To address both credit assignment problems simultaneously, we propose the Q Updates with Immediate Counterfactual Rewards-learning (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. Instead of assuming that an agent s value function can be made independent of other agents, this method suppresses the impact of other agents using counterfactual rewards. Results on multi-agent grid-world problems over multiple topologies show that QUICR-learning can achieve up to thirty fold improvements in performance over both conventional and local Q-learning in the largest tested systems.

Agogino, Adrian K.↗

NASA NTRS2006

Efficient Agent-Based Cluster Ensembles

Numerous domains ranging from distributed data acquisition to knowledge reuse need to solve the cluster ensemble problem of combining multiple clusterings into a single unified clustering. Unfortunately current non-agent-based cluster combining methods do not work in a distributed environment, are not robust to corrupted clusterings and require centralized access to all original clusterings. Overcoming these issues will allow cluster ensembles to be used in fundamentally distributed and failure-prone domains such as data acquisition from satellite constellations, in addition to domains demanding confidentiality such as combining clusterings of user profiles. This paper proposes an efficient, distributed, agent-based clustering ensemble method that addresses these issues. In this approach each agent is assigned a small subset of the data and votes on which final cluster its data points should belong to. The final clustering is then evaluated by a global utility, computed in a distributed way. This clustering is also evaluated using an agent-specific utility that is shown to be easier for the agents to maximize. Results show that agents using the agent-specific utility can achieve better performance than traditional non-agent based methods and are effective even when up to 50% of the agents fail.

Agogino, Adrian↗

NASA NTRS2005

NESTA: NASA Engineering Shuttle Telemetry Agent

The Spaceport Processing Systems Branch at NASA Kennedy Space Center has developed and deployed an agent based tool to monitor the Space Shuttle's ground processing telemetry stream. The application, the NASA Engineering Shuttle Telemetry Agent, increases situational awareness for system and hardware engineers during ground processing of the Shuttle's subsystems. The agent provides autonomous monitoring of the telemetry stream and automatically alerts system engineers when predefined criteria have been met. Efficiency and safety are improved through increased automation. Sandia National Labs' Java Expert System Shell is employed as the rule engine. The shell's predicate logic lends itself well to capturing the heuristics and specifying the engineering rules of this spaceport domain. The declarative paradigm of the rule-based agent yields a highly modular and scalable design spanning multiple subsystems of the Shuttle. Several hundred monitoring rules have been written thus far with corresponding notifications sent to Shuttle engineers. This paper discusses the rule-based telemetry agent used for Space Shuttle ground processing and explains the problem domain, development of the agent software, benefits of AT technology, and deployment and sustaining engineering of the product.

Semmel, Glenn S.↗

NASA NTRS2005

Monitoring Agents for Assisting NASA Engineers with Shuttle Ground Processing

The Spaceport Processing Systems Branch at NASA Kennedy Space Center has designed, developed, and deployed a rule-based agent to monitor the Space Shuttle's ground processing telemetry stream. The NASA Engineering Shuttle Telemetry Agent increases situational awareness for system and hardware engineers during ground processing of the Shuttle's subsystems. The agent provides autonomous monitoring of the telemetry stream and automatically alerts system engineers when user defined conditions are satisfied. Efficiency and safety are improved through increased automation. Sandia National Labs' Java Expert System Shell is employed as the agent's rule engine. The shell's predicate logic lends itself well to capturing the heuristics and specifying the engineering rules within this domain. The declarative paradigm of the rule-based agent yields a highly modular and scalable design spanning multiple subsystems of the Shuttle. Several hundred monitoring rules have been written thus far with corresponding notifications sent to Shuttle engineers. This chapter discusses the rule-based telemetry agent used for Space Shuttle ground processing. We present the problem domain along with design and development considerations such as information modeling, knowledge capture, and the deployment of the product. We also present ongoing work with other condition monitoring agents.

Semmel, Glenn S.↗

DOE OSTI2026

Scalable Fabrication of a Fibrous Amine-functionalized Matrix (FAM) Sorbent for Critical Mineral Recovery

We report a novel flat sheet Fibrous Amine-functionalized Matrix (FAM) sorbent platform designed for efficient and selective capture of CM from dilute solutions. The FAM sorbent features crosslinked amine microfilms coated onto/within a glass fiber matrix, providing fast mass transfer and excellent mechanical stability. Systematic batch and flow-through tests with FAM revealed rapid metal uptake kinetics and high capacity for representative species, achieving ~90 mg/g of Gallium, ~100 mg/g of Cobalt, and ~90 mg/g for Neodymium. Moreover, multiple eluents, including mineral acids and complexing agents, enabled highly effective desorption of adsorbed metals, demonstrating the feasibility of regenerating FAM sorbents. Importantly, tests with authentic coal ash leachate demonstrated strong selectivity toward U.S. Department of Energy (DOE)-listed CM and rare earth elements over abundant base cations, confirming the robustness of FAM in realistic complex solutions. The flat sheet geometry was amenable to scaling into durable spiral wound modules, highlighting the potential for future regeneration and reuse. This work establishes FAM sorbents as a promising platform for the recovery of CM from wastewaters, advancing both resource sustainability and environmental stewardship.

critical mineral recovery↗

NASA NTRS2001

Human-Vehicle Interface for Semi-Autonomous Operation of Uninhabited Aero Vehicles

The robustness of autonomous robotic systems to unanticipated circumstances is typically insufficient for use in the field. The many skills of human user often fill this gap in robotic capability. To incorporate the human into the system, a useful interaction between man and machine must exist. This interaction should enable useful communication to be exchanged in a natural way between human and robot on a variety of levels. This report describes the current human-robot interaction for the Stanford HUMMINGBIRD autonomous helicopter. In particular, the report discusses the elements of the system that enable multiple levels of communication. An intelligent system agent manages the different inputs given to the helicopter. An advanced user interface gives the user and helicopter a method for exchanging useful information. Using this human-robot interaction, the HUMMINGBIRD has carried out various autonomous search, tracking, and retrieval missions.

Jones, Henry L.↗

NASA NTRS2004

An Agent-Based Interface to Terrestrial Ecological Forecasting

This paper describes a flexible agent-based ecological forecasting system that combines multiple distributed data sources and models to provide near-real-time answers to questions about the state of the Earth system We build on novel techniques in automated constraint-based planning and natural language interfaces to automatically generate data products based on descriptions of the desired data products.

Golden, Keith↗

NASA NTRS2004

Spacecraft Fire Suppression: Testing and Evaluation

The objective of this project is the testing and evaluation of the effectiveness of a variety of fire suppressants and fire-response techniques that will be used in the next generation of spacecraft (Crew Exploration Vehicle, CEV) and planetary habitats. From the many lessons learned in the last 40 years of space travel, there is common agreement in the spacecraft fire safety community that a new fire suppression system will be needed for the various types of fire threats anticipated in new space vehicles and habitats. To date, there is no single fire extinguishing system that can address all possible fire situations in a spacecraft in an effective, reliable, clean, and safe way. The testing conducted under this investigation will not only validate the various numerical models that are currently being developed, but it will provide new design standards on fire suppression that can then be applied to the next generation of spacecraft extinguishment systems. The test program will provide validation of scaling methods by conducting small, medium, and large scale fires. A variety of suppression methods will be tested, such as water mist, carbon dioxide, and nitrogen with single and multiple injection points and direct or distributed agent deployment. These injection methods cover the current ISS fire suppression method of a portable hand-held fire extinguisher spraying through a port in a rack and also next generation spacecraft units that may have a multi-point suppression delivery system built into the design. Consideration will be given to the need of a crew to clean-up the agent and recharge the extinguishers in flight in a long-duration mission. The fire suppression methods mentioned above will be used to extinguish several fire scenarios that have been identified as the most relevant to spaceflight, such as overheated wires, cable bundles, and circuit boards, as well as burning cloth and paper. Further testing will be conducted in which obstructions and ventilation will be added to represent actual spacecraft conditions (e.g., a series of cards in a card rack).

Abbud-Madrid, Angel↗

NASA NTRS2005

Solution Concepts for Distributed Decision-Making without Coordination

Consider a single-stage problem in which we have a group N agents who are attempting to minimize the expected cost of their joint actions, without the benefit of communication or a pre-established protocol but with complete knowledge of the expected cost of any joint set of actions for the group. We call this situation a static coordination problem. The central issue in defining an appropriate solution concept for static coordination problems is considering how to deal with the fact that if the agents axe faced with a set of multiple (mixed) strategies that are equally attractive in terms of cost, a failure of coordination may lead to an expected cost value that is worse than that of any of the strategies in the set. In this proposal, we describe the notion of a general coordination problem, describe initial efforts at developing a solution concept for static coordination problems, and then outline a research agenda that centers on activities that will be basis for obtaining a complete understanding of solutions to static coordination problems.

Beling, Peter A.↗

DOE OSTI2026

Agentic traffic intelligence: Augmented human-in-the-loop scenario generation for microscopic traffic simulation

Traditional microscopic traffic simulation generation often relies on static datasets and manual design, limiting its ability to simulate complex conditions easily. This paper presents a novel framework, Agentic Traffic Intelligence, which combines human approval large language models (LLMs), the Real-Twin tool, and multi-agent systems to perform realistic microscopic traffic simulation scenario generation. The proposed framework incorporates human-in-the-loop (HIL) control, retrieval-augmented generation (RAG), and multi-agent control mechanisms. HIL mechanisms are used to guide multiple LLMs focused on attributes for microscopic simulation generation and to improve the interpretability and transparency of LLM execution for users. RAG enhances context extraction by dynamically integrating external knowledge sources for traffic scenario generation foundations. A multi-agent architecture with supervisory control coordinates the interaction of simulation components, including traffic simulators, control logic, and calibration tools. This enables the synthesis of simulation-ready scenarios that reflect dynamic demand profiles and behavior controls. Furthermore, the framework fuses multisource traffic data with unstructured context and supports iterative refinement through interactive user feedback. Validated through microscopic simulation using Simulation of Urban Mobility, the generated scenarios demonstrate high-fidelity network generation with inflow and turn movement and behavioral calibration, offering a robust and efficient tool for stress-testing and optimizing urban mobility systems.

Hierarchical multi-agent control↗

NASA NTRS2009

Complexity Science Applications to Dynamic Trajectory Management: Research Strategies

The promise of the Next Generation Air Transportation System (NextGen) is strongly tied to the concept of trajectory-based operations in the national airspace system. Existing efforts to develop trajectory management concepts are largely focused on individual trajectories, optimized independently, then de-conflicted among each other, and individually re-optimized, as possible. The benefits in capacity, fuel, and time are valuable, though perhaps could be greater through alternative strategies. The concept of agent-based trajectories offers a strategy for automation of simultaneous multiple trajectory management. The anticipated result of the strategy would be dynamic management of multiple trajectories with interacting and interdependent outcomes that satisfy multiple, conflicting constraints. These constraints would include the business case for operators, the capacity case for the Air Navigation Service Provider (ANSP), and the environmental case for noise and emissions. The benefits in capacity, fuel, and time might be improved over those possible under individual trajectory management approaches. The proposed approach relies on computational agent-based modeling (ABM), combinatorial mathematics, as well as application of "traffic physics" concepts to the challenge, and modeling and simulation capabilities. The proposed strategy could support transforming air traffic control from managing individual aircraft behaviors to managing systemic behavior of air traffic in the NAS. A system built on the approach could provide the ability to know when regions of airspace approach being "full," that is, having non-viable local solution space for optimizing trajectories in advance.

Sawhill, Bruce↗