Search NASA⌕ Search

SEARCH · Search NASA

Results for “multiagent systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Decentralized Voltage Control of Large-Scale Distribution System with PVs Based on MADRL

This paper proposes a model-free decentralized control framework for the voltage regulation of large-scale distribution systems through the coordinated control of PV inverters. This is achieved by developing a novel interaction mechanism between the surrogate model and the centralized training and decentralized execution multiagent deep reinforcement learning framework. Specifically, the sparse Gaussian processes regression method is first utilized to develop the surrogate model of the original distribution system for reward calculation during the training stage, where each agent represents a sub-region in the centralized fashion for coordination strategy learning. After that, the learned control rules are used to inform controllers within each sub-region for real-time decisions with only local measurements. Comparative tests among various methods on the EPRI Ckt5 test system demonstrate the effectiveness of the proposed method.

distribution system↗
NASA NTRS1995

Ground-based grasslands data to support remote sensing and ecosystem modeling of terrestrial primary production

Estimating terrestrial net primary production (NPP) using remote-sensing tools and ecosystem models requires adequate ground-based measurements for calibration, parameterization, and validation. These data needs were strongly endorsed at a recent meeting of ecosystem modelers organized by the International Geosphere-Biosphere Program's (IGBP's) Data and Information System (DIS) and its Global Analysis, Interpretation, and Modelling (GAIM) Task Force. To meet these needs, a multinational, multiagency project is being coordinated by the IGBP DIS to compile existing NPP data from field sites and to regionalize NPP point estimates to various-sized grid cells. Progress at Oak Ridge National Laboratory (ORNL) on compiling NPP data for grasslands as part of the IGBP DIS data initiative is described. Site data and associated documentation from diverse field studies are being acquired for selected grasslands and are being reviewed for completeness, consistency, and adequacy of documentation, including a description of sampling methods. Data are being compiled in a database with spatial, temporal, and thematic characteristics relevant to remote sensing and global modeling. NPP data are available from the ORNL Distributed Active Archive Center (DAAC) for biogeochemical dynamics. The ORNL DAAC is part of the Earth Observing System Data and Information System, of the US National Aeronautics and Space Administration.

Olson, R. J.↗
NASA NTRS1994

The 1995 NASA guide to graduate support

The future of the United States is in the classrooms of America and tomorrow's scientific and technological capabilities are derived from today's investments in research. In 1980, NASA initiated the Graduate Student Researchers Program (GSRP) to cultivate additional research ties to the academic community and to support promising students pursuing advanced degrees in science and engineering. Since then, approximately 1300 students have completed the program's requirements. In 1987, the program was expanded to include the Underrepresented Minority and Disabled Focus (UMDF) Component. This program was designed to increase participation of underrepresented groups in graduate study and research and, ultimately, in space science and aerospace technology careers. Approximately 270 minority students have completed the program's requirements while making significant contributions to the nation's aerospace efforts. Continuing to expand fellowship opportunities, NASA announced the Graduate Student Fellowships in Global Change Research in 1990. Designed to support the rapid growth in the study of earth as a system, more than 250 fellowships have been awarded. And, in 1992, NASA announced opportunities in the multiagency High Performance Computing and Communications (HPCC) Program designed to accelerate the development and application of massively parallel processing. Approximately five new fellowships will be awarded yearly. This booklet will guide you in your efforts to participate in programs for graduate student support.

Source record↗
DOE OSTI2023

Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning

Constrained multiagent reinforcement learning (C-MARL) is gaining importance as MARL algorithms find new applications in real-world systems ranging from energy systems to drone swarms. Most C-MARL algorithms use a primal-dual approach to enforce constraints through a penalty function added to the reward. In this paper, we study the structural effects of this penalty term on the MARL problem. First, we show that the standard practice of using the constraint function as the penalty leads to a weak notion of safety. However, by making simple modifications to the penalty term, we can enforce meaningful probabilistic (chance and conditional value at risk) constraints. Second, we quantify the effect of the penalty term on the value function, uncovering an improved value estimation procedure. We use these insights to propose a constrained multiagent advantage actor critic (C-MAA2C) algorithm. Simulations in a simple constrained multiagent environment affirm that our reinterpretation of the primal-dual method in terms of probabilistic constraints is effective, and that our proposed value estimate accelerates convergence to a safe joint policy.

chance constraints↗
DOE OSTI2021

A coupled human–natural system analysis of freshwater security under climate and population change

Limited water availability, population growth, and climate change have resulted in freshwater crises in many countries. Jordan’s situation is emblematic, compounded by conflict-induced population shocks. Integrating knowledge across hydrology, climatology, agriculture, political science, geography, and economics, we present the Jordan Water Model, a nationwide coupled human–natural-engineered systems model that is used to evaluate Jordan’s freshwater security under climate and socioeconomic changes. The complex systems model simulates the trajectory of Jordan’s water system, representing dynamic interactions between a hierarchy of actors and the natural and engineered water environment. A multiagent modeling approach enables the quantification of impacts at the level of thousands of representative agents across sectors, allowing for the evaluation of both systemwide and distributional outcomes translated into a suite of water-security metrics (vulnerability, equity, shortage duration, and economic well-being). Model results indicate severe, potentially destabilizing, declines in freshwater security. Per capita water availability decreases by approximately 50% by the end of the century. Without intervening measures, >90% of the low-income household population experiences critical insecurity by the end of the century, receiving <40 L per capita per day. Widening disparity in freshwater use, lengthening shortage durations, and declining economic welfare are prevalent across narratives. To gain a foothold on its freshwater future, Jordan must enact a sweeping portfolio of ambitious interventions that include large-scale desalinization and comprehensive water sector reform, with model results revealing exponential improvements in water security through the coordination of supply- and demand-side measures.

42 ENGINEERING↗
DOE OSTI2023

Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning: Preprint

We study multiagent reinforcement learning (MARL) with constraints. This setting is gaining importance as MARL algorithms find new applications in real-world systems ranging from power grids to drone swarms. Most constrained MARL (C-MARL) algorithms use a primal-dual approach to enforce constraints through a penalty function added to the reward. In this paper, we study the structural effects of the primal-dual approach on the constraints and value function. First, we show that using the constraint evaluation as the penalty leads to a weak notion of safety, but by making simple modifications to the penalty function, we can enforce meaningful probabilistic safety constraints. Second, we show that the penalty term changes the value function in a way that is easy to model, and demonstrate the consequences of not doing so. We conclude with simulations in a simple constrained multiagent environment to back up the theoretical results.

data-driven control↗
NASA NTRS2004

A Multiagent Modeling Environment for Simulating Work Practice in Organizations

In this paper we position Brahms as a tool for simulating organizational processes. Brahms is a modeling and simulation environment for analyzing human work practice, and for using such models to develop intelligent software agents to support the work practice in organizations. Brahms is the result of more than ten years of research at the Institute for Research on Learning (IRL), NYNEX Science & Technology (the former R&D institute of the Baby Bell telephone company in New York, now Verizon), and for the last six years at NASA Ames Research Center, in the Work Systems Design and Evaluation group, part of the Computational Sciences Division (Code IC). Brahms has been used on more than ten modeling and simulation research projects, and recently has been used as a distributed multiagent development environment for developing work practice support tools for human in-situ science exploration on planetary surfaces, in particular a human mission to Mars. Brahms was originally conceived of as a business process modeling and simulation tool that incorporates the social systems of work, by illuminating how formal process flow descriptions relate to people s actual located activities in the workplace. Our research started in the early nineties as a reaction to experiences with work process modeling and simulation . Although an effective tool for convincing management of the potential cost-savings of the newly designed work processes, the modeling and simulation environment was only able to describe work as a normative workflow. However, the social systems, uncovered in work practices studied by the design team played a significant role in how work actually got done-actual lived work. Multi- tasking, informal assistance and circumstantial work interactions could not easily be represented in a tool with a strict workflow modeling paradigm. In response, we began to develop a tool that would have the benefits of work process modeling and simulation, but be distinctively able to represent the relations of people, locations, systems, artifacts, communication and information content.

Sierhuis, Maarten↗
DOE OSTI2024

Quantum for Energy Systems and Technologies

Quantum Information Science (QIS) is expected to profoundly change the practice of science and engineering in the coming decades. QIS technology exploits quantum phenomena for performing tasks that are impossible to do today and is a rapidly progressing field, fueled by large investments from the private sector and governments. Its importance to the U.S. economy and national security is underscored by the National Quantum Initiative Act (NQIA) passed in December 2018, which creates a coordinated multiagency program to support research and training in QIS. After the NIQA signed into law, NETL has launched an initiative to apply QIS to problems encountered in energy technology development. In Jan. 2020, an Quantum for Energy Systems & Technologies (QUEST) working group was formed to establish a workforce capable of developing, reviewing, managing, and advising on QIS-related technologies for NETL and FECM. Since then, the QUEST team have been working on developing quantum sensing technology and performing quantum computing to solve energy-related problems. This poster summarized the QUEST team’s activities & accomplishments on QIS targeting energy-related applications.

Paudel, Hari P.↗
NASA NTRS2005

Distributed Evaluation Functions for Fault Tolerant Multi-Rover Systems

The ability to evolve fault tolerant control strategies for large collections of agents is critical to the successful application of evolutionary strategies to domains where failures are common. Furthermore, while evolutionary algorithms have been highly successful in discovering single-agent control strategies, extending such algorithms to multiagent domains has proven to be difficult. In this paper we present a method for shaping evaluation functions for agents that provide control strategies that both are tolerant to different types of failures and lead to coordinated behavior in a multi-agent setting. This method neither relies of a centralized strategy (susceptible to single point of failures) nor a distributed strategy where each agent uses a system wide evaluation function (severe credit assignment problem). In a multi-rover problem, we show that agents using our agent-specific evaluation perform up to 500% better than agents using the system evaluation. In addition we show that agents are still able to maintain a high level of performance when up to 60% of the agents fail due to actuator, communication or controller faults.

Agogino, Adrian↗
DOE OSTI2021

A Modular and Transferable Reinforcement Learning Framework for the Fleet Rebalancing Problem

Mobility on demand (MoD) systems show great promise in realizing flexible and efficient urban transportation. However, significant technical challenges arise from operational decision making associated with MoD vehicle dispatch and fleet rebalancing. For this reason, operators tend to employ simplified algorithms that have been demonstrated to work well in a particular setting. To help bridge the gap between novel and existing methods, we propose a modular framework for fleet rebalancing based on model-free reinforcement learning (RL) that can leverage an existing dispatch method to minimize system cost. In particular, by treating dispatch as part of the environment dynamics, a centralized agent can learn to intermittently direct the dispatcher to reposition free vehicles and mitigate against fleet imbalance. We formulate RL state and action spaces as distributions over a grid partitioning of the operating area, making the framework scalable and avoiding the complexities associated with multiagent RL. Numerical experiments, using real-world trip and network data, demonstrate that RL reduces waiting time by 28% to 38% for the same-day evaluation, 17% to 44% for cross-day evaluation, and 22% to 25% for cross-season evaluation compared with no rebalancing scenarios. This approach has several distinct advantages over baseline methods including: improved system cost; high degree of adaptability to the selected dispatch method; and the ability to perform scale-invariant transfer learning between problem instances with similar vehicle and request distributions.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS1992

Nuclear Electric Propulsion Technology Panel findings and recommendations

Summarized are the findings and recommendations of a triagency (NASA/DOE/DOD) panel on Nuclear Electric Propulsion (NEP) Technology. NEP has been identified as a candidate nuclear propulsion technology for exploration of the Moon and Mars as part of the Space Exploration Initiative (SEI). The findings are stated in areas of system and subsystem considerations, technology readiness, and ground test facilities. Recommendations made by the panel are summarized concerning: (1) existing space nuclear power and propulsion programs, and (2) the proposed multiagency NEP technology development program.

Doherty, Michael P.↗
DOE OSTI2019

A Cooperative Multi-Agent Deep Reinforcement Learning Framework for Real-Time Residential Load Scheduling

Internet-of-Things (IoT) enabled monitoring and control capabilities are enabling increasing numbers of household users with controllable loads to actively participate in smart grid energy management. Realizing an efficient real-time energy management system that takes advantage of these developments requires novel techniques for managing the increased complexity of the control action space in resolving multiple challenges such as the uncertainty in energy prices and renewable energy output along with the need to satisy physical grid constraints such as transformer capacity. Addressing these challenges, we develop a multi-household energy management framework for residential units connected to the same transformer and containing DERs such as PV, ESS and controllable loads. The goal of our framework is to schedule controllable household appliances and ESS such that the cost of procuring electricity from the utility over a horizon is minimized while physical grid constraints are satisfied at each scheduling step. Traditional energy management frameworks either perform global optimization to satisfy grid constraints but suffer from high computational complexity (for example Integer Program, Mixed Integer Programming frameworks and centralized reinforcement learning) or perform decentralized real-time energy management without satisfying global grid constraints (for example multi-agent reinforcement learning with no cooperation). In contrast, we propose a cooperative multiagent reinforcement learning (MARL) framework that i) operates in real-time, and ii) performs explicit collaboration to satisfy global grid constraints. The novelty in our framework is two fold. Firstly, our framework trains multiple independent learners (IL) for each household in parallel using historical data and performs real-time inferencing of control actions using the most recent system state. Secondly, our framework contains a low complexity knapsack based cooperation agent which combines the outputs of ILs to minimize cost while satisfying grid constraints. Simulation results show that our cooperative MARL approach achieves significant cost improvement over centralized reinforcement learning and day-ahead planning baselines. Moreover, our approach strictly satisfies physical constraints with no apriori knowledge of system dynamics while the baseline approaches have occasional violations. We also measure the training and inference time by ranging the number of households from 1 to 25. Results show that our cooperative MARL approach scales best among various approaches.

14 SOLAR ENERGY↗