Search NASA⌕ Search

SEARCH · Search NASA

Results for “multiagent systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

46 records · Page 3

A Cooperative Multi-Agent Deep Reinforcement Learning Framework for Real-Time Residential Load Scheduling

Internet-of-Things (IoT) enabled monitoring and control capabilities are enabling increasing numbers of household users with controllable loads to actively participate in smart grid energy management. Realizing an efficient real-time energy management system that takes advantage of these developments requires novel techniques for managing the increased complexity of the control action space in resolving multiple challenges such as the uncertainty in energy prices and renewable energy output along with the need to satisy physical grid constraints such as transformer capacity. Addressing these challenges, we develop a multi-household energy management framework for residential units connected to the same transformer and containing DERs such as PV, ESS and controllable loads. The goal of our framework is to schedule controllable household appliances and ESS such that the cost of procuring electricity from the utility over a horizon is minimized while physical grid constraints are satisfied at each scheduling step. Traditional energy management frameworks either perform global optimization to satisfy grid constraints but suffer from high computational complexity (for example Integer Program, Mixed Integer Programming frameworks and centralized reinforcement learning) or perform decentralized real-time energy management without satisfying global grid constraints (for example multi-agent reinforcement learning with no cooperation). In contrast, we propose a cooperative multiagent reinforcement learning (MARL) framework that i) operates in real-time, and ii) performs explicit collaboration to satisfy global grid constraints. The novelty in our framework is two fold. Firstly, our framework trains multiple independent learners (IL) for each household in parallel using historical data and performs real-time inferencing of control actions using the most recent system state. Secondly, our framework contains a low complexity knapsack based cooperation agent which combines the outputs of ILs to minimize cost while satisfying grid constraints. Simulation results show that our cooperative MARL approach achieves significant cost improvement over centralized reinforcement learning and day-ahead planning baselines. Moreover, our approach strictly satisfies physical constraints with no apriori knowledge of system dynamics while the baseline approaches have occasional violations. We also measure the training and inference time by ranging the number of households from 1 to 25. Results show that our cooperative MARL approach scales best among various approaches.

14 SOLAR ENERGY↗

Understanding and Predictability of Integrated Mountain Hydroclimate (Workshop Report)

Mountainous systems cover approximately 23% of Earth’s land and are distributed across all continents. They can capture and store atmospheric moisture that is then cycled through the terrestrial surface and subsurface system, released to downstream communities, and cycled back to the atmosphere. Mountain hydroclimate—characterized by steep gradients, geological, ecological, and biogeochemical diversity—is influenced by topographic forcing and elevated warming and susceptible to large subseasonal to multidecadal variability and rapid changes. Terrestrial hydrological and biogeochemical cycles also experience cascading effects from global warming impacts, such as multidecadal declines in mountain snowpack, longer growing seasons, and increased frequency and severity of extreme events like droughts and wildfires. However, little is known about the effects of these impacts and their feedbacks on climate systems and surface-subsurface compartments. Also unknown are the full implications of changing hydroclimate and extreme events on hydro biogeochemical cycles across atmosphere, terrestrial, and human systems in mountain regions and beyond. This knowledge gap is critical, given human reliance on mountain systems for stable water supply and quality. Mountain systems’ increasing vulnerability to climate change and human perturbations motivates the need to improve understanding of integrated mountain hydroclimate (IMHC) systems and their feedbacks and impacts on humans across scales. However, due to large heterogeneity and strong gradients, coupled natural-human processes in mountain regions present significant challenges for observations, modeling, predictions, and projections. Motivated by gaps in mountain hydroclimate understanding, observations, and modeling and the need for credible projections of future changes, the U.S. Department of Energy’s (DOE) Biological and Environmental Research (BER) program organized a virtual workshop on “Understanding and Predictability of Integrated Mountain Hydroclimate.” Sponsored by BER’s Earth and Environmental Systems Sciences Division (EESSD), the workshop aimed to inform and catalyze EESSD’s growing interests in enhancing predictive understanding of IMHC. Organizers structured the workshop to identify (1) knowledge gaps, (2) observational and modeling challenges, (3) short-term (1 to 3 years) and long-term (10 years and beyond) research opportunities, and (4) strategies for fostering collaboration and coordination. To address the outstanding challenges of IMHC, the workshop included two sessions organized by disciplinary, cross-disciplinary, and crosscutting science topics. The disciplinary and cross-disciplinary topics focused on essential IMHC elements: atmosphere, terrestrial, and human systems and their interactions. Breakout sessions on disciplinary and cross-disciplinary topics facilitated identification of crosscutting topics and central emerging themes. Session 1 focused on connecting existing DOE investments to accelerate progress related to scientific challenges in understanding mountain hydroclimate. In Session 2, participants further explored key Session 1 takeaways through the lens of multiagency collaborations and coordination.

54 ENVIRONMENTAL SCIENCES↗

Multirobot Collaborative Pursuit Target Robot by Improved MADDPG

Policy formulation is one of the main problems in multirobot systems, especially in multirobot pursuit-evasion scenarios, where both sparse rewards and random environment changes bring great difficulties to find better strategy. Existing multirobot decision-making methods mostly use environmental rewards to promote robots to complete the target task that cannot achieve good results. This paper proposes a multirobot pursuit method based on improved multiagent deep deterministic policy gradient (MADDPG), which solves the problem of sparse rewards in multirobot pursuit-evasion scenarios by combining the intrinsic reward and the external environment. The state similarity module based on the threshold constraint is as a part of the intrinsic reward signal output by the intrinsic curiosity module, which is used to balance overexploration and insufficient exploration, so that the agent can use the intrinsic reward more effectively to learn better strategies. The simulation experiment results show that the proposed method can improve the reward value of robots and the success rate of the pursuit task significantly. The intuitive change is obviously reflected in the real-time distance between the pursuer and the escapee, the pursuer using the improved algorithm for training can get closer to the escapee more quickly, and the average following distance also decreases.

Zhou, Xiao↗

Extreme-scale EV charging infrastructure planning for last-mile delivery using high-performance parallel computing

Here, this paper addresses stochastic charger location and allocation problems under queue congestion for last-mile delivery using electric vehicles (EVs). The objective is to decide where to open charging stations and how many chargers of each type to install, subject to budgetary and waiting-time constraints. We formulate the problem as a mixed-integer non-linear program, where each station-charger pair is modeled as a multiserver queue with stochastic arrivals and service times to capture the notion of waiting in fleet operations. The model is extremely large, with billions of variables and constraints for a typical metropolitan area; even loading the model in solver memory is difficult, let alone solving it. To address this challenge, we develop a Lagrangian-based dual decomposition framework that decomposes the problem by station and leverages parallelization on high-performance computing systems, where the subproblems are solved by using a cutting plane method and their solutions are collected at the master level. We also develop a three-step rounding heuristic to transform the fractional subproblem solutions into feasible integral solutions. Computational experiments on data from the Chicago metropolitan area with hundreds of thousands of households and thousands of candidate stations show that our approach produces high-quality solutions in cases where existing exact methods cannot even load the model in memory. We also analyze various policy scenarios, demonstrating that combining existing depots with newly built stations under multiagency collaboration substantially reduces costs and congestion. These findings offer a scalable and efficient framework for developing sustainable large-scale EV charging networks.

Capacity allocation↗

Controlled environment agriculture: An opportunity to strengthen interagency research collaboration in the US government

Challenges facing food production and agricultural systems are increasingly interconnected with economic, security, health, and equity issues, among others. Threats such as extreme weather, economic volatility, and shrinking water resources and arable land, influence our ability to maintain a safe and resilient food supply. One promising solution to these threats is controlled environment agriculture (CEA). In many cases, CEA can drastically reduce the amount of water and land used in crop production while increasing productivity. Operations may be established in nearly any environment and harvests can take place year-round, supporting food system resiliency and sustainability. CEA sits at the nexus of a number of disciplines and industries, making it well suited for transdisciplinary and multi-institutional research coordination. Herein, authors from multiple US government agencies present CEA as a case study in improving cross-agency research collaboration. The federal government houses a range of scientific expertise and research capabilities, positioning scientists to lead national and global efforts in transdisciplinary, interagency approaches to complex challenges. Navigating cross-agency collaboration can be a challenge, especially coordinating across different scientific disciplines, geographic locations, and funding mechanisms. To enhance multiagency efforts, collaborators could prioritize (i) organizing personnel and resources, (ii) enhancing existing multiagency collaborations, and (iii) focusing on further opportunities for coordination. Adopting these approaches could enable federal researchers to reinforce and advance academic and industry efforts to address current CEA challenges while solidifying the United States as a leader in this arena.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Joint Planning of EV Fast Charging Stations and Power Distribution Systems With Balanced Traffic Flow Assignment

To tackle the challenges introduced by the fast-growing charging demand of electric vehicles (EVs), the power distribution systems (PDSs) and fast charging stations (FCSs) of EVs should be planned and operated in a more coordinated fashion. However, existing planning approaches generally aim to minimize investment costs in PDSs while ignoring the risk of worsening traffic conditions. To overcome this research gap, this article integrates the interests of traffic networks into PDS and FCS joint planning model to mitigate negative impacts on traffic conditions caused by installing FCSs. First, a novel microscopic method that is different from traditional assignment methods is proposed to simulate the influences of FCSs on traffic flows and EV charging loads. Then, a multiobjective joint planning model is developed to minimize both the planning costs and unbalanced traffic flows. A new bilayer Benders decomposition algorithm is designed to solve the proposed joint planning model. Numerical results on two practical systems in China validate the feasibility of our microscopic method by comparing the simulated results with real data. Compared with existing approaches, it is also demonstrated that the proposed joint planning approach helps to balance traffic flow assignments and relieve traffic congestion.

bilayer expanded Benders decomposition↗

PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems [SWR-22-07]

NREL's PowerGridworld provides a modular simulation environment for training heterogenous, grid-aware, multi-agent reinforcement learning (RL) policies at scale. The package enables the user to create component gym environments that can be composed into more complex agents. For example, a grid interactive building environment can be created by composing together component environments each encapsulating the building, PV, and battery physics. These multi-component environments can then be combined into multi-agent simulation where each agent's power consumption/injection becomes an input for solving the optimal power flow on a distribution feeder modeled in OpenDSS. Information from OpenDSS, such as bus voltages and line flows, can be included in the agents' observation spaces to enable grid-aware rewards. The default API for the PowerGridworld simulator conforms to RLLib's MultiAgent API and thus enables distributed training using HPC and cloud resources.

Biagioni, David↗

Directing Nanoparticle Organization in Response to Diverse Chemical Inputs

Signaling cascades are crucial for transducing stimuli in biological systems, enabling multiple stimuli to regulate a downstream target with precisely controlled timing and amplifying signals through a series of intermediary reactions. Developing a robust signaling system with such capabilities would be pivotal for programming complex behaviors in synthetic DNA-based molecular devices. However, although “software” such as nucleic acid circuits could potentially be harnessed to relay signals to DNA-based nanostructure hardware, such explorations have been limited. Here, in this study, we develop a platform for transducing a variety of stimuli via messenger-mediated reactions to regulate the release and reloading of gold nanoparticles (AuNPs) in a 3D DNA framework. In the first step, an in vitro transcription circuit is engineered to sense and amplify chemical stimuli, including arbitrary DNA sequences and proteins, producing RNA. In the second step, the RNA releases the DNA-coated AuNPs from the DNA framework via a strand displacement reaction. AuNP reloading is controlled by a separate step driven by degradation of the RNA. Our platform holds promise for applications requiring dynamic multiagent control over DNA-based devices, offering a versatile tool for advanced molecular device engineering.

36 MATERIALS SCIENCE↗

%22Trojan Horse%22 strategy for deconstruction of biomass for biofuels production.

Production of renewable biofuels to displace fossil fuels currently consumed in the transportation sector is a pressing multiagency national priority (DOE/USDA/EERE). Currently, nearly all fuel ethanol is produced from corn-derived starch. Dedicated 'energy crops' and agricultural waste are preferred long-term solutions for renewable, cheap, and globally available biofuels as they avoid some of the market pressures and secondary greenhouse gas emission challenges currently facing corn ethanol. These sources of lignocellulosic biomass are converted to fermentable sugars using a variety of chemical and thermochemical pretreatments, which disrupt cellulose and lignin cross-links, allowing exogenously added recombinant microbial enzymes to more efficiently hydrolyze the cellulose for 'deconstruction' into glucose. This process is plagued with inefficiencies, primarily due to the recalcitrance of cellulosic biomass, mass transfer issues during deconstruction, and low activity of recombinant deconstruction enzymes. Costs are also high due to the requirement for enzymes and reagents, and energy-intensive cumbersome pretreatment steps. One potential solution to these problems is found in synthetic biology-engineered plants that self-produce a suite of cellulase enzymes. Deconstruction can then be integrated into a one-step process, thereby increasing efficiency (cellulose-cellulase mass-transfer rates) and reducing costs. The unique aspects of our approach are the rationally engineered enzymes which become Trojan horses during pretreatment conditions. During this study we rationally engineered Cazy enzymes and then integrated them into plant cells by multiple transformation techniques. The regenerated plants were assayed for first expression of these messages and then for the resulting proteins. The plants were then subjected to consolidated bioprocessing and characterized in detail. Our results and possible implications of this work on developing dedicated energy crops and their advantage in a consolidated bioprocessing system.

09 BIOMASS FUELS↗