Search NASA⌕ Search

SEARCH · Search NASA

Results for “learning-based control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Learning-based position control of a closed-kinematic chain robot end-effector

A trajectory control scheme whose design is based on learning theory, for a six-degree-of-freedom (DOF) robot end-effector built to study robotic assembly of NASA hardwares in space is presented. The control scheme consists of two control systems: the feedback control system and the learning control system. The feedback control system is designed using the concept of linearization about a selected operating point, and the method of pole placement so that the closed-loop linearized system is stabilized. The learning control scheme consisting of PD-type learning controllers, provides additional inputs to improve the end-effector performance after each trial. Experimental studies performed on a 2 DOF end-effector built at CUA, for three tracking cases show that actual trajectories approach desired trajectories as the number of trials increases. The tracking errors are substantially reduced after only five trials.

Nguyen, Charles C.↗

Upper Torso Control for HOAP-2 Using Neural Networks

Humanoid robots have similar physical builds and motion patterns as humans. Not only does this provide a suitable operating environment for the humanoid but it also opens up many research doors on how humans function. The overall objective is replacing humans operating in unsafe environments. A first target application is assembly of structures for future lunar-planetary bases. The initial development platform is a Fujitsu HOAP-2 humanoid robot. The goal for the project is to demonstrate the capability of a HOAP-2 to autonomously construct a cubic frame using provided tubes and joints. This task will require the robot to identify several items, pick them up, transport them to the build location, then properly assemble the structure. The ability to grasp and assemble the pieces will require improved motor control and the addition of tactile feedback sensors. In recent years, learning-based control is becoming more and more popular; for implementing this method we will be using the Adaptive Neural Fuzzy Inference System (ANFIS). When using neural networks for control, no complex models of the system must be constructed in advance-only input/output relationships are required to model the system.

Sandoval, Steven P.↗

A reinforcement learning-based architecture for fuzzy logic control

This paper introduces a new method for learning to refine a rule-based fuzzy logic controller. A reinforcement learning technique is used in conjunction with a multilayer neural network model of a fuzzy controller. The approximate reasoning based intelligent control (ARIC) architecture proposed here learns by updating its prediction of the physical system's behavior and fine tunes a control knowledge base. Its theory is related to Sutton's temporal difference (TD) method. Because ARIC has the advantage of using the control knowledge of an experienced operator and fine tuning it through the process of learning, it learns faster than systems that train networks from scratch. The approach is applied to a cart-pole balancing system.

Berenji, Hamid R.↗

Learning-Based State-Dependent Coefficient Form Task Space Tracking Control of Soft Robot

n this paper, a data-driven modeling and control framework is developed for task space control of a soft robot gripper which consists of four individual soft fingers. Each of the four fingers is modeled as a manipulator with high degrees of freedom. The corresponding task space dynamics of the manipulator are derived using a rigid-link approximation of the continuum manipulator. A neural network approach is used to learn the derived dynamics in State Dependent Coefficient (SDC) form. Using the learned SDC matrices, an asymptotically stable optimal closed-loop tracking controller which is based on solving the State Dependent Riccati Equation (SDRE) is derived. The model learning and trajectory tracking controller is implemented on an open source Soft Motion (SoMo) platform simulating the soft gripper motion and corresponding tracking results are presented.

Rounak Bhattacharya↗

Multi-Objective Reinforcement Learning-Based Deep Neural Networks for Cognitive Space Communications

Future communication subsystems of space exploration missions can potentially benefit from software-defined radios (SDRs) controlled by machine learning algorithms. In this paper, we propose a novel hybrid radio resource allocation management control algorithm that integrates multi-objective reinforcement learning and deep artificial neural networks. The objective is to efficiently manage communications system resources by monitoring performance functions with common dependent variables that result in conflicting goals. The uncertainty in the performance of thousands of different possible combinations of radio parameters makes the trade-off between exploration and exploitation in reinforcement learning (RL) much more challenging for future critical space-based missions. Thus, the system should spend as little time as possible on exploring actions, and whenever it explores an action, it should perform at acceptable levels most of the time. The proposed approach enables on-line learning by interactions with the environment and restricts poor resource allocation performance through virtual environment exploration. Improvements in the multiobjective performance can be achieved via transmitter parameter adaptation on a packet-basis, with poorly predicted performance promptly resulting in rejected decisions. Simulations presented in this work considered the DVB-S2 standard adaptive transmitter parameters and additional ones expected to be present in future adaptive radio systems. Performance results are provided by analysis of the proposed hybrid algorithm when operating across a satellite communication channel from Earth to GEO orbit during clear sky conditions. The proposed approach constitutes part of the core cognitive engine proof-of-concept to be delivered to the NASA Glenn Research Center SCaN Testbed located onboard the International Space Station.

space archtiecture↗

Multi-Objective Reinforcement Learning-based Deep Neural Networks for Cognitive Space Communications

Future communication subsystems of space exploration missions can potentially benefit from software-defined radios (SDRs) controlled by machine learning algorithms. In this paper, we propose a novel hybrid radio resource allocation management control algorithm that integrates multi-objective reinforcement learning and deep artificial neural networks. The objective is to efficiently manage communications system resources by monitoring performance functions with common dependent variables that result in conflicting goals. The uncertainty in the performance of thousands of different possible combinations of radio parameters makes the trade-off between exploration and exploitation in reinforcement learning (RL) much more challenging for future critical space-based missions. Thus, the system should spend as little time as possible on exploring actions, and whenever it explores an action, it should perform at acceptable levels most of the time. The proposed approach enables on-line learning by interactions with the environment and restricts poor resource allocation performance through virtual environment exploration. Improvements in the multiobjective performance can be achieved via transmitter parameter adaptation on a packet-basis, with poorly predicted performance promptly resulting in rejected decisions. Simulations presented in this work considered the DVB-S2 standard adaptive transmitter parameters and additional ones expected to be present in future adaptive radio systems. Performance results are provided by analysis of the proposed hybrid algorithm when operating across a satellite communication channel from Earth to GEO orbit during clear sky conditions. The proposed approach constitutes part of the core cognitive engine proof-of-concept to be delivered to the NASA Glenn Research Center SCaN Testbed located onboard the International Space Station.

space archtiecture↗

Learning Based Edge Computing in Air-to-Air Communication Network

This paper studies learning-based edge computing and communication in a dynamic Air-to-Air Ad-hoc Network (AAAN). Due to spectrum scarcity, we assume the number of Air-to-Air (A2A) communication links is greater than that of the available frequency channels, such that some communication links have to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the aggregate spectrum utilization efficiency under resource and fairness constraints. A distributed deep Q learning-based edge computing and communication algorithm is proposed to find the optimal solution. In particular, we design two different neural network structures and each communication link can converge to the optimal operation by exploiting only the local information from its neighbors, making it scalable to large networks. Finally, experimental results demonstrate the effectiveness of the proposed solution in various AAAN scenarios.

Zhe Wang↗

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗

Predictive Model for Workload in Remote Operators During sUAS Contingency Scenarios

The increase in automated capabilities of small Uncrewed Aerial Systems (sUAS) has enabled the human operators to manage larger numbers of vehicles simultaneously. As this happens, the operational paradigm shifts to an m:N configuration where multiple operators (m) are managing multiple vehicles (N) together. However, many questions about how operators will interact with each other and share interaction across the vehicle pool are yet unanswered. Therefore, stakeholders from government and industry have partnered to develop ground control station concepts for such operations. The work presented in this paper aims to identify factors that contribute to operator workload. A supervised machine learning-based method built using Support Vector Machines and K-fold cross-validation was used to create workload prediction models for various NASA TLX subscales by leveraging features related to interactions and their relative timings during m:N operations. Results show that the models yielded fairly high predictive accuracies ranging from ~60-75%.

workload prediction↗

Monitoring Airspace Complexity and Determining Contributing Factors

The national airspace has evolved over many years to accommodate increased traffic demand [1] while simultaneously maintaining one of the safest forms of transportation [2], [3]. One of the reasons for this success is the ability of the system and the operators to adapt and accommodate to situations that routinely disrupt optimal operations. These situations may include: adverse weather, delays, early arrivals, equipment outages, and other factors that are outside the operators ability to control. These factors can lead to states where automation is unable to properly handle these issues and therefore air traffic controllers and pilots have to intervene, ultimately increasing communication between operators resulting in higher workload. As controller workload increases to handle sub-optimal operating conditions this can be viewed as an increase in complexity. The reasoning for this is because humans are now required to make tactical decisions in response to external factors, resulting in a departure from the strategic plan where operations would be more efficiently managed. Human operators control airspace complexity under rigid regulations that are constantly changing. The airspace is divided into sectors and the number of aircraft assigned to each controller is limited for safe handling. There has been past work that devised airspace complexity metrics in commercial aviation and related these metrics to controller workload (e.g., [4],[5]). The upper bounds on the system load are pre-determined. Such bounds on complexity make for a safe system, but the system cannot scale and adapt to autonomous, dense, and heterogeneous traffic, including the many types of Unmanned Aerial Vehicles (UAVs) envisioned to be added to the operations. We hypothesize that, as traffic density and heterogeneity grow, and other key metrics change, there will be phase transitions at which the way traffic should be managed changes significantly [6]. We offer a method for in-time detection of contributing factors that lead to phase transitions, characterized by increased complexity. To the best of our knowledge, there is no tool similar to our proposed effort that identifies such contributing factors or precursor patterns. To define the scope we are proposing to measure complexity from the viewpoint of the Terminal Radar Approach Control Facilities (TRACON) controller’s perspective. In particular we are analyzing arrivals into KSFO. With safety as the top concern for airspace operators, it is important to recognize that as density and heterogeneity grow, the focus of the system will change. Times of the day when the airspace has low density and heterogeneity, the flights will follow more efficient paths where the aircraft move on established routes that are more or less directly to the destination. However, when density and heterogeneity increases, the system will begin changing focus to avoiding conflicts and collisions and route the flights in a more flexible way. Higher flexibility requires more communication and coordination between controllers and pilots which the current automation is unable to handle. This paper proposes a novel approach that monitors airspace complexity at multiple scales, uses a Machine Learning-based tool that predicts when operations will transition to a regime of greater complexity, and identifies actions that can reduce the complexity while still maintaining efficient and safe operations. We demonstrate our proposed approach using data from multiple complementary sources. This includes, but is not limited to: historical aircraft surveillance data from NASA’s Sherlock Data Warehouse [7], METAR weather data, and airport configuration data from Aviation System Performance Metrics (ASPM). The surveillance data flight paths are sampled at a variable sample rate — increasing as the aircraft approaches the airport. This is due to how Sherlock manages flight track stitching between different radar facilities which have different sampling rates. The weather and performance data are logged at defined intervals throughout the day at a courser refresh rate. In addition to the logged data and metrics, we leverage pre-defined Standard Terminal Arrival Routes (STARs) procedures to characterize the path of each flight. Each flight files for one of these routes in the flight plan well before entering the terminal airspace, and approximately follows the route until it leaves the STAR, typically on the final fix of a runway transition. However, most flights do not always fly the full STAR procedure to completion [8], but the majority do adhere to the fixes within the common route of the procedure. Our approach leverages fixes in the common route of each of the STARs to build a reference path to the airport. This allows us to characterize the flight paths in what we are defining as the “maneuvering area” (the airspace between the STAR and before the flight is lined up on the runway’s final approach) to determine how off nominal the flights are to calculate its complexity score. Determining airspace complexity is a concept that does not have a concrete answer. In designing this metric, we consider what increases the workload for the air traffic controllers. Consequently more specialized vectoring maneuvers results in higher workload. Accordingly, we start with a theory: each flight has a direct path it takes from the STAR’s common route to the final approach’s outer marker fix for the flight’s landing runway. It is important to note that the direct path is only used as a reference. If the majority of the flights have a large consistent offset as compared to other routes it does not necessarily mean that those flights have higher complexity. We are merely building a distribution based on this direct path for that particular STAR and runway pair to determine the normal mode of operations for that route. Flights that are in the upper tail of these distributions will result in higher complexity scores and flights that fly in the median will represent the normal mode of operations and therefore will have lower complexity scores. Since flights following each STAR route take different paths to the airport, we have a different distribution for each STAR route and therefore can model these distributions to compute a complexity score from their respective normalized distributions. To evaluate the effectiveness of our proposed airspace complexity metric we will compare against an established approach based on trajectory clustering [9]. This unsupervised learning technique consists of the following steps: (1) identify the general maneuvering areas (waypoints) by performing $\kappa$-means or DBSCAN clustering on locations where aircraft frequently turn based on the surveillance radar track data, (2) map flight trajectories onto sequences of waypoints, and (3) cluster the sequences based on their common subsequences. From a high-level perspective, this baseline model learns nominal operations in the airspace through the sequence of waypoints that are representative of where aircraft change direction and defines deviations from the nominal operations as “complex.” Therefore, more deviations from the nominal operations correspond to higher complexity values. For our validation, we re-implemented this technique and tune model hyper-parameters to correctly detect waypoints for the arrival traffic into the San Francisco bay area. We will compute the complexity measure over a one-year period using our proposed technique as well as the baseline. Our validation will be based on each technique’s ability to detect a set of undesirable outcomes (e.g., go-arounds, holding patterns, average time in the airspace, etc.). Since our current complexity metric is derived from the offset from the direct reference path, it’s important to understand what causes these offsets. In many of the flights with high offset distance, flights performing holding patterns and S turns can be observed. These maneuvering tactics are utilized to add distance between the aircraft and the destination runway to prevent multiple flights from having conflicting arrival times. In order to predict a rise in complexity (or the precursor to complexity), it’s necessary to be able to identify these potential conflicts (which in turn, result in higher offsets). To do this, we define a “representative flight” for each STAR route and runway pair. This flight is approximately the path the flight would take if there was a clear path with no other flights in the airspace — including the time remaining to the airport. We first identify the flights for a given STAR runway pair using the offset to the reference path distributions that fall between the 44-55 percentiles. This yields the flights that conform to the most normal mode of operation. Each of these flights is partitioned based on the percent complete from the entry point into the maneuvering areas from 0\% – 100\% complete. Then for each percent “bin”, we take the median value of the flight’s latitude/longitude coordinates, airspeed, and (non causal) time remaining to the airport to construct a lookup table for each percent complete bin on a given route. As a flight enters the maneuvering area, we can find the estimated arrival time of a flight to the airport by finding the closest point to the representative path’s percent complete bin (relative to the flight’s current position at any snapshot in the airspace) and therefore retrieve the corresponding remaining time left on the “representative path”. We assume that the flight will follow the representative path to completion when deriving these estimates. We can then compare these estimated arrival times against other flights for the same snapshot in time to identify potential conflicts. If more flights are estimated to arrive within a tolerance window than there are runways available, then we have a potential conflict. We can use this derived measure along with other factors expected to add disruption to the operation such as weather and runway configuration changes as an input to machine learning tools to detect precursors that increases in our complexity measure. This novel method will assist in uncovering insights into the contributing factors that lead to increased complexity that may allow for in-time responses to avoid reaching a high complexity state in the airspace.

complexity↗

Event Report for The Ethical Artificial Intelligence Quantification Workshop

Artificial Intelligence (AI) is a powerful emerging technology area which requires special attention to using it ethically. AI ethics is still an emerging field, and the partners for this workshop and report seek to move AI ethics discussion ahead by experimenting with ways to measure AI ethics criteria. The following document describes the outcomes and learnings from The Ethical Artificial Intelligence Quantification Workshop held at the National Institute for Aerospace (NIA), Hampton, Virginia on May 12th, 2022. The purpose of the workshop was for participants to evaluate and experiment-with the methodology and process presented by AIEthics.World in cooperation with Intel Corporation. The meeting participants learned about the Ethical AI Certification and Maturity Model™ and applied the methodology to selected notional AI systems. The workshop facilitated the evaluation of the maturity of the AI system according to ethical considerations relevant to NASA, NIA and other participants. The workshop consisted of three main phases. The first phase focused on understanding and summarizing NASA’s ethical approaches, mission and values based on published documentation, discussions and individual insights & opinions of participants. This information was prioritized, weighted, ordered, and quantified in phase two, to formulate an alignment between human values (ethics) and their applicability to AI systems during all lifecycle phases. The first two phases were summarized as a form of ethical genealogy for artificial intelligence, specific to NASA’s ethical approaches. In the third and last phase of the workshop the participants evaluated notional examples of artificial intelligence to qualify and quantify its ability to adhere to the organizational ethics approaches, using the Ethical AI Certification and Maturity Model™. The workshop uses the concept of genealogy, in the traditional sense: the study and traceability of lines of ancestors in the process of evolutionary development from earlier forms. However, as it is applied to an Ethical AI definition, it is providing the insights to the necessary and mandatory traceability of content, data, metrics, telemetry, elements, and structures which are used in the AI’s lifecycle to foster and measure AI ethics in all steps of its lifecycle. The Ethical Artificial Intelligence Quantification Workshop provided NASA with the opportunity to apply the Ethical AI Certification and Maturity Model™, in combination with existing and well-known decision-making and quality control methods to identify the metrics and measurements for an Ethical AI and assess its ethical condition and quality aligned with NASA ethics approaches. The result of the workshop is the capacity for NASA to apply the maturity model assessment to its AI Systems as desired and if necessary, publish the ability of these AI Systems to adhere to the organizational ethical goals. AI ethics frameworks need to be customized for each application domain, for example, individual NASA Mission Directorates. General principles that work in one area such as AI/Machine Learning-based text analysis (the ethics of information-extraction) may need to be adapted for another such as sense-and-avoid decision-making in a flight environment. The workshop was conducted among approximately twenty NASA subject matter experts, so the elements noted above should be considered examples, not definitive NASA ethical AI principles, genealogy, etc. Generating a definitive AI ethics framework for an organization as diverse as NASA would require far more discussion, debate, review, etc. However, the workshop provided valuable insight into mechanisms and processes for quantifying AI ethical qualities.

Artificial Intelligence↗

A Strategic Approach for Dense, Integrated, Vehicle Navigation

Drone usage has been on the rise in recent years with applications that include parcel delivery, wildlife protection, precision farming, law enforcement, and industrial inspection, just to name a few. Once regulations and safety policies are put in place to allow for the widespread use of unmanned drones, the number of aircraft in the National Airspace System (NAS) is expected to skyrocket to millions, potentially congesting the airspace which increases the likelihood of separation violations and possibly incidents. Currently, flight infrastructure can only support a few thousand aircraft flying over the United States National Airspace System (NAS) at any given time. A delay at one airport can send ripple effects throughout the system, causing more delays and missed connections. In air traffic control, separation is the concept of keeping an “ownship” aircraft outside a minimum distance from “intruder” aircraft to reduce the risk of the aircraft colliding, as well as preventing accidents due to secondary factors, such as wake turbulence. Maintaining proper separation is often a safety critical property for fixed-wing drones in the airspace. This paper addresses drone separation in time and distance for high volume corridors (en route) and lanes (on ground), merging as well as crossing intersections of multiple corridors/lanes. In this paper, the term drone is applied to both Unmanned Aerial Vehicle (UAV) and small Unmanned Aircraft System (UAS) vehicles operating autonomously. There exists a gamut of approaches to the merging and crossing problems. At one end of the extreme are the conservative yet low cost and verifiable solutions of today that deal with two drones at a time. At the other end are complex Machine Learning-based solutions with high computing requirements for fully autonomous drones of the future that are expected to handle all contentions. This paper presents a feasible and verifiable strategic approach to these problems that is based on distributed cooperation between the drones and the infrastructure. Three phases of the strategic approach (Prepare, Adjust, Commit) are presented. Simulation results are presented that show the proposed approach is stable and resilient to induced perturbations and guarantees a set of fixed-wing drones to merge and cross intersections by adjusting their speed based on their distance to the aircraft in front of them while remaining in the equilibrium state. The equilibrium state is defined as the state when a set of n aircraft move at a relatively constant speed and uniform spacing from each other in a congested system. A congested system is defined as the state when at least one aircraft cannot move at its maximum allowed speed. Unlike existing centralized and pre-planned approaches, the proposed solution is fully distributed and enables autonomous aircraft to decide to adjust their speed and distance with respect to the preceding aircraft, dynamically. Simulation results are presented that assess the feasibility of the approach.

Distributed↗

A Strategic Approach for Dense, Integrated, Vehicle Navigation

Drone usage has been on the rise in recent years with applications that include parcel delivery, wildlife protection, precision farming, law enforcement, and industrial inspection, just to name a few. Once regulations and safety policies are put in place to allow for the widespread use of unmanned drones, the number of aircraft in the National Airspace System (NAS) is expected to skyrocket to millions, potentially congesting the airspace which increases the likelihood of separation violations and possibly incidents. Currently, flight infrastructure can only support a few thousand aircraft flying over the United States National Airspace System (NAS) at any given time. A delay at one airport can send ripple effects throughout the system, causing more delays and missed connections. In air traffic control, separation is the concept of keeping an “ownship” aircraft outside a minimum distance from “intruder” aircraft to reduce the risk of the aircraft colliding, as well as preventing accidents due to secondary factors, such as wake turbulence. Maintaining proper separation is often a safety critical property for fixed-wing drones in the airspace. This paper addresses drone separation in time and distance for high volume corridors (en route) and lanes (on ground), merging as well as crossing intersections of multiple corridors/lanes. In this paper, the term drone is applied to both Unmanned Aerial Vehicle (UAV) and small Unmanned Aircraft System (UAS) vehicles operating autonomously. There exists a gamut of approaches to the merging and crossing problems. At one end of the extreme are the conservative yet low cost and verifiable solutions of today that deal with two drones at a time. At the other end are complex Machine Learning-based solutions with high computing requirements for fully autonomous drones of the future that are expected to handle all contentions. This paper presents a feasible and verifiable strategic approach to these problems that is based on distributed cooperation between the drones and the infrastructure. Three phases of the strategic approach (Prepare, Adjust, Commit) are presented. Simulation results are presented that show the proposed approach is stable and resilient to induced perturbations and guarantees a set of fixed-wing drones to merge and cross intersections by adjusting their speed based on their distance to the aircraft in front of them while remaining in the equilibrium state. The equilibrium state is defined as the state when a set of n aircraft move at a relatively constant speed and uniform spacing from each other in a congested system. A congested system is defined as the state when at least one aircraft cannot move at its maximum allowed speed. Unlike existing centralized and pre-planned approaches, the proposed solution is fully distributed and enables autonomous aircraft to decide to adjust their speed and distance with respect to the preceding aircraft, dynamically. Simulation results are presented that assess the feasibility of the approach.

Distributed↗

Deep Learning-Based Negotiation Strategy Selection for Cooperative Conflict Resolution in Urban Air Mobility

This paper presents a collaborative conflict resolution technique using deep neural network-based intelligent search of the solution space. This approach offers a rapid convergence to a mutually acceptable solution for real-time conflict resolution, suitable for urban air mobility operations. Furthermore, the presented technique allows operational flexibility to the urban air mobility agents where these agents can collaboratively devise the solution via integrative negotiation, based on their local utility functions, as long as such a solution does not violate the global safety thresholds. The presented machine-to-machine negotiation method is built on our prior work on holistic assessment of the airspace and potential conflict detection implemented at-the-edge, onboard the unmanned aircraft systems. This paper extends the prior work to augment decision-making at-the-edge, thereby, promising a true distributed control architecture for urban air mobility. In this approach, each agent (a) builds a potential in-flight conflict map, (b) identifies the conflicting agents, (c) dynamically prepares a list of alternatives based on its current utility functions, (d) negotiates with the conflicting agents to pick one of these alternatives, and (e) implements the negotiated alternative to mutually resolve the conflict. Note that such an approach does not require a contingency plan to be made pre-flight, as the conflict resolution strategies are decided and negotiated in real time based on the present state of the agent. The contingency plan, if available, can serve as an input to the real-time conflict resolution strategy formulation, and also can be used as a fallback plan in case the negotiation fails and the impacted agents need to switch to a rule-based/supervisory resolution mode from the discussed distributed resolution mode. The presented collaborative negotiation-based conflict resolution technique incorporates a time-dependent reward function to catalyze collaborative resolution by incentivizing the agents with local and global rewards beneficial to their business operations.

Advanced Air Mobility↗

Tradeoffs When Considering Deep Reinforcement Learning for Contingency Management in Advanced Air Mobility

Air transportation is undergoing a rapid evolution globally with the introduction of Advanced Air Mobility (AAM) and with it comes novel challenges and opportunities for transforming aviation. As AAM operations introduce increasing heterogeneity in vehicle capabilities and density, increased levels of automation are likely necessary to achieve operational safety and efficiency goals. This paper focuses on one example where increased automation has been suggested. Autonomous operations will need contingency management systems that can monitor evolving risk across a span of interrelated (or interdependent) hazards and, if necessary, execute appropriate control interventions via supervised or automated decision making. Accommodating this complex environment may require automated functions (autonomy) that apply artificial intelligence (AI) techniques that can adapt and respond to a quickly changing environment. This paper explores the use of Deep Reinforcement Learning (DRL) which has shown promising performance in complex and high-dimensional environments where the objective can be constructed as a sequential decision-making problem. An extension of a prior formulation of the contingency management problem as a Markov Decision Process (MDP) is presented and uses a DRL framework to train agents that mitigate hazards present in the simulation environment. A comparison of these learning-based agents and classical techniques is presented in terms of their performance, verification difficulties, and development process.

machine learningautonomous systems; flight simulat↗

Tradeoffs When Considering Deep Reinforcement Learning for Contingency Management in Advanced Air Mobility

Air transportation is undergoing a rapid evolution globally with the introduction of Advanced Air Mobility (AAM) and with it comes novel challenges and opportunities for transforming aviation. As AAM operations introduce increasing heterogeneity in vehicle capabilities and density, increased levels of automation are likely necessary to achieve operational safety and efficiency goals. This paper focuses on one example where increased automation has been suggested. Autonomous operations will need contingency management systems that can monitor evolving risk across a span of interrelated (or interdependent) hazards and, if necessary, execute appropriate control interventions via supervised or automated decision making. Accommodating this complex environment may require automated functions (autonomy) that apply artificial intelligence (AI) techniques that can adapt and respond to a quickly changing environment. This paper explores the use of Deep Reinforcement Learning (DRL) which has shown promising performance in complex and high-dimensional environments where the objective can be constructed as a sequential decision-making problem. An extension of a prior formulation of the contingency management problem as a Markov Decision Process (MDP) is presented and uses a DRL framework to train agents that mitigate hazards present in the simulation environment. A comparison of these learning-based agents and classical techniques is presented in terms of their performance, verification difficulties, and development process.

machine learning↗