Search NASA⌕ Search

SEARCH · Search NASA

Results for “ACAS X”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Helicopter Pilot Evaluations of the Airborne Collision Avoidance System Xr in a High-Fidelity Motion Simulation

New aircraft and aerial Urban Air Mobility operations require updated technologies to maintain vehicle separation during flight. Specifically, higher-density airspace will need more than traditional air traffic control to keep vehicles well clear of each other and avoid Near Midair Collisions. In response, the Federal Aviation Administration is currently developing the Airborne Collision Avoidance System X (ACAS X) for next-generation air traffic deconfliction, which provides caution-level and warning-level alerts in response to multiple aircraft types. This study recruited six helicopter pilots to fly an electric vertical takeoff and landing vehicle model in simulated operations under Visual Flight Rules (VFR). Flights were accomplished using the high-fidelity Vertical Motion Simulator at the NASA Ames Research Center. Participants controlled the vehicle using two side inceptors and foot rudders. The rotorcraft variant of ACAS X (ACAS Xr) was provided for alerting and guidance during traffic conflicts. Pilots used this system while giving feedback to the researchers through questionnaires, debriefs, and other discussions. Variables of interest to the study were phases of flight (i.e., Cruise, Hover, and Approach) and ACAS Xr configurations: The Collision Avoidance System configuration behaves similarly to current commercial traffic systems used for tactical deconfliction in crewed vehicles, and the Detect and Avoid configuration was developed to provide extra, corrective-level guidance for unmanned aircraft systems. Results showed that pilots found the alerting and guidance from ACAS Xr useful, effective, and acceptable for VFR operations. Certain elements, like speed guidance and text banners, were found to be of no use to the pilots. Hover and Approach scenarios were considered the most difficult for ACAS Xr alerting. Reasons for this difficulty were partially due to learning interference (i.e., overcoming previously learned behavior) and partially due to the vehicle model (i.e., NASA’s Lift Plus Cruise design). Still, alerting-based confounds reveal the need for more development for ACAS Xr during these Hover and Approach flight phases. Study caveats and future projects are discussed.

air taxis↗
NASA NTRSDate not supplied

Helicopter Pilot Evaluations of the Airborne Collision Avoidance System Xr in a High-Fidelity Motion Simulation

New aircraft and aerial Urban Air Mobility operations require updated technologies to maintain vehicle separation during flight. Specifically, higher-density airspace will need more than traditional air traffic control to keep vehicles well clear of each other and avoid Near Midair Collisions. In response, the Federal Aviation Administration is currently developing the Airborne Collision Avoidance System X (ACAS X) for next-generation air traffic deconfliction, which provides caution-level and warning-level alerts in response to multiple aircraft types. This study recruited six helicopter pilots to fly an electric vertical takeoff and landing vehicle model in simulated operations under Visual Flight Rules (VFR). Flights were accomplished using the high-fidelity Vertical Motion Simulator at the NASA Ames Research Center. Participants controlled the vehicle using two side inceptors and foot rudders. The rotorcraft variant of ACAS X (ACAS Xr) was provided for alerting and guidance during traffic conflicts. Pilots used this system while giving feedback to the researchers through questionnaires, debriefs, and other discussions. Variables of interest to the study were phases of flight (i.e., Cruise, Hover, and Approach) and ACAS Xr configurations: The Collision Avoidance System configuration behaves similarly to current commercial traffic systems used for tactical deconfliction in crewed vehicles, and the Detect and Avoid configuration was developed to provide extra, corrective-level guidance for unmanned aircraft systems. Results showed that pilots found the alerting and guidance from ACAS Xr useful, effective, and acceptable for VFR operations. Certain elements, like speed guidance and text banners, were found to be of no use to the pilots. Hover and Approach scenarios were considered the most difficult for ACAS Xr alerting. Reasons for this difficulty were partially due to learning interference (i.e., overcoming previously learned behavior) and partially due to the vehicle model (i.e., NASA’s Lift Plus Cruise design). Still, alerting-based confounds reveal the need for more development for ACAS Xr during these Hover and Approach flight phases. Study caveats and future projects are discussed.

air taxis↗
NASA NTRSDate not supplied

Integration of Automated Systems (IAS) Flight Test Overview

The Integration of Automated Systems (IAS) Project is conducting a series of 2023 flight tests supporting NASA's Advanced Air Mobility (AAM) and National Campaign efforts. These flights include crewed, test (i.e., ownship) and traffic (i.e., intruder) aircraft that will fly with unique technologies onboard. The presentation will include overviews of the Hazard Perception and Avoidance (HPA) and Flight Path Management (FPM) technical areas but will focus primarily on HPA. HPA will test the FAA's Airborne Collision Avoidance System X (ACAS X), a next-generation collision avoidance tool developed to support different aircraft types and operations. The rotorcraft variant, ACAS Xr, is designed to accommodate existing helicopter platforms and in-development, vertical takeoff and landing (VTOL) concepts, which are critical to the emerging AAM concept of operations. Two configurations of ACAS Xr will be examined: Collision Avoidance System (CAS, similar to the Traffic Collision Avoidance System [TCAS] II) and Detect and Avoid (DAA, previously developed to provide added situational awareness for uncrewed aircraft). Additionally, this system will be explored during cruise and low-speed flight as well as flights within en-route, structured (i.e., dense/urban), and terminal airspaces. Scripted flight conflicts will be conducted, and these conflicts will be mitigated through maneuvers that are manual (i.e., performed by the pilots) or automated (i.e., achieved by the cooperation of the program middleware and onboard ownship systems). Objective data will be collected involving system and pilot performance as well as pilot decisions; subjective data will include pilot opinions of ACAS Xr's alerting and guidance as well as the automated maneuvers.

detect and avoid↗
NASA NTRSDate not supplied

Integration of Automated Systems (IAS) Flight Test - Hazard Perception & Avoidance (HPA) Results

The Integration of Automated Systems (IAS) flight test series concluded in October 2023 in support of NASA's Advanced Air Mobility (AAM) project. These flights include crewed, test (i.e., ownship) and traffic (i.e., intruder) aircraft that flew with unique technologies onboard. The presentation includes overviews of the flight tet itself and the specific results that pertain to the Hazard Perception and Avoidance (HPA) technical areas. HPA tested the FAA's Airborne Collision Avoidance System X (ACAS X), a next-generation collision avoidance tool developed to support different aircraft types and operations. The rotorcraft variant, ACAS Xr, is designed to accommodate existing helicopter platforms and in-development, vertical takeoff and landing (VTOL) concepts, which are critical to the emerging AAM concept of operations. Two configurations of ACAS Xr were examined: Collision Avoidance System (CAS, similar to the Traffic Collision Avoidance System [TCAS] II) and Detect and Avoid (DAA, previously developed to provide added situational awareness for remote pilots). Additionally, this system was flown in cruise and low-speed flight regimes as well as within en-route and (emulated), structured (i.e., dense/urban), and terminal airspaces. Results include the types of alerts generated by ACAS Xr across the different configurations, the distances at which the alerts were generated, response times, manuever sizes, miss distances, and general comments from pilots. Key takeaways and next steps are also provided.

detect and avoid↗
NASA NTRSDate not supplied

Helicopter Pilot Assessments of the Airborne Collision Avoidance System XR With Automated Maneuvering

The Airborne Collision Avoidance System X (ACAS X) is a next-generation collision avoidance tool developed to support different aircraft types and operations. The rotorcraft variant – referred to as ACAS XR – is designed to accommodate existing helicopter platforms as well as in-development, electric vertical takeoff and landing concepts, which are critical to the emerging concept of operations referred to as Advanced Air Mobility. The fundamental role of ACAS XR is to provide Detect and Avoid (DAA) and/or Collision Avoidance (CA) protection against airborne traffic. DAA alerting and guidance in the context of ACAS XR is caution-level and “suggestive,” and is to be used by the pilot if, and when, they decide to maneuver against an identified threat to DAA “well clear.” The CA alerting, by contrast, is warning-level and “directive,” with the pilot required to comply with the associated guidance to prevent a predicted Near Midair Collision (NMAC). The CA alerts generated by ACAS XR are referred to as Resolution Advisories (RAs) consistent with previous CA systems. Unlike earlier CA systems, ACAS XR issues RAs in the horizontal and vertical dimensions as well as multi-axis RAs (referred to as “Blended” RAs). According to the Minimal Operational Performance Standards of DAA systems for Unmanned Aircraft Systems, maneuvers to comply with RAs may be automated, whereas maneuvers based on DAA alerting assume a manual response. The current study was a human-in-the-loop simulation that presented rotorcraft pilots with ACAS XR alerts and guidance in a fixed-base eVTOL simulator with varying levels of automation. Objective results showed that pilots complied with all RAs within the expected 5-second time window and responded to DAA alerts quicker than in earlier studies with ACAS XU. Pilots often made larger horizontal deviations during Manual RAs, but often favored vertical and blended maneuvers. No NMACs occurred, and losses of well clear were mainly attributed to the obligation of the pilots and system to maneuver only after the CA phase of the encounter had begun. Other losses were due to pilots’ noncompliance or disregard for ACAS XR’s alerting and guidance. Noncompliance with RAs most frequently occurred when pilots determined they were too close to the terrain to continue to follow Descend RAs, performing vertical maneuvers instead of following Horizontal RAs, or rejecting Horizontal RA updates because they felt that enough maneuvering had been performed. Subjectively, pilots found the DAA and RA alerting and guidance intuitive and useful for VFR helicopter operations. Slightly more pilots preferred the Automated RA condition to the Manual RA condition. Lastly, they also felt that ACAS led to occasional unsafe Descend RAs. Caveats and future implications are discussed.

eVTOL↗
NASA NTRSDate not supplied

Helicopter Pilot Assessments of the Airborne Collision Avoidance System XR With Automated Maneuvering

The Airborne Collision Avoidance System X (ACAS X) is a next-generation collision avoidance tool developed to support different aircraft types and operations. The rotorcraft variant – referred to as ACAS XR – is designed to accommodate existing helicopter platforms as well as in-development, electric vertical takeoff and landing concepts, which are critical to the emerging concept of operations referred to as Advanced Air Mobility. The fundamental role of ACAS XR is to provide Detect and Avoid (DAA) and/or Collision Avoidance (CA) protection against airborne traffic. DAA alerting and guidance in the context of ACAS XR is caution-level and “suggestive,” and is to be used by the pilot if, and when, they decide to maneuver against an identified threat to DAA “well clear.” The CA alerting, by contrast, is warning-level and “directive,” with the pilot required to comply with the associated guidance to prevent a predicted Near Midair Collision (NMAC). The CA alerts generated by ACAS XR are referred to as Resolution Advisories (RAs) consistent with previous CA systems. Unlike earlier CA systems, ACAS XR issues RAs in the horizontal and vertical dimensions as well as multi-axis RAs (referred to as “Blended” RAs). According to the Minimal Operational Performance Standards of DAA systems for Unmanned Aircraft Systems, maneuvers to comply with RAs may be automated, whereas maneuvers based on DAA alerting assume a manual response. The current study was a human-in-the-loop simulation that presented rotorcraft pilots with ACAS XR alerts and guidance in a fixed-base eVTOL simulator with varying levels of automation. Objective results showed that pilots complied with all RAs within the expected 5-second time window and responded to DAA alerts quicker than in earlier studies with ACAS XU. Pilots often made larger horizontal deviations during Manual RAs, but often favored vertical and blended maneuvers. No NMACs occurred, and losses of well clear were mainly attributed to the obligation of the pilots and system to maneuver only after the CA phase of the encounter had begun. Other losses were due to pilots’ noncompliance or disregard for ACAS XR’s alerting and guidance. Noncompliance with RAs most frequently occurred when pilots determined they were too close to the terrain to continue to follow Descend RAs, performing vertical maneuvers instead of following Horizontal RAs, or rejecting Horizontal RA updates because they felt that enough maneuvering had been performed. Subjectively, pilots found the DAA and RA alerting and guidance intuitive and useful for VFR helicopter operations. Slightly more pilots preferred the Automated RA condition to the Manual RA condition. Lastly, they also felt that ACAS led to occasional unsafe Descend RAs. Caveats and future implications are discussed.

eVTOL↗
NASA NTRSDate not supplied

Advanced Air Mobility Operations & Automation Part II Technical Lecture

The Airborne Collision Avoidance System X (ACAS X) is a next-generation collision avoidance tool developed to support different aircraft types and operations. The rotorcraft variant – referred to as ACAS XR – is designed to accommodate existing helicopter platforms as well as in-development, electric vertical takeoff and landing concepts, which are critical to the emerging concept of operations referred to as Advanced Air Mobility. The fundamental role of ACAS XR is to provide Detect and Avoid (DAA) and/or Collision Avoidance (CA) protection against airborne traffic. DAA alerting and guidance in the context of ACAS XR is caution-level and “suggestive,” and is to be used by the pilot if, and when, they decide to maneuver against an identified threat to DAA “well clear.” The CA alerting, by contrast, is warning-level and “directive,” with the pilot required to comply with the associated guidance to prevent a predicted Near Midair Collision (NMAC). The CA alerts generated by ACAS XR are referred to as Resolution Advisories (RAs) consistent with previous CA systems. Unlike earlier CA systems, ACAS XR issues RAs in the horizontal and vertical dimensions as well as multi-axis RAs (referred to as “Blended” RAs). According to the Minimal Operational Performance Standards of DAA systems for Unmanned Aircraft Systems, maneuvers to comply with RAs may be automated, whereas maneuvers based on DAA alerting assume a manual response. The current presentation discusses two human-in-the-loop simulations that presented rotorcraft pilots with ACAS XR alerts and guidance in eVTOL simulators with varying levels of automation. It also presents overviews of these studies as well as how they assisted live flight tests, which will occur throughout 2023.

air taxis↗
NASA NTRS2017

Towards Verification and Validation for Increased Autonomy

This presentation goes over the work we have performed over the last few years on verification and validation of the next generation onboard collision avoidance system, ACAS X, for commercial aircraft. It describes our work on probabilistic verification and synthesis of the model that ACAS X is based on, and goes on to the validation of that model with respect to actual simulation and flight data. The presentation then moves on to identify the characteristics of ACAS X that are related to autonomy and to discuss the challenges that autonomy pauses on VV. All work presented has already been published.

Giannakopoulou, Dimitra↗
NASA NTRS2017

Towards Verification and Validation for Increased Autonomy

This presentation goes over the work we have performed over the last few years on verification and validation of the next generation onboard collision avoidance system, ACAS X, for commercial aircraft. It describes our work on probabilistic verification and synthesis of the model that ACAS X is based on, and goes on to the validation of that model with respect to actual simulation and flight data. The presentation then moves on to identify the characteristics of ACAS X that are related to autonomy and to discuss the challenges that autonomy pauses on VV. All work presented has already been published.

Giannakopoulou, Dimitra↗
NASA NTRS2017

Unmanned Aircraft Systems (UAS) Integration in the National Airspace System (NAS) Project: Advanced Collision Avoidance System for UAS (ACAS Xu) Interoperability White Paper Presentation

The Phase 1 DAA Minimum Operational Performance Standards (MOPS) provided requirements for two classes of DAA equipment: equipment Class 1 contains the basic DAA equipment required to assist a pilot in remaining well clear, while equipment Class 2 integrates the Traffic Alert and Collision Avoidance (TCAS) II system. Thus, the Class 1 system provides RWC functionality only, while the Class 2 system is intended to provide both RWC and Collision Avoidance (CA) functionality, in compliance with the Minimum Aviation System Performance (MASPS) for the Interoperability of Airborne Collision Avoidance Systems. The FAAs TCAS Program Office is currently developing Airborne Collision Avoidance System X (ACAS X) to support the objectives of the Federal Aviation Administrations (FAA) Next Generation Air Transportation System Program (NextGen). ACAS X has a suite of variants with a common underlying design that are intended to be optimized for their intended airframes and operations. ACAS Xu being is designed for UAS and allows for new surveillance technologies and tailored logic for platforms with different performance characteristics. In addition to Collision Avoidance (CA) alerting and guidance, ACAS Xu is being tuned to provide RWC alerting and guidance in compliance with the SC 228 DAA MOPS. With a single logic performing both RWC and CA functions, ACAS Xu will provide industry with an integrated DAA solution that addresses many of the interoperability shortcomings of Phase I systems. While the MOPS for ACAS Xu will specify an integrated DAA system, it will need to show compliance with the RWC alerting thresholds and alerting requirements defined in the DAA Phase 2 MOPS. Further, some functional components of the ACAS Xu system such as the remote pilots displayed guidance might be mostly references to the corresponding requirements in the DAA MOPS. To provide a seamless, integrated, RWC-CA system to assist the pilot in remaining well clear and avoiding collisions, several issues need to be addressed within the Phase 2 SC-228 DAA efforts. Interoperability of the RWC and CA alerting and guidance, and ensuring pilot comprehension, compliance and performance, will be a primary research area.

detect and avoid↗
NASA NTRS2018

Differential Adaptive Stress Testing of Airborne Collision Avoidance Systems

The next-generation Airborne Collision Avoidance System (ACAS X) is currently being developed and tested to replace the Traffic Alert and Collision Avoidance System (TCAS) as the next international standard for collision avoidance. To validate the safety of the system, stress testing in simulation is one of several approaches for analyzing near mid-air collisions (NMACs). Understanding how NMACs can occur is important for characterizing risk and informingdevelopment of the system. Recently, adaptive stress testing (AST) has been proposed as a way to find the most likely path to a failure event. The simulation-based approach accelerates search by formulating stress testing as a sequential decision process then optimizing it using reinforcement learning. The approach has been successfully applied to stress test a prototype of ACAS Xin various simulated aircraft encounters. In some applications, we are not as interestedin the system's absolute performance as its performance relative to another system. Such situations arise, for example, during regression testing or when deciding whether a new system should replace an existing system. In our collision avoidance application, we are interested in finding cases where ACAS X fails but TCAS succeeds in resolving a conflict. Existing approaches do not provide an efficient means to perform this type of analysis. This paper extends the AST approach to differential analysis by searching two simulators simultaneously and maximizing the difference between their outcomes. We call this approach differential adaptive stress testing (DAST). We apply DAST to compare a prototype of ACAS X against TCAS and show examples of encounters found by the algorithm.

Lee, Ritchie↗
NASA NTRSDate not supplied

Assessing Helicopter Pilots’ Detect and Avoid and Collision Avoidance Performance With ACAS Xr

The latest variant of the Federal Aviation Administration’s Airborne Collision Avoidance System (ACAS X) is being designed for both crewed and uncrewed rotorcraft. Referred to as ACAS Xr, the system joins a suite of other ACAS X variants poised to replace the Traffic Alert and Collision Avoidance System (TCAS II). ACAS Xr is tuned to support current-day helicopter platforms as well as electric Vertical Takeoff and Landing (eVTOL) vehicles that are still under development. Given this flexibility, ACAS Xr may be used by helicopter crews currently in operation or by remotely-operated eVTOL aircraft in the emerging Advanced Air Mobility (AAM) market. To cover the range of potential uses, two distinct configurations are being proposed for ACAS Xr: Collision Avoidance System (CAS) and Detect and Avoid (DAA). Under the CAS configuration, ACAS Xr provides minimal caution-level alerting but issues directive warning-level alerting and guidance. The DAA configuration, by contrast, provides caution-level alerting and guidance, in addition to the warning-level alerting and guidance. The current study was performed as part of the National Aeronautics and Space Administration’s AAM project. Six helicopter pilots were recruited to fly a variety of scripted traffic scenarios in a full-motion, crewed eVTOL simulator. Participants flew 60 encounters over two days, reacting to pre-recorded intruder aircraft that were scripted to fly into the participant’s aircraft from different approach angles, relative altitudes, and during different phases of flight. The pilots flew half of the encounters with the CAS configuration and half with the DAA configuration. Within each block of 30 encounters, pilots experienced 10 conflicts while in cruise, 10 in hover, and 10 while on approach to a heliport. Results showed that pilot response times were consistently under 5 seconds for RAs and under 10 seconds for DAA alerts, when present, during all three phases of flight. Unsurprisingly, the DAA configuration was associated with lower rates of en-route and high-severity losses of DAA well clear compared to the CAS configuration in all phases of flight except for the terminal area. Rates of losses of DAA well clear were found to be substantially higher in the Hover scenario, compared to Cruise. Pilots failed to fully comply with RAs at a rate of 0.10-0.18 in all conditions except for the DAA configuration in the Hover scenario, which was associated with a higher non-compliance rate of 0.4 due to Descend RAs issued at low altitudes. The implications of these results with regards to the ongoing development of ACAS Xr is discussed.

air taxis↗
NASA NTRSDate not supplied

Assessing Helicopter Pilots’ Detect and Avoid and Collision Avoidance Performance with ACAS Xr

The latest variant of the Federal Aviation Administration’s Airborne Collision Avoidance System (ACAS X) is being designed for both crewed and uncrewed rotorcraft. Referred to as ACAS Xr, the system joins a suite of other ACAS X variants poised to replace the Traffic Alert and Collision Avoidance System (TCAS II). ACAS Xr is tuned to support current-day helicopter platforms as well as electric Vertical Takeoff and Landing (eVTOL) vehicles that are still under development. Given this flexibility, ACAS Xr may be used by helicopter crews currently in operation or by remotely-operated eVTOL aircraft in the emerging Advanced Air Mobility (AAM) market. To cover the range of potential uses, two distinct configurations are being proposed for ACAS Xr: Collision Avoidance System (CAS) and Detect and Avoid (DAA). Under the CAS configuration, ACAS Xr provides minimal caution-level alerting but issues directive warning-level alerting and guidance. The DAA configuration, by contrast, provides caution-level alerting and guidance, in addition to the warning-level alerting and guidance. The current study was performed as part of the National Aeronautics and Space Administration’s AAM project. Six helicopter pilots were recruited to fly a variety of scripted traffic scenarios in a full-motion, crewed eVTOL simulator. Participants flew 60 encounters over two days, reacting to pre-recorded intruder aircraft that were scripted to fly into the participant’s aircraft from different approach angles, relative altitudes, and during different phases of flight. The pilots flew half of the encounters with the CAS configuration and half with the DAA configuration. Within each block of 30 encounters, pilots experienced 10 conflicts while in cruise, 10 in hover, and 10 while on approach to a heliport. Results showed that pilot response times were consistently under 5 seconds for RAs and under 10 seconds for DAA alerts, when present, during all three phases of flight. Unsurprisingly, the DAA configuration was associated with lower rates of en-route and high-severity losses of DAA well clear compared to the CAS configuration in all phases of flight except for the terminal area. Rates of losses of DAA well clear were found to be substantially higher in the Hover scenario, compared to Cruise. Pilots failed to fully comply with RAs at a rate of 0.10-0.18 in all conditions except for the DAA configuration in the Hover scenario, which was associated with a higher non-compliance rate of 0.4 due to Descend RAs issued at low altitudes. The implications of these results with regards to the ongoing development of ACAS Xr is discussed.

air taxis↗
NASA NTRSDate not supplied

Are we ready for the first EASA guidance on the use of ML in Aviation?

NASA has been working for the past 12 years on software tools for the assurance of software in Aviation critical systems. For now two years, NASA has focused more on the use of AI-based techniques in Aviation than the traditional software systems used in the past. The primary focus has been on machine learning (ML), and more specifically, on supervised off-line learning ML systems. NSA’s research has been driven by case studies such as a vision-based centerline tracking system (implemented using deep neural networks) and the new generation of collision avoidance systems developed under the FAA guidance, i.e., the family of ACAS-X products. Since EASA has recently released its first usable guidance for Level 1 machine learning applications, it is opportunity to see how the research done at NASA is mapping to this first guidance for ML. In this talk I will use the EASA guidance document as a guide to present the past, present, and future tools and techniques being developed at NASA. The intent is to not only provide an overview of the research effort at NASA but also to see how this effort is addressing the concerns listed in the EASA first usable guidance for ML.

Guillaume Brat↗
NASA NTRS2015

Adaptive Stress Testing of Airborne Collision Avoidance Systems

This paper presents a scalable method to efficiently search for the most likely state trajectory leading to an event given only a simulator of a system. Our approach uses a reinforcement learning formulation and solves it using Monte Carlo Tree Search (MCTS). The approach places very few requirements on the underlying system, requiring only that the simulator provide some basic controls, the ability to evaluate certain conditions, and a mechanism to control the stochasticity in the system. Access to the system state is not required, allowing the method to support systems with hidden state. The method is applied to stress test a prototype aircraft collision avoidance system to identify trajectories that are likely to lead to near mid-air collisions. We present results for both single and multi-threat encounters and discuss their relevance. Compared with direct Monte Carlo search, this MCTS method performs significantly better both in finding events and in maximizing their likelihood.

Verification and Validation↗
NASA NTRSDate not supplied

Adaptive Stress Testing of Collision Avoidance Systems for Small UASs with Deep Reinforcement Learning

The next-generation Airborne Collision Avoidance System for smaller UASs (ACAS sXu) is currently being developed and tested by the Federal Aviation Administration (FAA) to provide detect-and-avoid capability for small unmanned aircraft operating beyond line-of-sight. Due to the complexity and safety-critical nature of the system, safety validation is important not only for the certification of the final system, but also for informing changes during the iterative development process. In this paper, we analyze a prototype of ACAS sXu in simulated aircraft encounters to discover scenarios of small near mid-air collisions (sNMACs), an important safety event in which two aircraft come closer than 50 feet horizontally and 15 feet vertically. Due to the size and complexity of the system as well as rarity of sNMAC events, traditional methods such as Monte Carlo testing often require informed setup and targeting to elicit failures. However, such a dependence on domain knowledge can be incompatible with the independent verification and validation (IV&V) process, the aim of which is to discover unforeseen issues. To address these challenges, we apply an accelerated validation method called adaptive stress testing (AST) to find the most likely sNMAC scenarios without reliance on system introspection. AST uses reinforcement learning to adapt the search towards the most promising areas of the search space as it progresses. We use a state-of-the-art deep reinforcement learning algorithm, proximate policy optimization, to more efficiently search the large and continuous state space. We find that this approach significantly improves the performance of AST compared to a prior approach based on Monte Carlo tree search. We perform experiments using AST to find sNMAC events under various encounter configurations, varying parameters pertaining to dynamics and coordination. Our experiments show AST to be very effective at finding sNMAC scenarios. We summarize our findings, presenting high-level categories of discovered sNMACs and specific examples of encounters in each category.

aircraft collision avoidance↗
NASA NTRS2020

Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning

Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates.As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.

Verification and Validation↗
NASA NTRSDate not supplied

Pathfinding for Airspace with Autonomous Vehicles (PAAV) ACAS Xr Terminal Area Simulation Overview

This presentation provides an overview of an upcoming human-in-the-loop simulation planned by the Pathfinding for Airspace with Autonomous Vehicles (PAAV) subproject, as part of the Air Traffic Management eXploration project. The study will investigate Detect and Avoid (DAA) concepts in and around the DAA terminal area (DTA). The DTA has received little investigation in real-time, pilot-in-the-loop simulation. This study proposes to use current remotely piloted aircraft system (RPAS) pilots to investigate the effectiveness of current DTA requirements using the FAA's Airborne Collision Avoidance System X for Rotorcraft (ACAS Xr) as the DAA system under test. The findings of this stufy will be used to inform DTA and ACAS Xr standards development. This presentation will be provided to RTCA Special Committee 228 as an overview and as an opportunity to collect feedback from the larger RPAS/DAA community on the study design and key scenarios and metrics. The presentation provides a brief overview of open areas of research with ACAS Xr, the experimental design, the simulation environment and the simulation schedule.

advanced air mobility↗