Search NASA⌕ Search

Engineering topics

Randall J Mumaw

Publications and source records attributed to Randall J Mumaw.

12 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2020

Evaluation Issues for a Flight Deck Interface; CAST SE-210 Output 2: Report 4 of 6

This report is part of a series of reports that addresses flight deck design and evaluation, written as a response to loss of control accidents. In particular, this activity is directed at failures in airplane state awareness, in which the pilot loses awareness of the airplane’s energy state or attitude and enters an upset condition. Another report in this series of reports speaks directly to flight deck evaluation methods and metrics for the types of attention and awareness issues that were revealed from the airplane state awareness events. In this report, we describe a wide range of flight deck evaluation issues tied to flight crew performance. The objectives are to establish a framework for thinking about how the flight deck interface should support the performance of the flight crew, and to aid the Federal Aviation Administration (FAA) in identifying relevant human performance issues during the evaluation/certification process. Issues are broken out into sections that cover physical ergonomics, design for usability, data integration and display content, attention and task management, flight crew problem solving, and flight crew teaming. For each issue, we recommend specific ways that the flight deck interface should support the flight crew. For each, we also identify existing 14 CFR Part 25 rules and guidance that are relevant to the issue. This allows the FAA to determine what current regulatory materials can support them in raising the issue with the applicant.

commercial aviation↗
NASA NTRS2020

Identification of Scenarios for System Interface Design Evaluation: CAST SE-210 Output 2 Report 5 of 6

This report is one in a series of reports describing research to develop enhanced approaches to design and evaluation of flight deck interaction. The research was conducted to support the Commercial Aviation Safety Team (CAST) response to incidents and accidents caused by failure of the flight crew to maintain aircraft attitude or energy state awareness. This report partially responds to Safety Enhancement (SE) 210.2 (CAST, 2014) by describing development of operational scenarios for evaluation of flight crew interaction. Evaluations of human performance with an interface rely on asking humans to perform operational tasks in an appropriate operational context. The report includes the major sources of material for developing operational scenarios—which include system state, operational tasks, operational context, and safety events—and a method for developing an appropriate set of operational scenarios for interface evaluation.

commercial aviation↗
NASA NTRS2020

Analysis of Pilot Monitoring Skills and a Review of Training Effectiveness

The commercial aviation industry world-wide has identified a need for improved pilot monitoring and awareness (e.g., FAA, 2013, ICAO, 2016). More specifically, aviation safety data indicate that failures in pilots’ flight path management (FPM) monitoring and awareness have contributed to a range of undesired outcomes: accidents, major upsets, and non-compliance with air traffic control (ATC) guidance. The Federal Aviation Administration (FAA) has further stated that these types of FPM failures are likely to worsen with the increasingly complex air traffic control systems and FPM concepts proposed for NextGen (https://www.faa. gov/nextgen/what_is_nextgen/) operations (e.g., see Hah et al., 2017). Adding to this complexity is the introduction of increasingly automated aircraft systems that can increase monitoring burdens. One potential mitigation for this situation is to enhance pilot training for effective monitoring. NASA Ames Research Center was asked to identify and evaluate training approaches that have the potential to enhance pilots’ ability to effectively monitor for FPM (with the result of improved awareness). The focus of this work is to identify, develop or validate training guidance to improve pilot monitoring/awareness regarding FPM and mitigate the recent trend of accidents and incidents, especially loss of control (LOC) events. The result of this work should be input for improved industry standards and FAA guidance to reduce the risk of incidents and accidents due to inadequate pilot monitoring/awareness. This is the first of three reports that were developed for this project.

aviation human factors↗
NASA NTRS2019

The Role of Alerting System Failures in Loss of Control Accidents CAST SE-210 Output 2

This report is part of a series of reports that address flight deck design and evaluation, written as a response to loss of control accidents. In particular, this activity is directed at failures in airplane state awareness in which the pilot loses awareness of the airplane's energy state or attitude and enters an upset condition. In a report by the Commercial Aviation Safety Team, an analysis of accidents and incidents related to loss of airplane state awareness determined that hazard alerting was not effective in producing the appropriate pilot response to a hazard (CAST, 2014). In the current report, we take a detailed look at 28 airplane state awareness accidents and incidents to determine how well the hazard alerting worked. We describe a five-step integrated alerting-to-recovery sequence that prescribes how hazard alerting should lead to effective flight crew actions for managing the hazard. Then, for each hazard in each of the 28 events, we determine if that sequence failed and, if so, how it failed. The results show that there was an alerting failure in every one of the 28 safety events, and that the most frequent failure (20/28) was tied to the flight crew not orienting to (not being aware of) the hazard. The discussion section summarizes findings and identifies alerting issues that are being addressed and issues that are not currently being addressed. We identify a few recent upgrades that have addressed certain alerting failures. Two of these upgrades address alerting design, but one response to the safety events is to upgrade training for approach to stall and stall recovery. We also describe issues that are not being addressed adequately: better alert integration for flight path management types of hazards, airplanes in the fleet that do not meet the current alerting regulations, a lack of innovation for addressing cases of channelized attention, and existing vulnerabilities in managing data validity.

aviation safety↗
NASA NTRS2019

Factors that Influenced Airplane State Awareness Accidents and Incidents: CAST SE-210 Output 2: Report 2 of 6

This report is part of a series of reports that address flight deck design and evaluation, written as a response to loss of control accidents. In particular, this activity is directed at failures in airplane state awareness in which the pilot loses awareness of the airplane’s energy state or attitude and enters an upset condition. The Commercial Aviation Safety Team (CAST) created a team to analyze a set of incidents and accidents associated with the flight crew’s loss of awareness of aircraft attitude or energy state. The CAST activity focused on a set of 18 loss of control (LOC) accidents and incidents. In those events, the pilot seemed to lose awareness of the state of the airplane, especially related to its energy state or its attitude. To understand the roots of that loss of awareness, we analyzed the events in relation to a set of factors that were present in many of them. We show how these factors contributed to lost awareness. Then, we use that understanding to identify potentially useful changes to the flight deck interface and how those flight deck changes could be evaluated.

commercial aviation↗
NASA NTRSDate not supplied

How Do Different Knowledge Frameworks Help Us Learn From Aviation Line Observations?

Human performance includes actions that increase safety, as well as actions that can reduce safety. Ensuring safety in complex dynamic operations like commercial aviation depends on the ability to institute appropriate responses based on what is learned from flightcrew performance and the contexts in which it occurs. To do this systemically at the organization level requires collecting data on flightcrew performance, developing effective approaches to analyzing those data, and understanding how to translate what has been learned into policies, procedures, and practice. Systematic observation of front-line operators is a vital source of human performance data. Much has been learned from such observations, including methodological principles. Most observations have been based on a framework focused on managing safety challenges and the ensuing unsafe events. A complementary perspective focuses on flexibility and actions that promote continued safe and effective operation. We consider lessons learned about observational methods from an established framework focused on undesired actions and how these might be extended for a framework focused on desired actions.

safety↗
NASA NTRSDate not supplied

Eye-Tracking Analysis from a Flight-Director-Use and Pilot-Monitoring Study

Eye tracking may be a useful tool to investigate pilot monitoring and develop and conduct training. There is increased interest from airlines to use eye-tracking technologies in flight simulators. However, much is still unknown about how to best utilize eye-tracking data in pilot training. This paper presents eye-tracking results from a pilot-monitoring training study with 19 pilots. All pilots completed 15 monitoring challenges across four operational scenarios in a B737-700 full flight simulator. In addition, the study investigated the impact of having the flight director engaged or disengaged on the pilot monitoring side in the final approach. It was hypothesized that pilots would focus less on the primary flight display with the flight director off and look more around in the cockpit. To assess this, pilots performed half of the scenarios with the flight director on and half with the flight director off. However, pilots monitoring tended to look less at the primary flight display with the flight director on as indicated by lower Proportion Dwell Times, contrary to the hypothesis. Next, eye-tracking data were analyzed from two monitoring challenges involving waypoint restrictions and two involving extending the flaps at appropriate airspeeds. Pilots that successfully completed the challenges appeared to focus more on areas of interest that contained the most relevant information to successfully complete the challenge. In addition, successful pilots seemed to adapt their monitoring strategy more to the challenge at hand as observed by a distinct shift in focus on either the primary flight display or the navigation display depending on the challenge. Our findings suggest the importance of flexible gaze allocation across specific situations and raise the question whether and to what degree prespecified patterns of eye fixation can be identified and trained.

eye tracking↗
NASA NTRSDate not supplied

Eye-Tracking Analysis from a Flight-Director-Use and Pilot-Monitoring Study

Eye tracking may be a useful tool to investigate pilot monitoring and develop and conduct training. There is increased interest from airlines to use eye-tracking technologies in flight simulators. However, much is still unknown about how to best utilize eye-tracking data in pilot training. This paper presents eye-tracking results from a pilot-monitoring training study with 19 pilots. All pilots completed 15 monitoring challenges across four operational scenarios in a B737-700 full flight simulator. In addition, the study investigated the impact of having the flight director engaged or disengaged on the pilot monitoring side in the final approach. It was hypothesized that pilots would focus less on the primary flight display with the flight director off and look more around in the cockpit. To assess this, pilots performed half of the scenarios with the flight director on and half with the flight director off. However, pilots monitoring tended to look less at the primary flight display with the flight director on as indicated by lower Proportion Dwell Times, contrary to the hypothesis. Next, eye-tracking data were analyzed from two monitoring challenges involving waypoint restrictions and two involving extending the flaps at appropriate airspeeds. Pilots that successfully completed the challenges appeared to focus more on areas of interest that contained the most relevant information to successfully complete the challenge. In addition, successful pilots seemed to adapt their monitoring strategy more to the challenge at hand as observed by a distinct shift in focus on either the primary flight display or the navigation display depending on the challenge. Our findings suggest the importance of flexible gaze allocation across specific situations and raise the question whether and to what degree prespecified patterns of eye fixation can be identified and trained.

eye tracking↗
NASA NTRSDate not supplied

Exploring Informal Learning at the Airlines

Airline pilot training is extensive, highly structured, and defined by aircraft and airspace system operating requirements, yet pilots describe a tradition of between-pilot knowledge transfer and self-directed learning. This learning supplements their approved training programs. While industry and regulators focus on “formal learning” systems, pilots report relying on “informal learning” to build operational expertise. The persistence of informal learning suggests gaps in how successfully formal learning prepares pilots to handle operational complexities. The community that researches learning has extensively studied informal learning, and its characteristics seem to align with how pilots report increasing their skills and knowledge informally. However, no research into informal learning practices among airline pilots seems to exist. In this paper we provide examples of informal learning in commercial aviation, how they fit into two existing frameworks for workplace learning, and propose that researching informal learning might help identify opportunities to improve formal aviation learning systems.

pilot learning↗
NASA NTRSDate not supplied

Statistical Analysis of Recent Go Around Flight Data to Study and Evaluate Skilled Pilot Monitoring

A statistical analysis was performed on a batch of recent go-around flight data of a few major US airlines spanning a one year period. The statistical analysis investigates a few select relevant aspects of these go arounds, such as initiation altitude and altitude change, aircraft configuration, deviation from the flightpath, and energy state. These data show that only one third of the landings are aborted below 400ft, but the majority of the go arounds occur at significantly higher altitudes. Current go-around training is specifically focused on the final approach and landing phase at low altitudes and low speeds. The results of this study will support the development of a broader set of relevant scenarios for an FAA simulator experiment to study and evaluate the skills of the pilots who monitor this wide spectrum of diverse go-around conditions, with the ultimate goal to help improve pilot go-around training and technologies.

go around↗
NASA NTRSDate not supplied

Scenario-Based Task Design for Airline Pilot Anticipatory Behaviors: Asynchronous Assessment of Complex Cognitive Skills

Airline pilots must anticipate potential threats to passenger safety and efficient flight. Such anticipation needs to occur at both the system and individual levels, yet no formal training for these anticipatory behaviors currently exists. The emerging field of resilience engineering provides a framework to explore this problem. Given the complex nature of this problem, the assessment portion of the project constituted a challenge. Simulator time is expensive as simulators are costly to run, and they are extremely limited in number. Also, by the nature of their work, pilots as a group are hard to connect with in person; thus, asynchronous methods were adopted. Further, to ensure that authentic pilot behaviors were observed, task-based scenarios were used rather than focus group or interview protocols. This session explores the design of asynchronous learning and a measure for assessing pilot anticipatory behaviors outside of a simulator to help design learning opportunities for pilots. The scenario-based and active assessment items will be discussed in detail, and demonstrations of these items will be shared. These items constitute the bulk of the challenge for this work. At the time of this writing, data is being collected to assess the effectiveness of the interactive training module and validate the measures. This data will be analyzed and presented as part of the session. This will include any linkages between the scenario-based and traditional assessment items.

task design↗