Search NASA⌕ Search

Engineering topics

Jon Holbrook

Publications and source records attributed to Jon Holbrook.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Benchmark Problem Development for Testing Maturity of Intelligent Contingency Management Tools

Increasingly autonomous Advanced Air Mobility (AAM) vehicles will be required to handle diverse conditions with limited human intervention. Intelligent contingency management (iCM) approaches are under development to address how automated agents can handle unforeseen, unplanned, and unanticipated events. Benchmark scenario is needed to test the maturity of the developed iCM tools and techniques.

Jon Holbrook↗
NASA NTRSDate not supplied

Benchmark Problem Development for Testing Maturity of Intelligent Contingency Management Tools

This paper presents a process used to develop appropriate scenarios and metrics for evaluating the maturity of intelligent contingency management algorithms. A benchmark scenario is a reference point against which something can be measured, compared, or assessed. Creating an accurate benchmark requires considerable research and expertise. The scenario itself is an artificial representation of a real-world event, designed to achieve a set of learning objectives through experiential learning. Designing an effective benchmark simulation scenario requires careful planning, including identification of clear objectives; capability assessment of the algorithm/tool being evaluated; assessment of necessary levels of fidelity; development of a process flow map of events and event interactions; and identification of metrics that map back to objectives. Thus, a benchmark scenarios for contingency management might consist of one or several commonly used functions taken from real world applications, used for evaluation, characterization and performance measurement of a contingency management algorithm. Behavior of the contingency management algorithm under different environmental conditions should then be able to be predicted using a set of benchmark functions. The paper describes the resulting benchmark problem as an illustration of the application of this process.

Jon Holbrook↗
NASA NTRSDate not supplied

Human Contribution to Safety: Human Performance is Not Just About Error

- When the only data that are available are about human failure, then data-driven designs only consider that humans fail. - Designs intended to "protect" the system from "error-prone" humans can design-out the capability for humans to effectively intervene/adapt, which is a far more common behavior.

Jon Holbrook↗
NASA NTRSDate not supplied

Learning About Routine Successful Pilot Techniques Using A Cued Retrospective Think-Aloud Task

Self-report can be a valuable method for collecting data about people’s goals and perceived motivations – data about aspects of crew thinking that are not otherwise readily observable. One of the challenges associated with collecting self-report data on routine successful performance, however, is that details may go unreported, be deemed unimportant, or may not be recalled. We report a study in which commercial airline flight crews participated in a video-cued retrospective think aloud after flying a high-fidelity simulated arrival into Charlotte airport. One day after flying the simulated arrival, crews were shown a video recording of their flight. The video was paused after each minute, and crew members were each asked to describe what they were doing and thinking during that interval. Reported data analysis focused on aspects of performance that are often ambiguously described as “pilot technique” or “airmanship,” in an attempt to provide more detail around these types of behaviors.

Jon Holbrook↗
NASA NTRSDate not supplied

Leveraging Human Performance Data to Change the Narrative that People are the Safety Problem

The study of errors and failure has a long and productive history in the behavioral sciences. By studying how systems fail, we rule out various mechanisms for how those systems might work, thereby refining our theories of how they actually work. Human performance, however, includes more than errors; human performance comprises both failures and successes. A systematic bias to collect and analyze data only on error affects the decisions we make as a community by promoting the narrative that “people are the safety problem.” This narrative manifests in both obvious and subtle ways in the design of systems intended for human use. When the only safety data that are available are about human failure, then “data-driven” designs can only consider that humans fail. Changing this narrative will depend on new data and new ways to examine data – specifically, data on the processes by which human create and contribute to safety. An alternate narrative is that people represent a primary source of safety, through their capability to anticipate, monitor for, respond to, and learn from expected and unexpected change. This presentation will describe research efforts to expand the range of safety-relevant events to include not just rare safety failures but frequent safety successes. These efforts include use of data from both operations and simulations to develop methods and metrics for learning from structured observation, self-report, and system data.

Jon Holbrook↗
NASA NTRSDate not supplied

Examining the Relationship Between Workload and Resilient Performance in Airline Flight Crews

Workload has long been associated with human performance in aviation. High workload is typically viewed as a system design problem. The aim of this work was to find observable behaviors that guard cognitive resource margins; thus, improve handling of perturbations that arise. Twelve commercial airline flight crews participated in an experiment at NASA Langley Research Center. We explored workload data (NASA Task Load Index) self-reported after completing flight simulation scenarios to see how different individuals, using the same technologies, cope with challenging situations. A trained observer from the same airline as the study participants assessed performance. We differentiated a lower workload crew from a higher workload crew and analyzed their workload and performance. Results indicate that the workload may be a component of pilots’ resilient performance. Initial strategies that reduced workload were classified and these types of behaviors, if implemented correctly, might assist pilots in maintaining functional workload resource margins.

Michael Stewart↗
NASA NTRSDate not supplied

Why Learning From All Operations Is Imperative

Learning from All Operations – Panel Discussion Introduction: Tzvetomir Blajev, Flight Safety Foundation Why Learning from All Operations is Imperative: Jon Holbrook, NASA Implemented Approaches to Learning from All Operations American Airlines Japan Airlines Southwest Airlines Delta Air Lines Why Learning from All Operations is Imperative (Summary): Immanuel Barshi, NASA

Jon Holbrook↗
NASA NTRSDate not supplied

The Unintended Consequences of Focusing on Human Error (And How You Can Help)

The literature on human performance is rich with findings of cognitive failures and methods to identify, label, and measure them. In many real-world contexts, however, outcomes are driven far more by successful than failed cognition. Designers of systems intended for human use, in an effort to be “data driven,” rely upon findings from the cognitive performance literature to inform their system designs. When most available data are about human error, data-driven designs focus on the human primarily as a source of failure. Designs intended to support or replace humans often fail to acknowledge or understand the capabilities that humans routinely contribute to successful performance. Consequently, designs intended to “protect” the system from “error-prone” humans can design-out the capability for the human to effectively intervene or adapt. The development of paradigms to study successful human performance represents a significant and largely untapped opportunity for research in cognition.

Jon Holbrook↗