Search NASA⌕ Search

Engineering topics

Immanuel Barshi

Publications and source records attributed to Immanuel Barshi.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2023

Operations: A Wholistic View

An operation is often constructed of discrete parts. These parts are often designed, developed, and practiced independently of each other. However, to work together effectively and efficiently, these parts must constitute a unified whole. To achieve its operational goals, this wholeness must be guided by a clear, coherent, consistent and comprehensive framework of procedures and policies. In this talk, I describe such a framework, motivate its structure, and demonstrate its effectiveness.

operations↗
NASA NTRS2022

Assessing Several Non-Traditional Data Sources for Value in Aviation Safety

The NASA System-Wide Safety (SWS) project and its predecessor projects have been developing Machine Learning (ML) algorithms for commercial aviation safety for many years. These algorithms have been applied to Flight Operations Quality Assurance (FOQA); radar track data (e.g., Threaded Track); and safety reports, including Aviation Safety Reporting System (ASRS) and Aviation Safety Action Plan (ASAP). SWS is working with partners to get access to other data that air carriers provide, such as maintenance data, and has been assisting carriers in working with other data, such as Line Operations Safety Audit (LOSA) data, using manual methods. However, the project has discussed whether there are other data that are not traditionally used in aviation safety analysis that may be useful. This paper discusses four sets of data and models that are not traditionally used in aviation safety but that have shown promise for such use. In the future, we plan to incorporate such data into ML algorithms to use with data that we have used before and determine the additional benefit that is actually achieved under different contexts from the inclusion of these non-traditional data sources.

Nikunj C. Oza↗
NASA NTRS2020

Effects of Long-Duration Space Flight on Training Retention and Transfer

The space environment imposes on the astronaut crew significant physiological, psycho-social, and cognitive loads that can not be replicated on the ground. These loads likely impact crew performance. To date, no systematic data collection has taken place to understand the effects of such loads on crew members’ ability to retain trained knowledge and skills, and to transfer such knowledge and skills to novel situations. The research described here was originally requested by HRP management to be the first such study to systematically collect data on the effects of long duration space flight on training retention and transfer. Because current theories of retention and transfer are based on results obtained in university laboratories using undergraduate students as research participants, and because crew time in space is very expensive, this study was designed to compare the performance of 4 groups of subjects: crew members in space, crew members on the ground, crew-like subjects, and university undergraduate students. Results from the ground-phase of the study reported here demonstrate that crew members’ performance under cognitive load can not be predicted from the performance of university undergraduate students. It is still an open question the extent to which crew members’ cognitive performance in space can be predicted from the performance of crew members on the ground.

training↗
NASA NTRSDate not supplied

Design for Operations

Designs are usually done in a well-lit, air conditioned, comfortable room under idealized assumptions. The reality of the operation is very different. For designers to be able to design systems and procedures that work well under realistic dynamic work conditions, they must understand the reality of the operation and of the operators. The talk discusses the gap between the idealized design and the real operation and proposes a framework, THE Model, for designing systems and procedures fit for real operations.

design↗
NASA NTRSDate not supplied

From the Field to the Lab and Back Again

University-based research is often driven by questions of theory and is carried out in artificial manner. Applied research is aimed at solving practical problems in ways that are informed by the theories and in a manner that is ecologically valid. The talk discusses the two approaches and describes two studies in which the practical problem drove laboratory studies resulting in practical solutions.

experimental design↗
NASA NTRSDate not supplied

Optimizing Procedures

Procedures designs are usually done in a well-lit, air conditioned, comfortable room under idealized assumptions. The reality of the operation is very different. For designers to be able to design procedures that work well under realistic dynamic work conditions, they must understand the reality of the operation and of the operators. The talk discusses the gap between the idealized design and the real operation and proposes a framework, THE Model, The 4Cs and the 4Ps, for designing procedures fit for real operations.

design↗
NASA NTRSDate not supplied

Operations. A Wholistic View

An operation is often constructed of discrete parts. These parts are often designed, developed, and practiced independently of each other. However, to work together effectively and efficiently, these parts must constitute a unified whole. To achieve its operational goals, this wholeness must be guided by a clear, coherent, consistent and comprehensive framework of procedures and policies. In this talk, I describe such a framework, motivate its structure, and demonstrate its effectiveness.

operations↗
NASA NTRSDate not supplied

Multitasking

Multitasking is endemic in modern life and work: drivers talk on cell phones, office workers type while answering phone calls, students do homework while text messaging, nurses prepare injections while responding to doctor’s calls, and air traffic controllers direct aircraft in one sector while handling additional traffic in another. Whether in daily life or at work, we are constantly bombarded with multiple concurrent demands, and we have all somehow come to believe in the myth that we can, and in fact are expected to, easily address them all - without any repercussions. However, accumulating scientific evidence is now suggesting that multitasking increases the probability of human error. This talk presents a set of NASA studies that characterize concurrent demands in routine airline flight operations, in order to illustrate the ways operational task demands together with the proclivity to manage them all concurrently make human performance in this and in any domain vulnerable to potentially serious errors and to accidents.

multitasking↗
NASA NTRSDate not supplied

Astronauts' Performance: The Retention and Transfer of Training

The space environment imposes on the astronaut crew significant physiological, psycho-social, and cognitive loads that can not be replicated on the ground. These loads likely impact crew performance. To date, no systematic data collection has taken place to understand the effects of such loads on crew members’ ability to retain trained knowledge and skills, and to transfer such knowledge and skills to novel situations. The research described here was originally designed to be the first such study to systematically collect data on the effects of long duration space flight on training retention and transfer. Because current theories of retention and transfer are based on results obtained in university laboratories using undergraduate students as research participants, and because crew time in space is very expensive, this study was designed to compare the performance of 4 groups of subjects: crew members in space, crew members on the ground, crew-like subjects, and university undergraduate students. Results from the ground-phase of the study reported here demonstrate that crew members’ performance under cognitive load can not be predicted from the performance of university undergraduate students. It is still an open question the extent to which crew members’ cognitive performance in space can be predicted from the performance of crew members on the ground.

training↗
NASA NTRSDate not supplied

Reports of Resilient Performance: Investigating Operators' Descriptions of Safety-producing Behaviors in the Aviation Safety Reporting System

While many existing taxonomies and frameworks provide a common vocabulary for describing how human operators fail in the context of sociotechnical systems, at present, there is no common vocabulary to describe how humans succeed. Such a framework would facilitate systematically collecting and analyzing data on how human performance can produce safety, not just how it can reduce safety. One potentially rich source of currently available information for exploring desired performance is the reports submitted to NASA’s Aviation Safety Reporting System (ASRS). These de-identified, confidential, and voluntary narrative reports are submitted by pilots, controllers, ground operators, and others within aviation operations. While these reports are primarily submitted to describe safety risks, incidents, and problems, they also often describe how those risks were mitigated, and provide a window into aspects of everyday work in aviation. This paper describes an analysis of ASRS narratives to understand how operators talk about their own resilient behaviors during adverse safety conditions and events. Guided by Erik Hollnagel’s Resilience Assessment Grid framework (i.e., anticipate, monitor, respond, learn), we illustrate our approach and methodology with examples from reports. We also highlight some of the challenges and how further research is needed in developing a taxonomy of operators’ descriptions of resilient performance.

resilient behaviors↗
NASA NTRSDate not supplied

Reports of Resilient Performance: Investigating Operators' Descriptions of Safety-producing Behaviors in the Aviation Safety Reporting System

While many existing taxonomies and frameworks provide a common vocabulary for describing how human operators fail in the context of sociotechnical systems, at present, there is no common vocabulary to describe how humans succeed. Such a framework would facilitate systematically collecting and analyzing data on how human performance can produce safety, not just how it can reduce safety. One potentially rich source of currently available information for exploring desired performance is the reports submitted to NASA’s Aviation Safety Reporting System (ASRS). These de-identified, confidential, and voluntary narrative reports are submitted by pilots, controllers, ground operators, and others within aviation operations. While these reports are primarily submitted to describe safety risks, incidents, and problems, they also often describe how those risks were mitigated, and provide a window into aspects of everyday work in aviation. These reports can be searched in a variety of ways. This paper describes methods for systematically examining ASRS narratives to understand how operators talk about their own resilient behaviors during adverse safety conditions and events. Guided by Erik Hollnagel’s Resilience Assessment Grid framework (i.e., anticipate, monitor, respond, learn), various approaches and tools for such inquiries are described. The approach, process, challenges, and suggestions to building an operator-based description of resilient performance are discussed, and tools to facilitate data analysis to maximize learning from these reports are described.

ASRS narratives↗
NASA NTRSDate not supplied

An Approach to Identifying Aspects of Positive Pilot Behavior within the Aviation Safety Reporting System

The National Airspace System (NAS) is constantly evolving as air traffic continues to ramp up to pre-pandemic numbers and projected to grow to unprecedented levels in the coming years. As well as increasing demand to the current system, emerging operations such as Unmanned Autonomous Systems are also expected to add to complexity in the airspace. To address these issues, the industry and government agencies supporting the NAS will need to rely upon additional automation and new technologies to address future operational requirements, while continuing to be a world-leading safe transportation system. As these new technologies are implemented, the system continues to rely on human pilots and controllers in the loop to monitor the system and intervene in situations the automation cannot handle. The goal of proactively addressing safety is of foremost concern to ensure passenger confidence. The industry has implemented various Safety Monitoring Systems to identify safety risks and proactively address them before they result in a serious incident or accident. One such program is the Aviation Safety Reporting System (ASRS). ASRS is a long-established system where pilots and controllers voluntarily and anonymously report safety incidents they experienced and observed during line operations by providing rich text narratives describing the events, the environment, and conditions leading to the safety event of concern. These narratives provide insight and context around events of interest and can be used to identify emerging problems. They can trigger investigations within Flight Operational Quality Assurance or Flight Data Monitoring programs. However, this process typically focuses on the adverse events and the unsafe aspects of the operations surrounding the reported or detected events. This perspective of investigating factors that went wrong around an adverse event is commonly referred to as Safety I. Alternatively, characterizing successful actions that operators perform every day under varying conditions that keep the system within safe operating bounds is a concept referred to as Safety II. The benefit of the Safety II view is that the scope is much larger than that of Safety I since a vast majority of the operations result in successful flights. Many of the successful techniques used to manage operational threats are not documented in standard operating procedures or taught during training. They are typically acquired over time by working with experienced pilots during line operations or in many cases after experiencing a problem for the first time and reacting to it in situ, drawing from years of experience to manage the threat. In an attempt to quantify these positive actions, we are proposing an approach to extracting key behaviors within ASRS reports that can support the Safety II concept. Our analysis assumes that ASRS reports contain some descriptions of corrective actions that operators performed to prevent a situation from leading to an accident. Leveraging recent advances in Natural Language Process modeling, we have developed an approach to extract positive sentiment from reports, embed these positive statements in a vector space where they can be numerically analyzed, and clustering these statements into similar contextual categories. From these contextualized categories we can attempt to summarized and distilled aspects of the positive behavior. The goal is to identify categories of behavior that describe consistent operator techniques that supports the Safety II concept. With this information, airlines may enable learning from these positive actions, or address procedures that need to be changed to avoid having pilots implement a workaround. These insights can provide a lens into what is “going right” in the operations that may otherwise not be known widely within the community. It is envisioned that this approach can be extended to other narrative programs such as Line Operation Safety Audit or Learning Improvement Team reports where similar observed behavior can be analyzed to extract positive actions and inform the overall operations.

NLP↗
NASA NTRSDate not supplied

Extracting Lessons of Resilience Using Machine Mining of the ASRS Database

NASA’s Aviation Safety Reporting System (ASRS) database is the world's largest repository of voluntary, confidential safety information provided by aviation's frontline personnel, including pilots, air traffic controllers, mechanics, flight attendants, dispatchers, and other members of the aviation community and the public. The database contains close to 2 million narratives, many of which describe everyday situations in which people saved the day. In these situations, people’s resilient behavior solved a problem, dealt with a malfunction, and maintained a safe operation despite a serious perturbation. To be able to extract lessons of such resilience from this large database, the use of machine learning algorithms is being explored. In this report, we describe a comparison between two such algorithms: Perilog and Word2Vec. An identical search using both programs was done on a database containing approximately 470,000 ASRS reports submitted between 1988 and 2022. The comparison reveals some of the strength and weaknesses of each algorithm as well as the challenges inherent in using such algorithms to extract lessons of resilience from the ASRS database.

resilience↗
NASA NTRSDate not supplied

Analyses of the Boeing 737-MAX Accidents: Formal Models and Psychological Perspectives

Two fatal accidents involving the B737MAX resulted from the flight crews’ inability to overcome the effects of the Maneuvering Characteristics Augmentation System (MCAS). MCAS was designed to mimic the control column feel pressure and pitching behavior of the B737NG, which was the certification basis for the B737MAX. We briefly describe the potential role of formally modeling different perspectives during system design, and how such modeling can reveal gaps and conflicts between perspectives. We also discuss some of the relevant human factors issues involved in these accidents and how the aircraft’s behavior may have affected the pilots’ psychological states. Implications for automation design are considered.

5737MAX↗
NASA NTRSDate not supplied

A Holistic Approach to Procedures

Aviation is a dynamic industry which is constantly changing. These changes require the continuous update of people’s knowledge and skills. A fundamental part of that knowledge and skill is procedures, because procedures form the backbone of aviation operations. On the flightdeck, on the ramp, and in the maintenance shop, procedures and checklists help support pilots, line crews, and mechanics in performing their work effectively, efficiently, and thus -- safely. To design effective and efficient procedures and checklists, one must take into account the full operational context within which these procedures are embedded. This context is defined by the requirements of the technology, the capabilities and limitations of the human operators, and the constraints and affordances of the operational environment. The complexity of this context arises from the interactions of the human, machine, and environment. Procedures are in place to govern those interactions. We present a model of that operational context, namely THE Model (Technology, Human, Environment in the context of a Mission), that lays a foundation for the analysis of each of these elements and their interactions. But procedures have their limitations, and there are risks involved in over-proceduralizing. To understand these limitations and risks, and to determine when to proceduralize and when not, we present the 4P Framework (Practice, Procedures, Policies, Philosophy) as a holistic approach to procedures. This approach brings together the understanding of human behavior, of organizational and operational factors, and of the technologies involved.

procedures↗