Managing Failure by Design In Sociotechnical Systems
Explore the source record for details and available documents.
Jon Holbrook↗
SEARCH · Search NASA
Results for “Sociotechnical systems”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Reports of Resilient Performance: Investigating Operators' Descriptions of Safety-producing Behaviors in the Aviation Safety Reporting System
While many existing taxonomies and frameworks provide a common vocabulary for describing how human operators fail in the context of sociotechnical systems, at present, there is no common vocabulary to describe how humans succeed. Such a framework would facilitate systematically collecting and analyzing data on how human performance can produce safety, not just how it can reduce safety. One potentially rich source of currently available information for exploring desired performance is the reports submitted to NASA’s Aviation Safety Reporting System (ASRS). These de-identified, confidential, and voluntary narrative reports are submitted by pilots, controllers, ground operators, and others within aviation operations. While these reports are primarily submitted to describe safety risks, incidents, and problems, they also often describe how those risks were mitigated, and provide a window into aspects of everyday work in aviation. This paper describes an analysis of ASRS narratives to understand how operators talk about their own resilient behaviors during adverse safety conditions and events. Guided by Erik Hollnagel’s Resilience Assessment Grid framework (i.e., anticipate, monitor, respond, learn), we illustrate our approach and methodology with examples from reports. We also highlight some of the challenges and how further research is needed in developing a taxonomy of operators’ descriptions of resilient performance.
resilient behaviors↗
Reports of Resilient Performance: Investigating Operators' Descriptions of Safety-producing Behaviors in the Aviation Safety Reporting System
While many existing taxonomies and frameworks provide a common vocabulary for describing how human operators fail in the context of sociotechnical systems, at present, there is no common vocabulary to describe how humans succeed. Such a framework would facilitate systematically collecting and analyzing data on how human performance can produce safety, not just how it can reduce safety. One potentially rich source of currently available information for exploring desired performance is the reports submitted to NASA’s Aviation Safety Reporting System (ASRS). These de-identified, confidential, and voluntary narrative reports are submitted by pilots, controllers, ground operators, and others within aviation operations. While these reports are primarily submitted to describe safety risks, incidents, and problems, they also often describe how those risks were mitigated, and provide a window into aspects of everyday work in aviation. These reports can be searched in a variety of ways. This paper describes methods for systematically examining ASRS narratives to understand how operators talk about their own resilient behaviors during adverse safety conditions and events. Guided by Erik Hollnagel’s Resilience Assessment Grid framework (i.e., anticipate, monitor, respond, learn), various approaches and tools for such inquiries are described. The approach, process, challenges, and suggestions to building an operator-based description of resilient performance are discussed, and tools to facilitate data analysis to maximize learning from these reports are described.
ASRS narratives↗
Preliminary Work Domain Analysis for Human Extravehicular Activity
A work domain analysis (WDA) of human extravehicular activity (EVA) is presented in this study. A formative methodology such as Cognitive Work Analysis (CWA) offers a new perspective to the knowledge gained from the past 50 years of living and working in space for the development of future EVA support systems. EVA is a vital component of human spaceflight and provides a case study example of applying a work domain analysis (WDA) to a complex sociotechnical system. The WDA presented here illustrates how the physical characteristics of the environment, hardware, and life support systems of the domain guide the potential avenues and functional needs of future EVA decision support system development.
McGuire, Kerry↗
NextGen Future Safety Assessment Game
The successful implementation of the next generation infrastructure systems requires solid understanding of their technical, social, political and economic aspects along with their interactions. The lack of historical data that relate to the long-term planning of complex systems introduces unique challenges for decision makers and involved stakeholders which in turn result in unsustainable systems. Also, the need to understand the infrastructure at the societal level and capture the interaction between multiple stakeholders becomes important. This paper proposes a methodology in order to develop a holistic approach aiming to provide an alternative subject-matter expert (SME) elicitation and data collection method for future sociotechnical systems. The methodology is adapted to Next Generation Air Transportation System (NextGen) decision making environment in order to demonstrate the benefits of this holistic approach.
Ancel, Ersin↗
NextGen Future Safety Assessment Game
The successful implementation of the next generation infrastructure systems requires solid understanding of their technical, social, political and economic aspects along with their interactions. The lack of historical data that relate to the long-term planning of complex systems introduces unique challenges for decision makers and involved stakeholders which in turn result in unsustainable systems. Also, the need to understand the infrastructure at the societal level and capture the interaction between multiple stakeholders becomes important. This paper proposes a methodology in order to develop a holistic approach aiming to provide an alternative subject-matter expert (SME) elicitation and data collection method for future sociotechnical systems. The methodology is adapted to Next Generation Air Transportation System (NextGen) decision making environment in order to demonstrate the benefits of this holistic approach.
Ancel, Ersin↗
Crew collaboration in space: a naturalistic decision-making perspective
Successful long-duration space missions will depend on the ability of crewmembers to respond promptly and effectively to unanticipated problems that arise under highly stressful conditions. Naturalistic decision making (NDM) exploits the knowledge and experience of decision makers in meaningful work domains, especially complex sociotechnical systems, including aviation and space. Decision making in these ambiguous, dynamic, high-risk environments is a complex task that involves defining the nature of the problem and crafting a response to achieve one's goals. Goal conflicts, time pressures, and uncertain outcomes may further complicate the process. This paper reviews theory and research pertaining to the NDM model and traces some of the implications for space crews and other groups that perform meaningful work in extreme environments. It concludes with specific recommendations for preparing exploration crews to use NDM effectively.
Review, Tutorial↗
Collective Intelligence and Three Aspects of Planning in Organizations: A NASA Example
For many complex sociotechnical systems, planning seems to require explicit coordination; certainly, in executing a plan the activities of different actors must be tightly coupled. However, distributing the needed planning information can be very burdensome and error prone, because different groups need different collections of information, updated or kept current on different time cycles. Further, the information needed to form successful plans is often highly distributed, and while feedback about the success of prior plans may exist, it may not be available to those in a position of using this to improve plans or to detect and resolve other problems in the system (Weick, 1995). Tools to support various aspects of planning have been developed, and can provide a huge benefit to the individuals working on that aspect. To be tractable, most solutions address a quite bounded slice of work, isolating it from the larger context. Prospective planning takes place over multiple, nested cycles of decision making. This builds a plan that specifies activities of different granularity. "Subplans" may specify multiple parallel activities by different groups and individuals, as well as sequential, nested actions by a single actor. Planning produces valuable, sharable, external, representations: in addition to prospective use, plans support retrospective assessment and also action in the present. Viewing planning in a larger context - both temporal and organizational -- enables noticing what one does not know and generating more systemic and effective solutions. Viewing a problem as one of collective intelligence invites thinking about the larger organizational context. Many approaches to supporting collective intelligence do not support execution of highly contingent actions, distributed across many players, and hence provide incomplete support for planning. However, CI technology maybe helpful in managing the processes of gathering information for decision making in planning and of distributing plan information to various actors, needed on various time cycles.
Billman, Dorrit↗
Evaluating Direct and Indirect Influence on EV Charging Stations Across the US
The adoption of new technology for electric vehicles (EV) and mobility applications can bring underappreciated vulnerabilities to the power grid. One area of potential fraud and adversarial influence is through the business ecosystem of startups that own and deploy EV technology. Yet, there are no models or analyses that map the network of organizations and people that have direct and indirect influence over technologies currently deployed in the grid. To fill this gap, we develop a multilayer network model to measure direct and indirect influence on EV charging stations. First, we create and adversarial socio-technical network (ASTN) model via a data fusion pipeline for different US regions of interest (ROI). Then, we develop an integrated ASTN for Chicago, Los Angeles, New York, and Philadelphia. We rank EV charging companies direct influence within each geographic region as well as indirect influence via social network analysis. While some companies have strong direct and indirect influence (i.e., ChargePoint) others show a mismatch between their influence over charging stations and their position within the social network. For example, Tesla has strong direct influence on stations and weak indirect influence over competitors. In contrast, 7Charge has weak direct influence over stations, but strong indirect influence over competitors.
29 - ENERGY PLANNING, POLICY AND ECONOMY↗
Visualizing Organizational Influence on Energy Infrastructure
Energy Infrastructure components depend on an evolving, interdependent business ecosystem exposed to long-term, legal, adversarial tactics. An INL-Naval Postgraduate School partnership was designed to support INL Lab Directed Research and Development, NPS graduate research projects, and joint publications. The Technology, Organization, and Person of interest Graph Extraction, Analysis, and Reporting (TOP GEAR) enumerates networks of organizations and people that own, operate, and maintain regional infrastructure assets. TOP GEAR allows analysts to model current and future state what-if scenarios that include technological and policy mitigations.
29 - ENERGY PLANNING, POLICY AND ECONOMY↗
Realizing a Spacesuit Compatible Augmented Reality System to Meet the Work Needs of Future Human Spaceflight Exploration
In this paper, we identify the broad range of challenges associated with creating an augmented reality for future spaceflight missions. The challenges were framed within a sociotechnical framework to explore how an imagined augmented reality system might one day help crew cope with future exploration mission demands. Key challenges include an augmented reality system cannot simply be designed to help a single suited astronaut and must enable the broader mission team. There exists a host of optical, compute, and control technical challenges that must be simultaneously addressed while the work domain itself is in development. There exist limited methods to help cope with the envisioned world problem; however, we emphasize a few strategies to articulate work design assumptions and host this detail in a transparent fashion. Finally, we note the broader organizational constraints that shape the creation of such novel technology for spaceflight. By addressing these broad range of challenges directly, early in the design and development process, key trade-offs can be considered and fundamental concerns within the community can be addressed to yield desired mission outcomes.
Matthew J. Miller↗
Using Virtual Reality to Envision Deployment of Spacesuit-Compatible Augmented Reality Displays for Lunar Surface Operations
The National Aeronautics and Space Administration (NASA) will soon land crew on the lunar surface to establish a sustainable presence and develop operational concepts for future long-duration missions. New technologies will be necessary to extend planning and execution capabilities for lunar surface activities. The Joint Augmented Reality Visual Informatics System (Joint AR) at NASA Johnson Space Center (JSC) is one such technology. Joint AR is a suit-mounted augmented reality (AR) display and compute system which facilitates unprecedented information exchange and data visualization capabilities between mission support operators and suited crew. This paper describes challenges associated with developing an AR technology for an envisioned work domain by applying a sociotechnical lens to the iterative testing and development of novel AR technology through virtual reality (VR). A foundational VR testbed was established, providing a high-fidelity approximation of the lunar surface and enabling testing of envisioned AR system features in parallel with real world product development. Using this environment, a series of human-in-the-loop experiments were conducted using VR to deploy a notional AR system supporting use cases envisioned for lunar exploration extravehicular activity (xEVA). Our findings indicate VR is a powerful and immersive tool for testing capabilities which extend beyond the limitations of current AR technology. Our VR testbed enables early testing of proposed system features, advancing identification of high-value features and driving present-day development of Joint AR. Future work directions are discussed, including lessons learned and a roadmap for an iterative research and development approach applying VR (among other testbeds) to accelerate Joint AR system maturation.
Augmented reality↗
Using Virtual Reality to Envision Deployment of Spacesuit-Compatible Augmented Reality Displays for Lunar Surface Operations
The National Aeronautics and Space Administration (NASA) aims to land crew on the lunar surface to establish a sustainable presence and develop operational concepts for future long-duration missions. New technologies will be necessary to extend planning and execution capabilities for lunar surface activities. NASA’s Joint Augmented Reality Visual Informatics System (Joint AR) is one such technology. Joint AR is a suit-mounted augmented reality (AR) display and computes system which facilitates unprecedented information exchange and data visualization capabilities between mission support operators and suited crew. This paper describes challenges associated with developing AR technology for an envisioned work domain by applying a sociotechnical lens to the iterative testing and development of novel AR technology through virtual reality (VR). A VR testbed was established to simulate a representative lunar surface environment, enabling a series of three human-in-the-loop (HITL) tests evaluating AR navigation interfaces for exploration extravehicular activity (xEVA). Our findings identify several considerations for future Joint AR design and testing efforts, including challenges with data overload, attentional demands, and environment-related perceptual challenges. Trade-offs and potential approaches are discussed to mitigate these challenges and improve future Joint AR testing fidelity.
Jacob Keller↗
Using Virtual Reality to Envision Deployment of Spacesuit-Compatible Augmented Reality Displays for Lunar Surface Operations
The National Aeronautics and Space Administration (NASA) aims to land crew on the lunar surface to establish a sustainable presence and develop operational concepts for future long-duration missions. New technologies will be necessary to extend planning and execution capabilities for lunar surface activities. NASA’s Joint Augmented Reality Visual Informatics System (Joint AR) is one such technology. Joint AR is a suit-mounted augmented reality (AR) display and computes system which facilitates unprecedented information exchange and data visualization capabilities between mission support operators and suited crew. This paper describes challenges associated with developing AR technology for an envisioned work domain by applying a sociotechnical lens to the iterative testing and development of novel AR technology through virtual reality (VR). A VR testbed was established to simulate a representative lunar surface environment, enabling a series of three human-in-the-loop (HITL) tests evaluating AR navigation interfaces for exploration extravehicular activity (xEVA). Our findings identify several considerations for future Joint AR design and testing efforts, including challenges with data overload, attentional demands, and environment-related perceptual challenges. Trade-offs and potential approaches are discussed to mitigate these challenges and improve future Joint AR testing fidelity.
Matthew Miller↗
Cockpit checklists - Concepts, design, and use
Although the aircraft checklist has long been regarded as a foundation of pilot standardization and cockpit safety, it has escaped the scrutiny of the human factors profession. The improper use, or nonuse, of the normal checklist by flight crews is often cited as a major contributing factor to aircraft accidents. This paper reports the results of a field study of flight deck checklists and examines this seemingly mundane yet critical device from several perspectives: its functions, format, design, length, and usage, and the limitations of the humans who must interact with it. Certain sociotechnical factors, such as the airline 'culture', cockpit resource management, and production pressures that influence the design and use of this device, are also discussed. Finally, a list of design guidelines for normal checklists is provided. Although the focus of this paper is on the air transport industry, most of the principles discussed apply equally well to other high-risk industries, such as maritime transportation, power production, weapons systems, space flight, and medical care.
Degani, Asaf↗
Developing socially and economically generative, resilient PV-energy systems for low- and moderate-income communities: Applications for Puerto Rico (Final Technical Report)
The nexus of energy and poverty is one of the most significant long-term challenges facing the energy and electricity sectors, with negative feedback loops between energy-related costs, risks, and insecurities, on the one hand, and diverse other social and economic insecurities, on the other, helping to undermine health, wellbeing, and resilience in low- and moderate-income communities around the US and the world (Biswas et al. 2022). The transition of energy systems to clean energy alternatives offers a potential opportunity to redesign future energy arrangements in ways that, instead, create beneficial social value for communities, reverses the energy-poverty nexus, and enhances community wellbeing (Biswas et al. 2020; Miller et al. 2018). The promise of clean energy transitions to address the energy-poverty nexus is especially significant when designed using principles of just energy transitions. To leverage just clean energy transitions to accomplish a reversal of the energy-poverty nexus requires improvement in three capabilities: (1) Understanding and mapping the community-scale dynamics and variability of the energy-poverty nexus, which vary considerably from community to community; (2) Evaluating the technical, social, and economic potential for different clean energy system designs to deliver social value to specific communities; and (3) Working collaboratively with communities to imagine and design clean energy systems solutions that meet their needs, fit their capabilities and contexts, and deliver multiple forms of community-desired social value/benefits that help reverse key facets of the energy-poverty nexus. This project developed a portfolio of novel tools, methodological approaches, and collaborative partnerships with four communities in Puerto Rico that further the objective of building these three capabilities.
14 SOLAR ENERGY↗