NOT ALL ERRORS ARE CREATED EQUAL: EXAMINING HUMAN-ALGORITHM SYSTEM PERFORMANCE FOR INTERNATIONAL SAFEGUARDS-INFORMED VISUAL SEARCH TASKS .
Abstract not provided.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Abstract not provided.
Integrated digital instrumentation and control (I&C) systems in new and advanced nuclear power plants (NPPs) will support operators in monitoring and controlling the plants. Even though digital systems typically are expected to be reliable, their potential for degradation or failure significantly could affect the operators performance and, consequently, jeopardize plant safety. This U.S. Nuclear Regulatory Commission (NRC) research investigated the effects of degraded I&C systems on human performance and on plant operations. The objective was to develop technical basis and guidance for human factors engineering (HFE) reviews addressing the operator's ability to detect and manage degraded digital I&C conditions. We reviewed pertinent standards and guidelines, empirical studies, and plant operating experience. In addition, we evaluated the potential effects of selected failure modes of the digital feedwater control system of a currently operating pressurized water reactor (PWR) on human-system interfaces (HSIs) and the operators performance. Our findings indicated that I&C degradations are prevalent in plants employing digital systems, and the overall effects on the plant's behavior can be significant, such as causing a reactor trip or equipment to operate unexpectedly. I&C degradations may affect the HSIs used by operators to monitor and control the plant. For example, deterioration of the sensors can complicate the operators interpretation of displays, and sometimes may mislead them by making it appear that a process disturbance has occurred. We used the findings as the technical basis upon which to develop HFE review guidance.
In providing crew life support for future exploration missions, overall exploration objectives will drive the life support solutions selected. Crew size, mission tasking, and exploration strategy will determine the performance required from life support systems. Human performance requirements, for example, may be offset by the availability of robotic assistance. Once established, exploration requirements for life support will be weighed against the financial and technical risks of developing new technologies and systems. Other considerations will include the demands that a particular life support strategy will make on planetary surface site selection, and the availability of precursor mission data to support EVA and in situ resource recovery planning. As space exploration progresses, the diversity of life support solutions that are implemented is bound to increase.
Within the Human Research Program, one risk delineates the uncertainty surrounding crew working with automation and robotics in spaceflight. The Risk of Inadequate Design of Human and Automation/Robotic Integration (HARI) is concerned with the detrimental effects on crew performance due to ineffective user interfaces, system designs and/or functional task allocation, potentially compromising mission success and safety. Risk arises because we have limited experience with complex automation and robotics. One key gap within HARI, is the gap related to functional allocation. The gap states: We need to evaluate, develop, and validate methods and guidelines for identifying human-automation/robot task information needs, function allocation, and team composition for future long duration, long distance space missions. Allocations determine the human-system performance as it identifies the functions and performance levels required by the automation/robotic system, and in turn, what work the crew is expected to perform and the necessary human performance requirements. Allocations must take into account each of the human, automation, and robotic systems capabilities and limitations. Some functions may be intuitively assigned to the human versus the robot, but to optimize efficiency and effectiveness, purposeful role assignments will be required. The role of automation and robotics will significantly change in future exploration missions, particularly as crew becomes more autonomous from ground controllers. Thus, we must understand the suitability of existing function allocation methods within NASA as well as the existing allocations established by the few robotic systems that are operational in spaceflight. In order to evaluate future methods of robotic allocations, we must first benchmark the allocations and allocation methods that have been used. We will present 1) documentation of human-automation-robotic allocations in existing, operational spaceflight systems; and 2) To gather existing lessons learned and best practices in these role assignments, from spaceflight operational experience of crew and ground teams that may be used to guide development for future systems. NASA and other space agencies have operational spaceflight experience with two key Human-Automation-Robotic (HAR) systems: heavy lift robotic arms and planetary robotic explorers. Additionally, NASA has invested in high-fidelity rover systems that can carry crew, building beyond Apollo's lunar rover. The heavy lift robotic arms reviewed are: Space Station Remote Manipulator System (SSRMS), Japanese Remote Manipulator System (JEMRMS), and the European Robotic Arm (ERA, designed but not deployed in space). The robotic rover systems reviewed are: Mars Exploration Rovers, Mars Science Laboratory rover, and the high-fidelity K10 rovers. Much of the design and operational feedback for these systems have been communicated to flight controllers and robotic design teams. As part of the mitigating the HARI risk for future human spaceflight operations, we must document function allocations between robots and humans that have worked well in practice.
A Laboratory-Directed Research and Development project was initiated in 2005 to investigate Human Performance Modeling in a System of Systems analytic environment. SAND2006-6569 and SAND2006-7911 document interim results from this effort; this report documents the final results. The problem is difficult because of the number of humans involved in a System of Systems environment and the generally poorly defined nature of the tasks that each human must perform. A two-pronged strategy was followed: one prong was to develop human models using a probability-based method similar to that first developed for relatively well-understood probability based performance modeling; another prong was to investigate more state-of-art human cognition models. The probability-based modeling resulted in a comprehensive addition of human-modeling capability to the existing SoSAT computer program. The cognitive modeling resulted in an increased understanding of what is necessary to incorporate cognition-based models to a System of Systems analytic environment.
Theoretical studies on human performance and control monitoring systems
The development and evaluation of computational human performance models is examined. An intention is to develop models which can be used to interact with system prototypes and simulations to perform system assessment. Currently LR is working on a set of models emulating cognitive, psychomotor, auditory, and visual activity for multiple operator positions of a command and control simulation system. These models, developed in conjunction with BBN Systems and Technologies, function within the simulation environment and allow for both unmanned system assessment and manned (human-in-loop) assessment of system interface and team interactions. These are relatively generic models with built-in flexibility which allows modification of some model parameters. These models have great potential for improving the efficiency and effectiveness of system design, test, and evaluation. However, the extent of the practical utility of these models is unclear. Initial verification efforts comparing model performance within the simulation to actual human operators on a similar, independent simulation have been performed and current efforts are directed at comparing human and model performance within the same simulation environment.
In this paper, we present a methodology to assess system performance of human-robotic systems in achievement of collective tasks such as habitat construction, geological sampling, and space exploration.
The purpose of this study was to compare flow constrained area (FCA) capacity setting methods for Collaborative Trajectory Options Program (CTOP) as they pertain to the Integrated Demand Management (IDM) concept. IDM uses flow balancing to manage air traffic across multiple FCAs with a common downstream constraint, as well as constraints at the respective FCA locations. FCA capacity rates can be set manually, but generating capacities for multiple, interdependent FCAs could potentially over-burden a user. A new enhancement to CTOP called the FCA Balance Algorithm (FBA) was developed at NASA Ames Research Center to improve the process of allocating capacity across multiple flow constrained segments in the airspace. The FBA evaluates the predicted demand and capacity across multiple FCAs and dynamically generates capacity settings for the FCAs that best meet capacity limits for all identified constraints. In a human-in-the-loop simulation study, both manual and automated capacity setting methods were evaluated in terms of their overall feasibility using measures of system performance, human performance, and qualitative feedback. Subject matter experts were asked to use three different methods to allocate capacity to three FCAs, either (1) by manually setting capacity for every 60-minute time window, (2) by manually setting capacity for every 15-minute time window, or (3) by using the FBA capability to automatically generate capacity settings. Results showed no significant differences in terms of overall system performance, indicated by similar ground delay and airport throughput numbers between methods. However, differences in individual strategies afforded by the manual methods allowed some participants to achieve system-wide delay that was much lower than the average. The FBA was the fastest method of capacity setting, and it received the lowest subjective rating scores on physical task load, mental task load, task difficulty and task complexity out of the three methods. Finally, participants explained through qualitative feedback that there were many benefits to using the FBA, such as ease of use, accuracy, and low risk of human input error. Participants did not experience the same limitations with the FBA that they did with the manual methods, such as reduced accuracy in the 60-minute manual condition, or high complexity in the 15-minute/manual condition. These results suggest that the FBA automation enhancement to CTOP maintains system performance while improving human performance. Therefore, the FBA could be introduced as a way to mitigate operator workload while planning a CTOP.
The purpose of this study was to compare flow constrained area (FCA) capacity setting methods for Collaborative Trajectory Options Program (CTOP) as they pertain to the Integrated Demand Management (IDM) concept. IDM uses flow balancing to manage air traffic across multiple FCAs with a common downstream constraint, as well as constraints at the respective FCA locations. FCA capacity rates can be set manually, but generating capacities for multiple, interdependent FCAs could potentially over-burden a user. A new enhancement to CTOP called the FCA Balance Algorithm (FBA) was developed at NASA Ames Research Center to improve the process of allocating capacity across multiple flow constrained segments in the airspace. The FBA evaluates the predicted demand and capacity across multiple FCAs and dynamically generates capacity settings for the FCAs that best meet capacity limits for all identified constraints. In a human-in-the-loop simulation study, both manual and automated capacity setting methods were evaluated in terms of their overall feasibility using measures of system performance, human performance, and qualitative feedback. Subject matter experts were asked to use three different methods to allocate capacity to three FCAs, either (1) by manually setting capacity for every 60-minute time window, (2) by manually setting capacity for every 15-minute time window, or (3) by using the FBA capability to automatically generate capacity settings. Results showed no significant differences in terms of overall system performance, indicated by similar ground delay and airport throughput numbers between methods. However, differences in individual strategies afforded by the manual methods allowed some participants to achieve system-wide delay that was much lower than the average. The FBA was the fastest method of capacity setting, and it received the lowest subjective rating scores on physical task load, mental task load, task difficulty and task complexity out of the three methods. Finally, participants explained through qualitative feedback that there were many benefits to using the FBA, such as ease of use, accuracy, and low risk of human input error. Participants did not experience the same limitations with the FBA that they did with the manual methods, such as reduced accuracy in the 60-minute manual condition, or high complexity in the 15-minute/manual condition. These results suggest that the FBA automation enhancement to CTOP maintains system performance while improving human performance. Therefore, the FBA could be introduced as a way to mitigate operator workload while planning a CTOP.
It is anticipated that Advanced Small Modular Reactors (AdvSMRs) will employ high degrees of automation. High levels of automation can enhance system performance, but often at the cost of reduced human performance. Automation can lead to human out-of the loop issues, unbalanced workload, complacency, and other problems if it is not designed properly. Researchers have proposed adaptive automation (defined as dynamic or flexible allocation of functions) as a way to get the benefits of higher levels of automation without the human performance costs. Adaptive automation has the potential to balance operator workload and enhance operator situation awareness by allocating functions to the operators in a way that is sensitive to overall workload and capabilities at the time of operation. However, there still a number of questions regarding how to effectively design adaptive automation to achieve that potential. One of those questions is related to how to initiate (or trigger) a shift in automation in order to provide maximal sensitivity to operator needs without introducing undesirable consequences (such as unpredictable mode changes). Several triggering mechanisms for shifts in adaptive automation have been proposed including: operator initiated, critical events, performance-based, physiological measurement, model-based, and hybrid methods. As part of a larger project to develop design guidance for human-automation collaboration in AdvSMRs, researchers at Idaho National Laboratory have investigated the effectiveness and applicability of each of these triggering mechanisms in the context of AdvSMR. Researchers reviewed the empirical literature on adaptive automation and assessed each triggering mechanism based on the human-system performance consequences of employing that mechanism. Researchers also assessed the practicality and feasibility of using the mechanism in the context of an AdvSMR control room. Results indicate that there are tradeoffs associated with each mechanism, but that some are more applicable to the AdvSMR domain. The two mechanisms that consistently improve performance in laboratory studies are operator initiated adaptive automation based on hierarchical task delegation and the Electroencephalogram(EEG) –based measure of engagement. Current EEG methods are intrusive and require intensive analysis; therefore it is not recommended for an AdvSMR control rooms at this time. Researchers also discuss limitations in the existing empirical literature and make recommendations for further research.
As automation and advanced technologies are introduced into transport systems ranging from the Next Generation Air Transportation System termed NextGen, to the advanced surface transportation systems as exemplified by the Intelligent Transportations Systems, to future systems designed for space exploration, there is an increased need to validly predict how the future systems will be vulnerable to error given the demands imposed by the assistive technologies. One formalized approach to study the impact of assistive technologies on the human operator in a safe and non-obtrusive manner is through the use of human performance models (HPMs). HPMs play an integral role when complex human-system designs are proposed, developed, and tested. One HPM tool termed the Man-machine Integration Design and Analysis System (MIDAS) is a NASA Ames Research Center HPM software tool that has been applied to predict human-system performance in various domains since 1986. MIDAS is a dynamic, integrated HPM and simulation environment that facilitates the design, visualization, and computational evaluation of complex man-machine system concepts in simulated operational environments. The paper will discuss a range of aviation specific applications including an approach used to model human error for NASA s Aviation Safety Program, and what-if analyses to evaluate flight deck technologies for NextGen operations. This chapter will culminate by raising two challenges for the field of predictive HPMs for complex human-system designs that evaluate assistive technologies: that of (1) model transparency and (2) model validation.
Designing space suits and vehicles for the diverse human population present unique challenges for the methods of traditional anthropometry. Space suits are bulky and allow the operator to shift position within the suit and inhibit the ability to identify body landmarks. Limited suit sizing options also cause variability in fit and performance between similarly sized individuals. Space vehicles are restrictive in volume in both the fit and the ability to collect data. NASA's Anthropometric and Biomechanics Facility (ABF) has utilized 3D scanning to shift from traditional linear anthropometry to explore and examine volumetric capabilities to provide anthropometric solutions for design. Overall, the key goals are to improve the human-system performance and develop new processes to aid in the design and evaluation of space systems. Four case studies are presented that illustrate the shift from purely linear analyses to an augmented volumetric toolset to predict and analyze the human within the space suit and vehicle. The first case study involves the calculation of maximal head volume to estimate total free volume in the helmet for proper air exchange. Traditional linear measurements resulted in an inaccurate representation of the head shape, yet limited data exists for the determination of a large head volume. Steps were first taken to identify and classify a maximum head volume and the resulting comparisons to the estimate are presented in this paper. This study illustrates the gap between linear components of anthropometry and the need for overall volume metrics in order to provide solutions. A second case study examines the overlay of the space suit scans and components onto scanned individuals to quantify fit and clearance to aid in sizing the suit to the individual. Restrictions in space suit size availability present unique challenges to optimally fit the individual within a limited sizing range while maintaining performance. Quantification of the clearance and fit between similarly sized individuals is critical in providing a greater understanding of the human body's function within the suit. The third case study presented in this paper explores the development of a conformal seat pan using scanning techniques, and details the challenges of volumetric analyses that were overcome in order to develop a universal seat pan that can be utilized across the entire user population. The final case study explores expanding volumetric capabilities through generation of boundary manikins. Boundary manikins are representative individuals from the population of interest that represent the extremes of the population spectrum. The ABF developed a technique to take three-dimensional scans of individuals and manipulate the scans to reflect the boundary manikins' anthropometry. In essence, this process generates a representative three-dimensional scan of an individual from anthropometry, using another individual's scanned image. The results from this process can be used in design process modeling and initial suit sizing work as a three dimensional, realistic example of individuals from the population, maintaining the variability between and correlation to the relevant dimensions of interest.
This document contains papers presented at the Dual-Use Space Technology Transfer Conference and Exhibition held at the Johnson Space Center February 1-3, 1994. Possible technology transfers covered during the conference were in the areas of information access; innovative microwave and optical applications; materials and structures; marketing and barriers; intelligent systems; human factors and habitation; communications and data systems; business process and technology transfer; software engineering; biotechnology and advanced bioinstrumentation; communications signal processing and analysis; new ways of doing business; medical care; applications derived from control center data systems; human performance evaluation; technology transfer methods; mathematics, modeling, and simulation; propulsion; software analysis and decision tools systems/processes in human support technology; networks, control centers, and distributed systems; power; rapid development perception and vision technologies; integrated vehicle health management; automation technologies; advanced avionics; ans robotics technologies. More than 77 papers, 20 presentations, and 20 exhibits covering various disciplines were presented b experts from NASA, universities, and industry.
This is the second volume of papers presented at the Dual-Use Space Technology Transfer Conference and Exhibition held at the Johnson Space Center February 1-3, 1994. Possible technology transfers covered during the conference were in the areas of information access; innovative microwave and optical applications; materials and structures; marketing and barriers; intelligent systems; human factors and habitation; communications and data systems; business process and technology transfer; software engineering; biotechnology and advanced bioinstrumentation; communications signal processing and analysis; medical care; applications derived from control center data systems; human performance evaluation; technology transfer methods; mathematics, modeling, and simulation; propulsion; software analysis and decision tools; systems/processes in human support technology; networks, control centers, and distributed systems; power; rapid development; perception and vision technologies; integrated vehicle health management; automation technologies; advanced avionics; and robotics technologies.
NASA and the FAA have initiated programs of research and development to provide flight crew, airline operations and air traffic managers with automation aids to increase capacity in en route and terminal area to support the goals of free flight for safe, flexible, predictable, and efficient operations. To support the development of those aiding systems human performance in automated aiding has been examined in empirical and computationally analytic studies. This paper presents a set of those studies in full mission simulation and the development of a predictive computational model of human performance. We have found that the combination of methodologies provide a powerful design-aiding process. We will describe three research programs in support of Free Flight Operations from the perspective of human performance requirements. We have examined procedures and communications in the use of voice and data-link operation at the transition between unconstrained (enroute) and constrained (terminal) airspace operations. We have examined the timing and form of Center TRACON Automation System (CTAS) advisories in descent. We have investigated the shape and dimensions of an 'alert zone' for air-based separation in unconstrained operations. Finally, we have examined the interaction of a ground-based conflict detection/resolution aiding system, as it interacts with a cockpit-based conflict alerting mechanism. Additional information is contained in the original extended abstract.
Much of the United States commercial nuclear power plant fleet is approaching the end of their operating licenses. To extend the life of these plants, advanced human-system interface (HSI) technologies are being researched to address aging and reliability concerns with the existing legacy instrumentation and controls. Human factors engineering (HFE) has a crucial role in ensuring these technologies are designed in a way that does not introduce new failure modes and promote optimal human-system performance. An important focus of HFE in NPP modernization is the early involvement to inform the design of the HSI. Here, traditional formative evaluations are done to collect design feedback. While these evaluations are absolutely useful, they are limited in providing convincing quantitative data for efficiency of use. This paper discusses the use of cognitive models to provide such quantitative data early in the HSI design process. A comparison of three open-source and readily accessible cognitive modeling tools are discussed.
Human-centered design (HCD) is the practice of designing systems for human use. This paper describes the discipline of human factors engineering, which has traditionally been applied to energy intensive complex systems such as automobiles and aircraft, and will look ahead to applications in sustainable systems design. The focus of HCD has been ensuring efficient and effective human-system performance, through application of design principles such as understanding the user, providing feedback, and minimizing cognitive load. These principles also apply to the design of sustainable socio-technical systems, both through specific device user interfaces and broader developments such as transportation systems and urban design. HCD design principles need to be embedded within a social-systems engineering model that considers the overall constraints of physical and institutional processes, the cultural aspects of designs and interventions, various comfort and convenience factors at the level of individual users, and the cognitive impacts of designs – the 5C model. Application of HCD will involve a more holistic view of human-systems than is traditionally adopted by government and industry, to encompass the embedded nature of energy services in lifestyle and work. HCD methods can be useful in disaggregating the energy intensive aspects of lifestyle and work, as well as applying design principles to address the 5 C’s.