NOT ALL ERRORS ARE CREATED EQUAL: EXAMINING HUMAN-ALGORITHM SYSTEM PERFORMANCE FOR INTERNATIONAL SAFEGUARDS-INFORMED VISUAL SEARCH TASKS .
Abstract not provided.
Gastelum, Zoe↗
SEARCH · Search NASA
Results for “human systems performance”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Physiological Characterization of Language Comprehension
In this project, our goal was to develop methods that would allow us to make accurate predictions about individual differences in human cognition. Understanding such differences is important for maximizing human and human-system performance. There is a large body of research on individual differences in the academic literature. Unfortunately, it is often difficult to connect this literature to applied problems, where we must predict how specific people will perform or process information. In an effort to bridge this gap, we set out to answer the question: can we train a model to make predictions about which people understand which languages? We chose language processing as our domain of interest because of the well- characterized differences in neural processing that occur when people are presented with linguistic stimuli that they do or do not understand. Although our original plan to conduct several electroencephalography (EEG) studies was disrupted by the COVID-19 pandemic, we were able to collect data from one EEG study and a series of behavioral experiments in which data were collected online. The results of this project indicate that machine learning tools can make reasonably accurate predictions about an individual?s proficiency in different languages, using EEG data or behavioral data alone.
42 ENGINEERING↗
Human Readiness Levels Explained
The Human Readiness Level scale complements and supplements the existing technology readiness level scale to support comprehensive and systematic evaluation of human system aspects throughout a system’s life cycle. The objective is to ensure humans can use a fielded technology or system as intended to support mission operations safely and effectively. This article defines the nine human readiness levels in the scale, explains their meaning, and illustrates their application using a helmet-mounted display example.
60 APPLIED LIFE SCIENCES↗
Characterization of Peripheral Neurophotonic Systems for High-Performance Human-Computer Interfaces (CRADA Final Report)
As part of the Cyclotron Road program, Morphosis Inc. sought to investigate a non-invasive neuromuscular sensing approach for use as an intuitive and secure human–computer interface. These highly miniaturized, wearable neural interfaces were completely non-invasive and maintained stable, high-bandwidth, long-term access to a user’s actions, intent, and identity, while offering an exceptionally high signal-to-noise ratio compared to contemporary neural recording technologies. The widespread adoption of neural interfaces had the potential to reshape how people interact with technology, with profound societal impacts. Millions worldwide suffered from movement and/or speech disabilities, and these tools had the potential to democratize access to technology to enhance autonomy and quality of life. More broadly, interfaces capable of accurately conveying intentions and safeguarding identities could serve as a cornerstone for privacy, trust, and personal authenticity in digital environments. The use of thought-driven control of digitally enabled devices and governance of digital identities had the potential to revolutionize relationships with technology, transforming how people learn, communicate, and interact with the world.
42 ENGINEERING↗
Rancor-HUNTER: A Virtual Plant and Operator Environment for Predicting Human Performance
Advances in simulation capabilities to model physical systems have outpaced the development of simulations for humans using those physical systems. There is an argument that the infinite span of potential human behaviors inherently render human modeling more challenging than physical systems. Despite this challenge, the need for modeling humans interacting with these complex systems is paramount. As technologies have improved, many of the failure modes originating from the physical systems have been solved. This means the overall proportion of human errors has increased, such that it is not uncommon to be the primary driver of system failure in modern complex systems. Moreover, technologies such as automated systems may introduce emerging contexts that can cause new, unanticipated modes of human error. Therefore, it is now more important than ever to develop models of human behavior to realize overall system error reductions and achieve established safety margins. To support new and novel concepts of operations for the anticipated wave of advanced nuclear reactor deployments, human factors and human reliability analysis researchers need to develop advanced simulation-based approaches. This talk presents a simulation environment suitable to both collect data and then perform Monte Carlo simulations to evaluate human performance and develop better models of human behavior. Specifically, the Rancor Microworld Simulator models a complex energy production system in a simplified manner. Rancor includes computer-based procedures, which serve as a framework to automatically classify human behaviors without manual, subjective experimenter coding during scenarios. This method supports a detailed level of analysis at the task level. It is feasible for collecting large sample sizes required to develop quantitative modelling elements that have historically challenged traditional full-scope simulator study approaches. Additionally, the other portion of this experimental platform, the Human Unimodel for Nuclear Technology to Enhance Reliability (HUNTER), is presented to show how the collected data can be used to evaluate novel scenarios based on the contextual factors, or performance shaping factors, derived from Rancor simulations. Rancor-HUNTER is being used to predict operator performance with new procedures, such as results from control room modernization or new-build situations. Rancor-HUNTER is also proving a useful surrogate platform to model human performance for other complex systems.
99 GENERAL AND MISCELLANEOUS↗
Rancor-HUNTER: Using a Simulator Engine for Realistic Human Performance Modeling of Nuclear Power Operations
The Human Unimodel for Nuclear Technology to Enhance Reliability (HUNTER) is a software system to simulate human performance in support of human reliability analysis (HRA) in nuclear power plants. This paper summarizes recent work to integrate HUNTER with a plant simulator, namely the Rancor Microworld Simulator. Rancor is an offshoot of earlier work at Idaho National Laboratory (INL) to support plant modernization. The graphical software tools used to mimic digital human-system interface upgrades at INL’s Human Systems Simulation Laboratory were linked to the Rancor Microworld Simulator, an INL-developed simplified plant model. HUNTER becomes a “virtual operator” coupled to the Rancor simulator, thereby allowing a tight coupling between a digital human twin and a digital twin of the plant. Rancor-HUNTER may be run through Monte Carlo iterations across a dynamic range of performance shaping factors, thereby producing distributions of human performance in terms of procedure paths, errors instantiations, and task durations. This paper overviews the various unique features of Rancor-HUNTER and presents an example run of Rancor-HUNTER for a startup scenario.
99 - GENERAL AND MISCELLANEOUS↗
Light Water Sustainability Program: Optimizing Information Automation Using a New Method Based on System-Theoretic Process Analysis
This report describes the interim progress for research supporting the design and optimization of information automation systems for nuclear power plants. Much of the domestic nuclear fleet is currently focused on modernizing technologies and processes, including transitioning toward digitalization in the control room and elsewhere throughout the plant, along with a greater use of automation, artificial intelligence, robotics, and other emerging technologies. While there are significant opportunities to apply these technologies toward greater plant safety, efficiency, and overall cost-effectiveness, optimizing their design and avoiding potential safety and performance risks depends on ensuring that human-performance-related organizational and technical design issues are identified and addressed. This report describes modeling tools and techniques, based on sociotechnical system theory, to support these design goals and their application in the current research effort. The report is intended for senior nuclear energy stakeholders, including regulators, corporate management, and senior plant management. We have developed and employed a method to design an optimized information automation ecosystem (IAE) based on the systems-theoretic constructs underlying sociotechnical systems theory in general and the Systems-Theoretic Accident Modeling and Processes (STAMP) approach in particular. We argue that an IAE can be modeled as an interactive information control system whose behavior can be understood in terms of dynamic control and feedback relationships amongst the system’s technical and organizational components. Up to this point, we have employed a Causal Analysis based on STAMP (CAST) technique to examine a performance- and safety-related incident at an industry partner’s plant that involved the unintentional activation of an emergency diesel generator. This analysis provided insight into the behavior of the plant’s current information control structure within the context of a specific, significant event. Our ongoing analysis is focused on identifying near-term process improvements and longer-term design requirements for an optimized IAE system. The latter analyses will employ a second STAMP-derived technique, System-Theoretic Process Analysis (STPA). STPA is a useful modeling tool for generating and analyzing actual or potential information control structures. Finally, we have begun modeling plantwide organizational relationships and processes. Organizational system modeling will supplement our CAST and STPA findings and provide a basis for mapping out a plantwide information control architecture. CAST analysis findings indicate an important underlying contributor to the incident under investigation, and a significant risk to information automation system performance, was perceived schedule pressure, which exposed weaknesses in interdepartmental coordination between and within responsible plant organizations and challenged the resilience of established plant processes, until a human caused the initiating event. These findings are discussed in terms of their risk to overall system performance and their implications for information automation system resilience and brittleness. We present two preliminary information automation models. The proactive issue resolution model is a test case of an information automation concept with significant near-term potential for application and subsequent reduction in significant plant events. The IAE model is a more general representation of a broader, plantwide information automation system. From our results, we have generated a set of preliminary system-level requirements and safety constraints. These requirements will be further developed over the remainder of our project in collaboration with nuclear industry subject matter experts and specialists in the technical systems under consideration. Additionally, we will continue to pursue the system analyses initiated in the first part of our effort, with a particular emphasis on STPA as the main tool to identify weak or weakening control structures that affect the resilience of organizations and programs. Our intent is to broaden the scope of the analysis from an individual use case to a related set of use cases (e.g., maintenance tasks, compliance tasks) with similar human-system performance challenges. This will enable more generalized findings to refine the Proactive Issue Resolution and IAE models, as well as their system-level requirements and safety constraints. We will use organizational system modeling analyses to supplement STPA findings and model development. We conclude the report with a set of summary recommendations and an initial draft list of system-level requirements and safety constraints for optimized information automation systems.
99 GENERAL AND MISCELLANEOUS↗
Incorporating the Role(s) of Human Actors in Complex System Design for Safety and Security
Traditional systems engineering demonstrates the importance of customer needs in scoping and defining design requirements; yet, in practice, other human stakeholders are often absent from early lifecycle phases. Human factors are often omitted in practice when evaluating and down-selecting design options due to constraints such as time, money, access to user populations, or difficulty in proving system robustness through the inclusion of human behaviors. Advances in systems engineering increasingly include non-technical influences into the design, deployment, operations, and maintenance of interacting components to achieve common performance objectives. Furthermore, such advances highlight the need to better account for the various roles of human actors to achieve desired performance outcomes in complex systems. Many of these efforts seek to infuse lessons and concepts from human factors (enhanced decision-making through Crew Resource Management), systems safety (Rasmussen's “drift toward danger”) and organization science (Giddens' recurrent human acts leading to emergent behaviors) into systems engineering to better understand how socio-technical interactions impact emergent system performance. Safety and security are examples of complex system performance outcomes that are directly impacted by varying roles of human actors. Using security performance of high consequence facilities as a representative use case, this article will outline the System Context Lenses to understand how to include various roles of human actors into systems engineering design. Several exemplar applications of this organizing lenses will be summarized and used to highlight more generalized insights for the broader systems engineering community.
42 ENGINEERING↗
Human Factors Design for Particle Accelerator Control Room Interfaces
Fermilab, the birthplace of many scientific discoveries in physics and particle accelerator sciences, is in the midst of a widescale modernization effort. The Accelerator Control Operations Research Network (ACORN project’s goal is to modernize the accelerator control system by replacing end-of-life power supplies and enhance future operations of the Fermilab accelerator complex with megawatt particle beams. Within ACORN, opportunities for process improvement concerning software development, human-system interface design, and task performance are also being considered. Human factors researchers from Idaho National Laboratory in collaboration with usability experts from Fermilab, are currently investigating human-centered design improvements for the accelerator control system. For example, substantial tribal knowledge and memory recall are required to effectively operate the accelerator system. This contributes to high cognitive workload and potential burnout of accelerator operators. Developing guidance for consistent visual and functional design enables a more intuitive interaction and relieves operators of cognitive burden. Additionally, developing more intuitive and integrated interfaces can also lead to improved accelerator efficacy by empowering operators with greater understanding and control of the systems. The challenge in developing such interfaces is in designing for a wide variety of user goals, system specifications, and level of experience in users. The challenges need to be met while e also considering the maintainability of the control system. The purpose of this paper is to detail the human factors process and design within the ACORN project, describe results gathered thus far, and discuss the larger implications for this work.
43 PARTICLE ACCELERATORS↗
Load Shedding for Voltage Regulation With Probabilistic Agent Compliance
With the increased observability and controllability of distribution systems, the share of behind-the-meter systems is trending upwards rapidly. As a consequence, the impact of human behaviors on system performance can no longer be ignored and should be reflected in the energy management system models. In this paper, we discuss the problem of distribution system voltage control by active power curtailment where the agent compliance of the load curtailment signal is probabilistic. We discuss the modeling of the optimal voltage control problem with probabilistic agent compliance as a chance-constrained optimization problem, its tractable safe approximation using convex restriction, and a scenario-based mixed-integer reformulation as well as the associated solution method based on augmented Lagrangian method. The numerical simulation on IEEE test system validates the effectiveness of the proposed approach in obtaining high-quality feasible load curtailment signal with low computational cost, which makes it a viable tool for real time decision making.
augmented Lagrangian method↗
Dynamic Human-in-the-Loop Simulated Nuclear Power Plant Thermal Dispatch System Demonstration and Evaluation Study
An Idaho National Laboratory research team performed a human-in-the-loop study with two formerly licensed retired operators to evaluate thermal power dispatch operations supported by the modified GSE Systems GPWR plant simulator. Virtual representations of the analog control panels were presented on touch-screen bays configured to mimic the control room layout in the newly renovated Human Systems Simulation Laboratory. The operators performed 15 scenarios covering normal evolutions to transition the plant from full turbine operation to joint turbine and thermal power dispatch operations in addition to transient response scenarios induced with simulated faults to evaluate the impact of thermal power dispatch system on operator and plant responses. A prototype human-system interface (HSI) was developed and displayed in tandem with the virtual analog panels to support the operators executing the procedurally drive evolutions and transient responses. An interdisciplinary team of operations experts, nuclear engineers, and human factors experts observed the operators performing the scenarios to evaluate the operations. Only a preliminary analysis of the results has been performed. The full analysis will be shared in a milestone report scheduled for release at the end of September; however, some high-level conclusions were readily available. Two high level findings for the system design were captured in the study. The manual control supported by the HSI to transition from standard operations to thermal power dispatch operation imposed a considerable amount of workload on the operators due to tedious manual valve manipulations and system monitoring required to verify their intended effect. An additional operator would be required in the control room to support the daily evolution. Automatic control for the transition was deemed a requirement for plant adoption without imposing additional staffing costs. The second finding was the necessity for an automatic thermal power dispatch system trip isolation function linked to a turbine and reactor trip signal. The operators completed scenarios with automatic isolation functionality and manually required actuation of the thermal power dispatch system. The operator response was sufficiently slower in the manual trip condition, such that operators were unable to manually actuate key post trip safety functions an indicate a degraded control capability that should be avoided. With an automatic trip signal little to no impact of the thermal power dispatch system was identified on the primary plant response and therefore the system could be readily and safely adopted. Together these two findings represent the need to support the adoption of thermal power dispatch capability into existing operations by leveraging automation to augment any additional operator tasking required to control and monitor an additional system beyond existing operations.
99 GENERAL AND MISCELLANEOUS↗
The Value of Forecasters‐in‐the‐Loop in Real‐Time Flood Forecasting in the Age of Machine Learning
Machine learning (ML) applications in hydrological forecasting are increasingly prevalent and show great potential. However, many previous studies have only evaluated performance through reanalysis or retrospective simulations compared to simplified baselines. This study provides the first assessment of ML performance against actual operational forecasting systems operated by the California Nevada River Forecast Center (CNRFC), which combines the Community Hydrologic Prediction System (CHPS) with forecasters-in-the-loop. Results demonstrate that forecasters-in-the-loop systems consistently outperform ML models in both general forecasts and flood alerting across lead times up to 96 hr, even when ML models use observed forcings, while CNRFC operational process relies on biased weather forecasts. Our analysis reveals that forecaster expertise maintains forecast reliability despite inaccurate precipitation inputs, with human-guided systems showing superior performance degradation characteristics at extended lead times. These findings highlight the irreplaceable value of human expertise in operational forecasting and caution against overstating current ML capabilities in real-world applications.
Tran, Vinh Ngoc [Univ. of Michigan, Ann Arbor, MI ↗
Human Dimensions of Energy Systems Workshop - September 6-7, 2022: Findings and Next Steps
Advanced energy technologies and informed policies are necessary but not sufficient to accelerate the energy transition at the speed required to meet our shared climate and energy resilience goals. To fully understand and implement integrated energy systems, we need to be able to model and analyze the complete system-of-systems, including the behavior of people interacting with and being impacted by energy systems. The purpose of this workshop is: Understand how human behavior and decisions affect the performance of energy systems with a focus on resilience and human well-being; Identify opportunities to improve energy system design to explicitly consider human behavior and well-being; Enhance our capability to model and predict human actions. Develop the ability to stress test these integrated systems; Identify or develop requirements and tools for more robust system designs that can be used for human-in-the-loop exercises and training; Identify opportunities for joint research, joint appointments, and collaboration; and Guide internal investments and strategic hires.
analyze↗
Visual Style Elements in Human–System Interface Design for Nuclear Power Operations: Does Style Affect Performance and Preference?
As the world increasingly adopts renewable and sustainable energy systems, transitionary solutions include nuclear power, which currently provides 20% of the United States’ electricity and is the largest single source of carbon-free electricity generation. Advanced reactors are a critical component of a carbon-free mixed energy portfolio that require careful design of first-of-a-kind control rooms. The application of Human Factors Engineering (HFE) is essential for scientific and iterative testing of novel human–system interface (HSI) concepts to ensure effective, efficient, and safe plant operations. Microworlds are simulators that use simplified physics models and control systems to distill nuclear power operations into essential functions. HFE scientists used the Rancor Microworld Simulator to obtain preference and performance metrics for novel and traditional static HSI design styles. Participants comprised advanced reactor company employees and nuclear industry consultants. A mixture of quantitative and qualitative data was captured. There was a preference for the basic graphical style that included high contrast and traditional color scheme elements. No single HSI design outperformed the others, and the participants did not perform better using their preferred HSI style. We report this experiment is the first in a series of HFE testing for HSIs in advanced reactor control room development. Clear user preferences emerged for elements within static displays. The cutting-edge neumorphic style was the least preferred. Future directions include tests of dynamic displays. HFE is used in evaluating and designing HSI devices that will improve the efficiency and safety of advanced nuclear power operations.
99 GENERAL AND MISCELLANEOUS↗
Fleet Algorithm Design for Pooled Rideshare: Integrating Human Factors, Simulation, and Optimization
This dissertation explores the study the integration of human factors modeling and rideshare fleet control algorithms. Pooled rideshare is a unique transportation mode offering that allows riders increased flexibility and accessibility over public transportation, and decreased cost relative to personal vehicles or traditional rideshare. Additionally, relative to personal vehicles, pooled rideshare offers reduced costs and options for those with difficulty obtaining transportation. Prior research in the space typically focused on modeling human behavior, or optimizing system performance, but a lack of integration of the concepts leads to unrealistic or underutilized outcomes. To tackle this problem, novel rideshare assignment, and repositioning strategies were designed and implemented in a simulation environment. Through a series of successive studies, improvements to current rideshare processes were identified, and beneficial outcomes for profitability, accessibility, and traffic were explored. Further, improved metrics to assess rideshare performance were designed and analyzed in the context of improved rideshare offerings. This research contributes to the field of transportation by tackling novel but pragmatic approaches to challenges facing the rideshare industry. Through the course of this dissertation, rideshares impacts on users, operators, and even regulators will be explored in detail. The justification behind the use of a simulation environment, a set of simulated regional models for testing, and the focus on realism and deployability is illustrated. The research identifies holes in potential markets for the use of both private, and public rideshare systems.
Paul, Joseph↗
Multi defect detection and analysis of electron microscopy images with deep learning
Electron microscopy is widely used to explore defects in crystal structures, but human detecting of defects is often time-consuming, error-prone, and unreliable, and is not scalable to large numbers of images or real-time analysis. In this work, we discuss the application of machine learning approaches to find the location and geometry of different defect clusters in irradiated steels. We show that a deep learning based Faster R-CNN analysis system has a performance comparable to human analysis with relatively small training data sets. Furthermore, this study proves the promising ability to apply deep learning to assist the development of automated microscopy data analysis even when multiple features are present and paves the way for fast, scalable, and reliable analysis systems for massive amounts of modern electron microscopy data.
36 MATERIALS SCIENCE↗
Ambient energy for buildings: Beyond energy efficiency
The following Key Messages comprise the salient findings of this study: 1. Ambient energy (from sun, air, ground, and sky) can heat and cool buildings; provide hot water, ventilation and daylighting; dry clothes; and cook food. These services account for about three-quarters of building energy consumption and a third of total US demand. Biophilic design (direct and indirect connections with nature) is an intrinsic adjunct to ambient energy systems, and improves wellness and human performance. 2. The current strategy of electrification and energy efficiency for buildings will not meet our climate goals, because the transition to an all-renewable electric grid is too slow. Widespread adoption of ambient energy is needed. Solar-heated buildings also flatten the seasonal demand for electricity compared to all-electric buildings, reducing required production capacity and long-term energy storage. In addition, ambient-conditioned buildings improve resilience by remaining livable during power outages. 3. National policies, incentives, and marketing should be enacted to promote ambient energy use. Federal administrative priorities should reflect the importance of ambient energy for buildings. Use of ambient energy should be encouraged through existing and new building codes and standards. 4. Ambient energy system design tools are needed for architects, engineers, builders, building scientists, realtors, appraisers, and consumers. PVWatts is used over 100 million times per year for photovoltaic system design. A similar, simple, and accessible tool for ambient design is crucial. 5. Training on ambient energy is needed throughout secondary, post-secondary, and continuing education for workforce development. Currently, only about 10% of colleges teach courses on passive heating and cooling systems. 6. Ambient-conditioned buildings should be demonstrated in all US climate zones. Performance should be monitored and reported, with quantitative case studies made widely available. 7. While current technology is sufficient to build high-performance ambient buildings now, research is needed to develop new technologies to harness ambient energy more effectively and more economically. Such advancements will facilitate adoption of ambient energy technologies in a wider range of buildings, including retrofits. Examples include windows with much lower thermal losses, use of the building shell as thermal storage, alternative light-weight thermal storage systems, sky radiation cooling systems, automated controls for solar gains and passive cooling, and ground coupling.
14 SOLAR ENERGY↗
Digital Twin Technology (“Morpheus”) for Optimized Building Operations [SWR-22-74]
The electrification of buildings is an important step to reducing greenhouse gas emissions across all industries. The management of increasingly electrified buildings is a complex pursuit, and there remains a need for cost-effective software capable of handling the computational burden required of such complexity. Through a partnership with Dallas Fort Worth (DFW) Airport, researchers at NREL have developed a digital twin modeling framework to optimize building operations, called Morpheus. Pairing predictive control with automatic fault detection and diagnostics, Morpheus decreases energy expenditures, costs, and faults for large facilities. Additionally, Morpheus employs artificial intelligence to continuously improve its performance using information provided by sensor systems, human experts with deep industry domain knowledge, and even from other similar machines or fleets of machines. Coupling this novel energy-management software with other digital twins, such as NREL’s Athena software for mobility operations, enables robust decision-making for asset and space management. The implementation of Morpheus at DFW has resulted in significantly improved HVAC system operations and reduced both peak power and overall energy consumption. This enhanced functionality comes at a more affordable price than previously developed digital twins and can be customized for other facilities’ geometries to provide optimal, individualized control of a facility’s energy consumption.
Chinde, Venkatesh↗