Search NASA⌕ Search

SEARCH · Search NASA

Results for “Human Factors Evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Human Factors Evaluation of Energy Visualization Dashboards

Energy visualization systems provide information about use in real-time to assist users with energy efficiency. In this study, three energy visualization dashboards for small businesses were developed and tested. Performance measurement, NASA Task Load Index (TLX) workload assessment, and posttest survey were used to conduct the usability testing. Compared with the dashboards that were designed using line charts and tables, a dashboard designed using visuals (e.g., gauges, pie charts, and flashing lights) produced quicker response time, lower mental and temporal demand and effort ratings, and higher ratings of engagement, interest, and trustworthiness.

Nimbarte, Ashish D. (ORCID:0000000210599112)↗
DOE OSTI2022

Demonstrating the Value of 3D Models to Support Large-Scale Digital Modifications at Nuclear Power Plants

Many Nuclear Power Plants are currently in the process of extending their operating licenses for continued generation. The use of three-dimensional (3D) modeling in the early stages of large scale NPP modernization efforts is one lower cost method that can verify proposed design changes against established guidelines and allows for visual presentation of the 3D model to various stakeholders in the project. Guidance from Nuclear Regulatory Commission NUREG 0711 and 0700, and other sources on performing HF/E for control rooms and design modifications can be visually represented in 3D models. Distance and measurements, workstation design, anthropometric considerations, and early feedback of modifications are used in 3D models to identify potential human issues early in the design process. 3D modeling is a useful tool for early design and help to reduce costs and present visuals to stakeholders early in the design.

3D Models↗
DOE OSTI2021

EXTRACTING HUMAN RELIABILITY FINDINGS FROM HUMAN FACTORS STUDIES IN THE HUMAN SYSTEMS SIMULATION LABORATORY

Modernization of U.S. nuclear power plants (NPPs) is widespread, with most plants currently replacing and transitioning equipment, control systems, and human system interfaces (HSI)s from analog to digital displays. This conversion remedies the obsolescence of analog parts along with needs for increased intuitiveness of design, safety, and capabilities. The Human Factors and Reliability team at Idaho National Laboratory (INL) carried out twelve control room modernization studies in the newly designed Human Systems Simulation Laboratory (HSSL) over nine years. The HSSL was constructed as a testbed for evaluating human factors techniques and performance measures, HSI frameworks, and cutting-edge operational concepts in NPPs. Installing a full-scope training simulator enabled direct design and evaluation work on the same instrumentation and control (I&C) and HSIs located at U.S. plants. The subsequent addition of glass top bays afforded crews opportunities to implement operations via the simulator using full-scale representations of their home NPP. Additionally, functional HSI prototypes were created, providing an environment for operator-in-the-loop benchmark studies. The HSSL has assisted in upgrades of six commercial NPP control rooms and served as an invaluable proving ground for new NPP operations technology. Human reliability analysis (HRA) was not originally the focus of the studies; however, data relating to HRA such as type and frequency of human errors can be extracted from the studies. INL is currently extracting data from the HSSL study reports to apprise how information gathered from simulation, HSI, and other related studies can create a broad look across different data sources to help inform HRA methods.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

A mathematical approach to using the forgetting curve to evaluate experience and training factors in human reliability analysis

Traditional human reliability analysis (HRA) methods have difficulty dealing with the dynamic nature of factors such as time and rely on static and expert-judgment-based assessments of performance-shaping factors (PSFs) across limited levels. In this study, we introduce a mathematical approach for dynamically evaluating the experience and training PSF. Our proposed method integrates the psychological concept of the “forgetting curve” to evaluate how PSFs are impacted by the number of trainings and the time elapsed since training. To confirm the validity of the model, we provide experimental data fitted by identifying the quantitative relationship between training and human performance. This research enables dynamic and objective assessments, thus reducing reliance on subjective expert judgment and improving the accuracy of HRA.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

Recommended methods for conducting human factors experiments on the subjective evaluation of colour rendition

This article explores the best practices for conducting psychophysical experiments that investigate how colour rendition influences the perception of architectural environments. Here, we offer guidance that covers all stages of research from preliminary development to publication, focusing especially on experiments that investigate qualities such as perceived naturalness, vividness, preference or acceptability in response to changes in the spectral power distribution of light sources. This article is intended to be a consolidated guide for researchers and reviewers of this type of research. Key recommendations include: (1) New work should be motivated by clearly expressed research questions and, when possible, explicit hypotheses that build on the existing body of knowledge, (2) visual stimuli comprising spectral power distributions and visual targets should be deliberately engineered to probe the research questions, (3) experiments should be designed to lessen potential biases, (4) reporting of experimental conditions and statistical analyses should be thorough and (5) Results should be contextual, resisting overgeneralization that cannot be supported by the data. Our motivation is to encourage high-quality research that is credible and discourage poor quality research that slows scientific progress and misuses resources.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗
DOE OSTI2025

Heuristic Evaluation Methods Applied to a Predictive Maintenance Chatbot

The need for an accessible iterative approach for evaluating prospective artificial intelligence (AI)/ML based technologies in the nuclear industry is needed, given the nature of algorithms and rapid advancements. This paper explores existing heuristic design principles for user-centered design and evaluates them based on their relevancy and usefulness for evaluating AI/ ML based technologies. Researchers at the Idaho National Laboratory (INL) have developed a machine learning software application called VIsualization for PrEdictive maintenance Recommendation (VIPER), which is used to help users understand and engage with the tool to learn more about work orders, data used, predictive maintenance, and machine learning (ML) algorithms. Early user research studies used to access VIPER’s technology readiness level have occurred; however, there is room for further improvement of the software through heuristic evaluations along with other methods and user testing. This work describes the applicability of heuristic evaluation methods and cognitive walkthroughs to help ensure human readiness for prospective AI/ ML based applications, using VIPER as a candidate use case. This work supports industry in ensuring that prospective AI/ML based technologies are usable and useful for plant personnel at nuclear power plants, ultimately leading to their safe, reliable, and efficient use.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

Heuristic Evaluation Methods Applied to a Predictive Maintenance Chatbot

The need for an accessible iterative approach for evaluating prospective artificial intelligence (AI)/ML based technologies in the nuclear industry is needed, given the nature of algorithms and rapid advancements. This paper explores existing heuristic design principles for user-centered design and evaluates them based on their relevancy and usefulness for evaluating AI/ ML based technologies. Researchers at the Idaho National Laboratory (INL) have developed a machine learning software application called VIsualization for PrEdictive maintenance Recommendation (VIPER), which is used to help users understand and engage with the tool to learn more about work orders, data used, predictive maintenance, and machine learning (ML) algorithms. Early user research studies used to access VIPER?s technology readiness level have occurred; however, there is room for further improvement of the software through heuristic evaluations along with other methods and user testing. This work describes the applicability of heuristic evaluation methods and cognitive walkthroughs to help ensure human readiness for prospective AI/ ML based applications, using VIPER as a candidate use case. This work supports industry in ensuring that prospective AI/ML based technologies are usable and useful for plant personnel at nuclear power plants, ultimately leading to their safe, reliable, and efficient use. PowerPoint for conference that was reviewed in PRS and LRS PRS/CON-25-05379 and INL/CON-25-82946

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Assessment of Electric Grid Transmission System Simulator for Human Factors Research

Over the past decades, various technologies have been developed for electric grid operations to support clean energy, meet rising electricity demands, and address infrastructure concerns. However, the human factors aspect is often overlooked during rapid integration. Questions persist about how these technologies impact human performance. Simulators play a critical role in supporting investigation of human factors design concepts and conducting comprehensive usability testing to evaluate human performance and assess human reliability. This paper aims to address human factors research simulator requirements and conduct a comparative study of six different simulators. A detailed evaluation reveals that the evaluated simulators lack the ability to customize user interfaces. Additionally, their user interface designs do not fulfill the basic human factors design principles, potentially leading to increased response variability and reduced statistical power when conducting controlled experimental research. In the future, it is essential to develop scripting tools to integrate customizable user interfaces and simulation models, ensuring meeting research requirements.

Li, Ruixuan↗
DOE OSTI2022

The evolution of the Human Systems and Simulation Laboratory in nuclear power research

The events at Three Mile Island in the United States brought about fundamental changes in the ways that simulation would be used in nuclear operations. The need for research simulators was identified to scientifically study human-centered risk and make recommendations for process control system designs. This paper documents the human factors research conducted at the Human Systems and Simulation Laboratory (HSSL) since its inception in 2010 at Idaho National Laboratory. The facility’s primary purposes are to provide support to utilities for system upgrades and to validate modernized control room concepts. In the last decade, however, as nuclear industry needs have evolved, so too have the purposes of the HSSL. Thus, beyond control room modernization, human factors researchers have evaluated the security of nuclear infrastructure from cyber adversaries and evaluated human-in-the-loop simulations for joint operations with an integrated hydrogen generation plant. Lastly, our review presents research using human reliability analysis techniques with data collected from HSSL-based studies and concludes with potential future directions for the HSSL, including severe accident management and advanced control room technologies.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

A Prospective Design Method for Nuclear Power: The Evaluation, Requirements, and Goals Outline for Nuclear (ERGON) Method

Human factors researchers at Idaho National Laboratory (INL) have worked on projects spanning control room modernization, operator support systems, visualization design, and novel system creation. These projects demonstrated the need for an explicit design method for nuclear power. Human factors teams found a high standard in the Human Factors Engineering Program Review Model (NUREG-0711) and needed a design methodology which could be successful in gaining approval. Previous work has been synthesized as the Evaluation, Requirements, and Goals Outline for Nuclear (ERGON) method here. Design tasks are broken into four phases: Context and Orientation, Human Factors Review, Prototyping and Evaluation, Iteration and Improvement. ERGON is intended as a flexible and direct design method for many applications in nuclear power. ERGON has been vetted through collaborative research and development with nuclear utilities and as such, the ERGON method can assist utilities to achieve approval from a NUREG-0711 summative evaluation for HSI implementations.

42 ENGINEERING↗
DOE OSTI2023

Human Factors in Discovery Phase of TRLs and HRLs

With rapid growth in technology, there has been a corresponding growth in research focused on the ways that human-machine interactions can be improved. As part of that work, researchers have explored how human expertise can inform technology design and evaluation. For example, interaction with subject matter experts (SMEs) or end users can help to design and enhance a machine. The human factors of technology release can be divided into five steps: discovery, planning, development, evaluation, and deployment. This framework is a higher-level abstraction of the Human Readiness Levels for technology use and adoption (See, et al., 2018). In this exposition, we discuss how human factors methodologies, principles, and practices can be realized in the first phase, Discovery, of the technology development process.

Jefferson, Brett A.↗
DOE OSTI2021

Using Qualified On-Site Nuclear Power Plant Simulators in Human Factors Validations of Control Room Upgrades

Existing commercial nuclear power plants (NPPs) are valuable assets in the infrastructure portfolio of the United States (U.S.) because they safely and reliably generate about 1/5th of all the electricity used. The instrumentation and control (I&C) systems in commercial NPPs are the ‘eyes and ears' of the operator, allowing operators to maintain situation awareness, thereby allowing the plant to operate safely and efficiently for all phases of operation. Because the I&C systems in NPPs are still mostly based on analog technologies, which are reliable but not especially cost-effective, upgrading the existing I&C in com-mercial NPPs to new digital I&C is very important. Furthermore, simulators and simulation are critical human factors engineering tools for this I&C modernization work. NPP control room simulators are essential test beds to simulate normal, abnormal, and emergency operations that allow human factors researchers to safely and realistically evaluate early design phase prototypes of the upgraded digital I&C, and validate final as-built digital I&C systems that have been modernized deployed in NPPs. This paper describes research performed using qualified on-site NPP simulators to perform human factors validations of digital I&C control room upgrades.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Competitive Metabolism of Polycyclic Aromatic Hydrocarbons (PAHs): An Assessment Using In Vitro Metabolism and Physiologically Based Pharmacokinetic (PBPK) Modeling

Humans are routinely exposed to complex mixtures such as polycyclic aromatic hydrocarbons (PAHs) rather than to single compounds, as are often assessed for hazards. Cytochrome P450 enzymes (CYPs) metabolize PAHs, and multiple PAHs found in mixtures can compete as substrates for individual CYPs (e.g., CYP1A1, CYP1B1, etc.). The objective of this study was to assess competitive inhibition of metabolism of PAH mixtures in humans and evaluate a key assumption of the Relative Potency Factor approach that common human exposures will not cause interactions among mixture components. To test this objective, we co-incubated binary mixtures of benzo[a]pyrene (BaP) and dibenzo[def,p]chrysene (DBC) in human hepatic microsomes and measured rates of enzymatic BaP and DBC disappearance. We observed competitive inhibition of BaP and DBC metabolism and measured inhibition coefficients (K i ), observing that BaP inhibited DBC metabolism more potently than DBC inhibited BaP metabolism (0.061 vs. 0.44 µM K i , respectively). We developed a physiologically based pharmacokinetic (PBPK) interaction model by integrating PBPK models of DBC and BaP and incorporating measured metabolism inhibition coefficients. The PBPK model predicts significant increases in BaP and DBC concentrations in blood AUCs following high oral doses of PAHs (≥100 mg), five orders of magnitude higher than typical human exposures. We also measured inhibition coefficients of Supermix-10, a mixture of the most abundant PAHs measured at the Portland Harbor Superfund Site, on BaP and DBC metabolism. We observed similar potencies of inhibition coefficients of Supermix-10 compared to BaP and DBC. Overall, results of this study demonstrate that these PAHs compete for the same enzymes and, at high doses, inhibit metabolism and alter internal dosimetry of exposed PAHs. This approach predicts that BaP and DBC exposures required to observe metabolic interaction are much higher than typical human exposures, consistent with assumptions used when applying the Relative Potency Factor approach for PAH mixture risk assessment.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Analytical and Performance-Based Evaluation Alternatives to Full-Scope, High-Fidelity Testbeds

One consequence the design and operational differences of advanced reactors is that integrated system validation (ISV) using full-scope, high-fidelity testbed might not be cost-justified or practical. The absence of traditional ISV may pose an issue for the conduct of safety evaluations since it may leave regulators without the information derived from this performance-based testing. The purpose of this research is to identify analytical and performance-based test and evaluation methods that are alternatives to full-scope, high-fidelity testbeds that have traditionally been used for ISV. We developed a method to test validation based on a multi-stage validation (MSV) framework. MSV is an approach to meeting validation objectives through incremental, successive validation activities beginning in the early stages of the design process and continuing through the later stages. MSV doesn't rely solely on late stage validation, rather it accommodates the use of a broad spectrum of analytical and performance-based information and a diversity of testbeds. In addition, MSV embraces the use of information from other types of evaluations and analyses.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

EVALUATION OF HRA METHODOLOGIES FOR APPLICATION IN SDP WORK

This study critically evaluates human reliability analysis (HRA) methodologies applicable to regulatory probabilistic safety assessment (PSA) model, with a particular focus on their role in supporting the significance determination process (SDP) in nuclear safety assessment. Firstly, three widely utilized HRA methods – IDHEAS-ECA, SPAR-H, and ASEP/THERP – were qualitatively and quantitatively assessed. Qualitative assessments were conducted using attributes from the NEA/CSNI/R(2015)1 report, while quantitative evaluations employed regression and correlation analyses to compare predicted human error probabilities (HEPs) against empirical data. Results reveal distinct strengths, for example, IDHEAS-ECA’s robust predictive accuracy and K-HRA’s alignment with operational practices. In addition, dependency analysis and recovery analysis were critically evaluated. For dependency analysis, the methods’ handling of inter-task dependencies and their impact on HEPs were examined, while recovery analysis highlighted strategies for mitigating failure events. Furthermore, strategies were proposed to evaluate performance-shaping factors under conditions of reduced human performance, such as stress, fatigue, or cognitive overload, addressing specific challenges faced in SDP evaluations. Human errors from KINS’s operational performance information system event reports were evaluated as a case study. This study identifies gaps and provides actionable insights to ensure their validity and applicability in SDP HRA applications. This paper is a part of research conducted by KINS, and it should be noted that this result does not represent the regulatory position of KINS.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

Incorporating the Role(s) of Human Actors in Complex System Design for Safety and Security

Traditional systems engineering demonstrates the importance of customer needs in scoping and defining design requirements; yet, in practice, other human stakeholders are often absent from early lifecycle phases. Human factors are often omitted in practice when evaluating and down-selecting design options due to constraints such as time, money, access to user populations, or difficulty in proving system robustness through the inclusion of human behaviors. Advances in systems engineering increasingly include non-technical influences into the design, deployment, operations, and maintenance of interacting components to achieve common performance objectives. Furthermore, such advances highlight the need to better account for the various roles of human actors to achieve desired performance outcomes in complex systems. Many of these efforts seek to infuse lessons and concepts from human factors (enhanced decision-making through Crew Resource Management), systems safety (Rasmussen's “drift toward danger”) and organization science (Giddens' recurrent human acts leading to emergent behaviors) into systems engineering to better understand how socio-technical interactions impact emergent system performance. Safety and security are examples of complex system performance outcomes that are directly impacted by varying roles of human actors. Using security performance of high consequence facilities as a representative use case, this article will outline the System Context Lenses to understand how to include various roles of human actors into systems engineering design. Several exemplar applications of this organizing lenses will be summarized and used to highlight more generalized insights for the broader systems engineering community.

42 ENGINEERING↗
DOE OSTI2020

Data Collection and Analysis Challenges and Mitigation Strategies for Quantitative Human Factors Research Studies in Nuclear Power Plant Modernization

The United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program Plant Modernization Pathway is conducting targeted research and development (R&D) to address aging and reliability concerns with the legacy instrumentation and control and related information systems of the U.S. LWR fleet. In this effort, the application of human factors engineering (HFE) provides an important role in ensuring new digital plant technologies enable broad innovation and business improvement with continued operational safety. Evaluation is a key activity in HFE, which often occurs iteratively through the system design lifecycle. While qualitative methods are important in collecting information of how users perform tasks through observations, quantitative methods are equally important in assessing system design based on performance. In collecting and analyzing this quantitative data, there are notable challenges in the nuclear HFE domain that may threaten the validity and reliability of the inferences made in these studies. Notable challenges include small sample size and limited resources, large error variance and small effect size, an “adding test features to losing degrees of freedom” dilemma, non-normal distribution, and heterogeneity of variance. The results in control room usability studies are often statistically non-significant, which makes it hard to interpret. This work discusses these challenges across different scientific viewpoints and provides real-world examples of these challenges in practice. Collectively, the objective of this work is to position these challenges to the larger data science community as a means of identifying future opportunities to address these issues.

99 GENERAL AND MISCELLANEOUS↗