Search NASA⌕ Search

Engineering topics

Wenskovitch, John E.

Publications and source records attributed to Wenskovitch, John E..

7 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Characterizing Interaction Uncertainty in Human-Machine Teams

With the increasing use and adoption of artificial intelligence (AI), the reliability of modern data systems will be driven by a tighter teaming between human experts and intelligent machine teammates. As in the case of human-human teams, the success of human-machine teams will also rely on clear communication about mutual goals and actions. In this paper, we combine related literature from cognitive psychology, human-machine teaming, uncertainty in data analysis, and multi-agent systems to propose a new form of uncertainty: interaction uncertainty for characterizing bidirectional communication in human-machine teams. We map the causes and effects of interaction uncertainty and outline potential ways to mitigate uncertainty for mutual trust in a high-consequence real-world scenario.

uncertainty, data analytics, interaction, trust, h↗
DOE OSTI2023

Human Factors for Machine Learning in Astronomy

In this work, we present a collection of human-centered pitfalls that can occur when using machine learning tools and techniques in modern astronomical research, and we recommend best practices in order to mitigate these pitfalls. Human concerns affect the adoption and evolution of machine learning (ML) techniques in both existing workflows and work cultures. We use current and future surveys such as ZTF and LSST, the data that they collect, and the techniques implemented to process that data as examples of these challenges and the potential application of these best practices, with the ultimate goal of maximizing the discovery potential of these surveys.

astronomy, machine learning, human factors, trust,↗
DOE OSTI2022

A Real-Time Operations Manual for the IEEE 118 Bus Transmission Model

This document presents a sample operations manual for the IEEE 118 Bus Model, which is a synthetic test case developed in 1962 from a section of the transmission grid operated by American Electric Power (AEP). The model is one of the most used synthetic test cases for development of power system applications and is referenced by over 9000 papers. However, the model lacks any context for use with real-time energy management system (EMS) applications, including common considerations, such as generator ramp rates, reactive capabilities, operating limits, and other information typically used by power system operators for real-time decision making. This manual divides the IEEE 118 Bus Model into three operating areas, defines various operating limits, and sets recommended operating procedures for responding to a few types of emergency operating conditions. The manual can be used in support of a wide variety of human-in-the-loop evaluation methodologies for new advanced power applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Operator Insights and Usability Evaluation of Machine Learning Assistance for Power Grid Contingency Analysis

Introducing machine learning (ML) assistance into any established process comes with adoption barriers, including entrenched procedures, technological and human readiness levels, human-machine trust, and work culture resistance to change. These barriers are even greater in critical operations such as operating a national or regional power grid, in which both regulatory frameworks and the importance of maintaining reliability levels causes additional resistance to the adoption of new computational support. Developers of future systems and job aides must consider not only technical aspects, but also whether new systems are usable by power system operators. This work presents the methodology and results of a study to evaluate the usability and readiness of a prototype recommender system for power grid contingency analysis. We explore operator cognitive load and evaluate operator performance when solving a collection of scenarios both with and without recommender assistance. We also examine operator trust in the system. We report insights gained on the readiness of the system using a collection of evaluation techniques.

Human-Machine Teaming, Power Systems, usability ev↗
DOE OSTI2022

A Methodology for Evaluating Operator Usage of Machine Learning Recommendations for Power Grid Contingency Analysis

This work presents the application of a methodology to measure domain expert trust and workload, elicit feedback, and understand the technological usability and impact when a machine learning assistant is introduced into contingency analysis for real-time power grid simulation. The goal of this framework is to rapidly collect and analyze a broad variety of human factors data in order to accelerate the development and evaluation loop for deploying machine learning applications. We describe our methodology and analysis, and we discuss insights gained from a pilot participant about the current usability state of an early technology readiness level (TRL) artificial neural network (ANN) recommender.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Beyond Visual Analytics: Human-Machine Teaming for AI-Driven Data Sensemaking

"Detect the expected, discover the unexpected" was the founding principle of the field of visual analytics. This mantra implies that human stakeholders, like a domain expert or data analyst, could leverage visual analytics techniques to seek answers to known unknowns and discover unknown unknowns in the course of the data sensemaking process. We argue that in the era of AI-driven automation, we need to recalibrate the roles of humans and machines (e.g., a machine learning model) as teammates. We posit that by realizing human-machine teams as a stakeholder unit, we can better achieve the best of both worlds: automation transparency and human reasoning efficacy. However, this also increases the burden on analysts and domain experts towards performing more cognitively demanding tasks than what they are used to. In this paper, we reflect on the complementary roles in a human-machine team through the lens of cognitive psychology and map them to existing and emerging research in the visual analytics community. We discuss open questions and challenges around the nature of human agency and analyze the shared responsibilities in human-machine teams.

Sensemaking, human-machine teaming, agency, artifi↗