Engineering topics

Dasgupta, Aritra

Publications and source records attributed to Dasgupta, Aritra.

9 records

DOE OSTI2024

Characterizing Interaction Uncertainty in Human-Machine Teams

With the increasing use and adoption of artificial intelligence (AI), the reliability of modern data systems will be driven by a tighter teaming between human experts and intelligent machine teammates. As in the case of human-human teams, the success of human-machine teams will also rely on clear communication about mutual goals and actions. In this paper, we combine related literature from cognitive psychology, human-machine teaming, uncertainty in data analysis, and multi-agent systems to propose a new form of uncertainty: interaction uncertainty for characterizing bidirectional communication in human-machine teams. We map the causes and effects of interaction uncertainty and outline potential ways to mitigate uncertainty for mutual trust in a high-consequence real-world scenario.

uncertainty, data analytics, interaction, trust, h↗

DOE OSTI2024

Forte: An Interactive Visual Analytic Tool for Trust-Augmented Net-Load Forecasting

Accurate net-load forecasting is vital for energy planning, aiding decisions on trade and load distribution. However, assessing the performance of forecasting models across diverse input variables, like temperature and humidity, remains challenging, particularly for eliciting a high degree of trust in the model outcomes. In this context, there is a growing need for data-driven technological interventions to aid scientists in comprehending how models react to both noisy and clean input variables, thus shedding light on complex behaviors and fostering confidence in the outcomes. In this paper, we present Forte, a visual analytics-based application to explore deep probabilistic net-load forecasting models across various input variables and understand the error rates for different scenarios. With carefully designed visual interventions, this web-based interface empowers scientists to derive insights about model performance by simulating diverse scenarios, facilitating an informed decision-making process. We discuss observations made using Forte and demonstrate the effectiveness of visualization techniques to provide valuable insights into the correlation between weather inputs and net-load forecasts, ultimately advancing grid capabilities by improving trust in forecasting models.

Bhattacharjee, Kaustav↗

DOE OSTI2021

Beyond Visual Analytics: Human-Machine Teaming for AI-Driven Data Sensemaking

"Detect the expected, discover the unexpected" was the founding principle of the field of visual analytics. This mantra implies that human stakeholders, like a domain expert or data analyst, could leverage visual analytics techniques to seek answers to known unknowns and discover unknown unknowns in the course of the data sensemaking process. We argue that in the era of AI-driven automation, we need to recalibrate the roles of humans and machines (e.g., a machine learning model) as teammates. We posit that by realizing human-machine teams as a stakeholder unit, we can better achieve the best of both worlds: automation transparency and human reasoning efficacy. However, this also increases the burden on analysts and domain experts towards performing more cognitively demanding tasks than what they are used to. In this paper, we reflect on the complementary roles in a human-machine team through the lens of cognitive psychology and map them to existing and emerging research in the visual analytics community. We discuss open questions and challenges around the nature of human agency and analyze the shared responsibilities in human-machine teams.

Sensemaking, human-machine teaming, agency, artifi↗

DOE OSTI2020

Towards Trust-Augmented Visual Analytics for Data-Driven Energy Modeling

The promise of data-driven predictive modeling is being increasingly realized in various science and engineering disciplines, where experts are used to the more conventional, simulation-driven modeling practices. However, trust remains a bottleneck for greater adoption of machine learning-based models for domain experts, who might not be necessarily trained in data science. In this paper, we focus on the building energy domain, where physics-based simulations are being complemented or replaced by machine learning-based methods for forecasting energy supply and demand at various spatio-temporal scales. We study the trust problem in close collaboration with energy scientists and engineers and describe how visual analytics can be leveraged for alleviating this trust bottleneck for stakeholders with varying degrees of expertise and analytics goals in this domain.

Kandakatla, Akshith R.↗

DOE OSTI2019

Separating the Wheat from the Chaff: Comparative Visual Cues for Transparent Diagnostics of Competing Models

Experts in data and physical sciences have to regularly grapple with the problem of competing models. Be it analytical or physics-based models, a cross-cutting challenge for experts is to reliably diagnose which model outcomes appropriately predict or simulate real-world phenomena. Expert judgment involves reconciling information across many, and often, conflicting criteria that describe the quality of model outcomes. In this paper, through a design study with climate scientists, we develop a deeper understanding of the problem and solution space of model diagnostics, resulting in the following contributions: i) a problem and task characterization using which we map experts’ model diagnostics goals to multi-way visual comparison tasks, ii) a design space of comparative visual cues for letting experts quickly understand the degree of disagreement among competing models and gauge the degree of stability of model outputs with respect to alternative criteria, and iii) design and evaluation of MyriadCues, an interactive visualization interface for exploring alternative hypotheses and insights about good and bad models by leveraging comparative visual cues. Here, we present case studies and subjective feedback by experts, which validate how MyriadCues enables more transparent model diagnostic mechanisms, as compared to the state of the art.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2018

Experts’ Familiarity Versus Optimality of Visualization Design: How Familiarity Affects Perceived and Objective Task Performance

Experts, cutting across domains like biology, climate science, cyber, energy, etc., frequently use visualizations as the principal medium for making analytical judgments or for presenting the results of their analysis. However, scientists are often skeptical about adopting new visualization methods over familiar ones, although the latter might be perceptually sub-optimal. This is due to the use of the familiarity heuristic, where the perceived cognitive ease in processing familiar representations of information leads scientists to undermine the effect of visualization best practices. Recent studies have shown that this often results in a discrepancy between scientists’ perceived and actual performance quality. It has also been shown that in some cases, participatory design sessions and qualitative and quantitative user studies are able to mitigate the effects of such bias. In this paper, we discuss the potential causes and effects of familiarity related biases with examples from recent studies and reflect on the related research questions.

visualization, climate science, cognitive bias, fa↗

DOE OSTI2018

Characterizing the Relative Importance Assigned to Physical Variables by Climate Scientists when Assessing Atmospheric Climate Model Fidelity

Evaluating a climate model’s fidelity (ability to simulate observed climate) is a critical step in establishing confidence in the model’s suitability for future climate projections, and in tuning climate model parameters. Model developers use their judgement in determining which trade-offs between different aspects of model fidelity are acceptable. However, little is known about the degree of consensus in these evaluations, and whether experts use the same criteria when different scientific objectives are defined. Here, we report on results from a broad community survey studying expert assessments of the relative importance of different output variables when evaluating a global atmospheric model’s mean climate. We find that experts adjust their ratings of variable importance in response to the scientific objective, for instance, scientists rate surface wind stress as significantly more important for Southern Ocean climate than for the water cycle in the Asian watershed. There is greater consensus on the importance of certain variables (e.g., shortwave cloud forcing) than others (e.g., aerosol optical depth). We find few differences in expert consensus between respondents with greater or less climate modeling experience, and no statistically significant differences between the responses of climate model developers and users. In conclusion, the concise variable lists and community ratings reported here provide baseline descriptive data on current expert understanding of certain aspects of model evaluation, and can serve as a starting point for further investigation, as well as developing more sophisticated evaluation and scoring criteria with respect to specific scientific objectives.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2017

A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations

In this paper, we propose a visual analytics workflow to help data scientists explore, diagnose, and understand the decisions made by a binary classifier. The approach leverages “instance-level explanations”, measures of local feature relevance that explain single instances and uses them to build a set of visual representations that guide the users in their investigation. The workflow is based on three main visual representations and steps: one based on aggregate statistics to see how data distributes across correct or incorrect decisions; one based on explanations to understand which features are used to make these decisions; and one based on raw data, to derive insights on potential root causes for the observed patterns. The work is validated through a long-term collaboration with a group of healthcare professionals who used our method to make sense of machine learning models they developed. The case studies from this collaboration demonstrate that the proposed method helps experts derive useful knowledge about the model and the phenomena it describes and also generates useful hypotheses on how a model can be improved.

Krause, Josua↗

DOE OSTI2017

Human Factors in Streaming Data Analysis: Challenges and Opportunities for Information Visualization: Human Factors in Streaming Data Analysis

State-of-the-art visual analytics models and frameworks mostly assume a static snapshot of the data, while in many cases it is a stream with constant updates and changes. Exploration of streaming data poses unique challenges as machine-level computations and abstractions need to be synchronized with the visual representation of the data and the temporally evolving human insights. In the visual analytics literature, we lack a thorough characterization of streaming data and analysis of the challenges associated with task abstraction, visualization design, and adaptation of the role of human-in-the-loop for exploration of data streams. We aim to fill this gap by conducting a survey of the state-of-the-art in visual analytics of streaming data for systematically describing the contributions and shortcomings of current techniques and analyzing the research gaps that need to be addressed in the future. Our contributions are: i) problem characterization for identifying challenges that are unique to streaming data analysis tasks, ii) a survey and analysis of the state-of-the-art in streaming data visualization research with a focus on the visualization design space for dynamic data and the role of the human-in-the-loop, and iii) reflections on the design-trade-offs for streaming visual analytics techniques and their practical applicability in real-world application scenarios.

streaming data↗