Eye aiming behavior during the solution of visual patterns.
Eye movement data used for studying nonvisual processes involved in ocular patterns
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Eye movement data used for studying nonvisual processes involved in ocular patterns
This paper proposes a method for inferring full-scope human reliability data based on the Simplified Human Error Experimental Program (SHEEP) data. It mainly focuses on the human errors observed when using simulators with different complexity levels. In the proposed method, the manner in which human error probabilities (HEPs) change as a result of increasing simulator complexity and how simulator complexity levels are quantified represent key information for inferring full-scope data. In the present study, SHEEP error data pertaining to actual professional operators using Rancor Microworld (Rancor) (i.e., a more simplified simulator) and Compact Nuclear Simulator (CNS) (i.e., a less simplified simulator) were compared with the HuREX error data. An approach to quantifying simulator complexity levels was then proposed based on information theory and acquired eye-tracker data.
The influence of different types of predictor displays in a longitudinal vertical takeoff and landing (VTOL) hover task is analyzed in a theoretical study. Several cases with differing amounts of predictive and rate information are compared. The optimal control model of the human operator is used to estimate human and system performance in terms of root-mean-square (rms) values and to compute optimized attention allocation. The only part of the model which is varied to predict these data is the observation matrix. Typical cases are selected for a subsequent experimental validation. The rms values as well as eye-movement data are recorded. The results agree favorably with those of the theoretical study in terms of relative differences. Better matching is achieved by revised model input data.
Aviation is constantly evolving, mostly due to the integration of automated systems into the National Airspace System. This presents a host of challenges and opportunities. NASA’s Advanced Air Mobility (AAM) project has taken a significant leap forward with a research flight test led by the Integration of Automated Systems (IAS) sub-project. This effort focused on assessing automated flight deck algorithms essential for supporting high-density Urban Air Mobility (UAM) operations. Carrying out a two-ship flight test in UAM Maturity Level 4 scenarios, the team evaluated state-of-the-art algorithms, including Hazard Perception and Avoidance (HPA) and flight path management (FPM) systems. This paper investigates into pilots’ physiological and subjective responses during the flight scenarios conducted. In collaboration with Lockheed Martin Advanced Technology Laboratories (ATL), we collected and analyzed live-flight biometric data using eye tracking, mobile brain imaging, and heart rate sensors. The study provides insights into pilots’ workload and cognitive engagement while navigating automated tools, leveraging both biometric data and post-encounter subjective assessments. The research highlights the interaction between human operators and automated systems, contributing valuable lessons learned about gathering human data in live-flight environments. The knowledge acquired from this study enhances our understanding of human factors in automated flight and informs future studies attempting to undertake similar feats.
Here, discrete wavelet methods, originally formulated in the setting of regularly sampled signals, can be adapted to data defined on a point cloud if some multiresolution structure is imposed on the cloud. A wide variety of hierarchical clustering algorithms can be used for this purpose, and the multiresolution structure obtained can be encoded by a hierarchical tree of subsets of the cloud. Prior work introduced the use of Haar-like bases defined with respect to such trees for approximation and learning tasks on unstructured data. This paper builds on that work in two directions. First, we present an algorithm for constructing Haar-like bases on general discrete hierarchical trees. Second, with an eye towards data compression, we present thresholding techniques for data defined on a point cloud with error controlled in the $L$ $\infty$ norm and in a Hölder-type norm. In a concluding trio of numerical examples, we apply our methods to compress a point cloud dataset, study the tightness of the $L$ $\infty$ error bound, and use thresholding to identify MNIST classifiers with good generalizability.
Quantifying pilot visual behavior allows researchers to determine not only where a pilot is looking and when, but holds implications for specific behavioral tracking when these data are coupled with flight technical performance. Remote eye tracking systems have been integrated into simulators at NASA Langley with effectively no impact on the pilot environment. This paper discusses the installation and use of a remote eye tracking system. The data collection techniques from a complex human-in-the-loop (HITL) research experiment are discussed; especially, the data reduction algorithms and logic to transform raw eye tracking data into quantified visual behavior metrics, and analysis methods to interpret visual behavior. The findings suggest superior performance for Head-Up Display (HUD) and improved attentional behavior for Head-Down Display (HDD) implementations of Synthetic Vision System (SVS) technologies for low visibility terminal area operations. Keywords: eye tracking, flight deck, NextGen, human machine interface, aviation
Human behavior often consists of a series of distinct activities, each characterized by a unique pattern of interaction with the visual environment. This is true even in a restricted domain, such as a piloting an aircraft, where activities with distinct visual signatures might be things like communicating, navigating, and monitoring. We propose a novel analysis method for gaze-tracking data, to perform blind discovery of these hypothetical activities. The method is in some respects similar to recurrence analysis, but here we compare not individual fixations, but groups of fixations aggregated over a fixed time interval. The duration of this interval is a parameter that we will refer to as delta. We assume that the environment has been divided into a set of N different areas-of-interest (AOIs). For a given interval of time of duration delta, we compute the proportion of time spent fixating each AOI, resulting in an N-dimensional vector. These proportions can be converted to integer counts by multiplying by delta divided by the average fixation duration (another parameter that we fix at 280 milliseconds). We compare different intervals by computing the chi-square statistic. The p-value associated with the statistic is the likelihood of observing the data under the hypothesis that the data in the two intervals were generated by a single process with a single set of probabilities governing the fixation of each AOI. The method has been applied to approximately 100 hours of eye movement data collected from pilots in a high-fidelity B747 flight simulator, and the results have been compared to synthetic data in which the each activity is represented as first-order Markov process with random probabilities assigned to the AOIs. Randomly-generated synthetic activities can require thousands of fixations to be discriminated with statistical significance, while the human data can be clustered using averaging windows of some 10's of seconds, suggesting that the actual activities are much more narrowly focused than random Markov models.
Abstract not provided.
Multivariate time-series datasets are intrinsic to the study of dynamic, naturalistic behavior, such as in the applications of finance and motion video analysis. Statistical models provide the ability to identify event patterns in these data under conditions of uncertainty, but researchers must be able to evaluate how well a model uses available information in a dataset for clustering decisions and for uncertainty information. The Hidden Markov Model (HMM) is an established method for clustering time-series data, where the hidden states of the HMM are the clusters. We develop novel methods for quantifying the uncertainty of the performance of and for visualizing the clustering performance and uncertainty of fitting a HMM to multivariate time-series data. We explain the usefulness of uncertainty quantification and visualization with evaluating the performance of clustering models, as well as how information exploitation of time-series datasets can be enhanced. We implement our methods to cluster patterns of scanpaths from raw eye tracking data.
We propose a sequential information maximization model as a general strategy for programming eye movements. The model reconstructs high-resolution visual information from a sequence of fixations, taking into account the fall-off in resolution from the fovea to the periphery. From this framework we get a simple rule for predicting fixation sequences: after each fixation, fixate next at the location that minimizes uncertainty (maximizes information) about the stimulus. By comparing our model performance to human eye movement data and to predictions from a saliency and random model, we demonstrate that our model is best at predicting fixation locations. Modeling additional biological constraints will improve the prediction of fixation sequences. Our results suggest that information maximization is a useful principle for programming eye movements.
This paper describes the results of experiments conducted with human line pilots and a cognitive pilot model during interaction with a new 40 Flight Management System (FMS). The aim of these experiments was to gather human pilot behavior data in order to calibrate the behavior of the model. Human behavior is mainly triggered by visual perception. Thus, the main aspect was to setup a profile of human pilots' visual attention allocation in a cockpit environment containing the new FMS. We first performed statistical analyses of eye tracker data and then compared our results to common results of familiar analyses in standard cockpit environments. The comparison has shown a significant influence of the new system on the visual performance of human pilots. Further on, analyses of the pilot models' visual performance have been performed. A comparison to human pilots' visual performance revealed important improvement potentials.
In this study, we present a novel application of sketch gesture recognition on eye-movement for biometric identification and estimating task expertise.The study was performed for the task of mammographic screening with simultaneous viewing of four coordinated breast views as typically done in clinical practice. Eye-tracking data and diagnostic decisions collected for 100 mammographic cases (25 normal, 25 benign, 50 malignant) and 10 readers (three board certified radiologists and seven radiology residents), formed the corpus for this study.Sketch gesture recognition techniques were employed to extract geometric and gesture-based features from saccadic eye-movements.Our results show that saccadic eye-movement, characterized using sketch-based features, result in more accurate models for predicting individual identity and level of expertise than more traditional eye-tracking features.
Diabetic retinopathy (DR) is a leading cause of blindness and common complication of diabetes. Many diabetic patients take antihypertensive drugs to prevent cardiovascular problems, but these drugs may have unintended consequences on eyesight. Six common classes of antihypertensive drug are angiotensin converting enzyme (ACE) inhibitors, alpha blockers, angiotensin receptor blockers (ARBs), -blockers, calcium channel blockers, and diuretics. Analysis of medical history data might indicate which of these drugs provide safe blood pressure control, and a literature review is often used to guide such analyses. Beyond manual reading of relevant publications, we sought to identify quantitative trends in literature from the biomedical database PubMed to compare with quantitative trends in the clinical data. By recording and analyzing PubMed search results, we found wide variation in the prevalence of each antihypertensive drug in DR literature. Drug classes developed more recently such as ACE inhibitors and ARBs were most prevalent. We also identified instances of change-over-time in publication patterns. We then compared these literature trends to a dataset of 500 diabetic patients from the UT Hamilton Eye Institute. Data for each patient included class of antihypertensive drug, presence and severity of DR. Graphical comparison revealed that older drug classes such as diuretics, calcium channel blockers, and -blockers were much more prevalent in the clinical data than in the DR and antihypertensive literature. Finally, quantitative analysis of the dataset revealed that patients taking -blockers were statistically more likely to have DR than patients taking other medications, controlling for presence of hypertension and year of diabetes onset. This finding was concerning given the prevalence of -blockers in the clinical data. We determined that clinical use of -blockers should be minimized in diabetic patients to prevent retinal damage.
Heuristics have been used since the early 1990's in the Human Computer Interaction (HCI) domain, for evaluating user interfaces. Most notably, heuristics developed by Nielsen and Molich (1990) have been used by the HCI community. Forsell and Johansson (2010) used the same methodology as Nielsen and Molich and developed a set of 10 heuristics for information visualization. Currently, neither the information visualization or the visual analytics community has adopted these heuristics. However, several studies have been conducted using heuristics. Hearst et al. (2015) found that using the heuristics for visualizations along with asking questions about the data resulted in complementary results as the questions about the data could compensate for heuristics that were difficult to understand and apply. Heli Väätäjä et al. (2016) found that heuristics related to interaction, veracity, and aesthetics needed to be added to the Forsell and Johansson (2010) set. They also noted that a lack of domain knowledge caused the evaluators issues in conducting a heuristic evaluation. We conducted a controlled experiment to determine what factors might impact the expert evaluators' capability to conduct a heuristic evaluation of a static information visualization. In particular, we wanted to understand how their familiarity with the visualizations and their experience in conducting heuristic evaluations impacted their evaluations of visualizations. We recruited 10 participants who were domain experts in the visualization field each having experience with one of more of the following: user experience design or research, user interface development, and human computer interaction research. We used a heuristic set proposed by Forsell and Johansson. Participants were also asked to complete a demographic survey in which they rated their experience with each visualization type generally, their experience conducting heuristic evaluations, their experience using each visualization type to extract information, and their experience developing and creating each visualization type. The participants were shown a series of 5 common visualizations (Scatterplots, Sunbursts, Tree Maps, Parallel Sets, and Area Graphs) all displaying the VAST 2008 Mini Challenge Two- Migrant Boats data . Participants were given a suggested analytical question related to each visualization to keep in mind as they applied the 10 heuristics to each visualization. They were asked if they could find any usability problems in the visualizations and which heuristic(s) helped them to identify these problems. The participants were asked to rate how relevant each heuristic was for evaluating each visualization along with rating how confident they felt in their ability to apply the heuristic to evaluate each visualization. At the conclusion of the study, participants provided qualitative feedback on the usability issues of each visualization. Additionally, we collected eye tracking data on 6 of the 10 participants. We will be using this data to determine how long participants looked at the various visualizations and what parts of the visualizations they focused on. We had two hypotheses for this empirical evaluation: 1.The Forsell and Johansson heuristics of Orientation and help, Data set reduction, and Prompting are less useful for static visualizations because they require some interaction in order to test completely. 2.Participants that do not have experience with heuristic evaluation will provide lower evaluation scores for the heuristics compared to those that have experience. Related to hypothesis 1, as our study used only static visualizations, interaction-related usability challenges should be harder to identify. This should result in lower usefulness ratings for those heuristics. Preliminary results indicate that participants found it more difficult to apply heuristics to unfamiliar visualizations.
Storyline visualizations offer an approach that promises to capture the spatio-temporal characteristics of individual observers and simultaneously illustrate emerging group behaviors. We develop a visual analytics approach to parsing, aligning, and clustering fixation sequences from eye tracking data. Visualization of the results captures the similarities and differences across a group of observers performing a common task. We apply our storyline approach to visualize gaze patterns of people watching dynamic movie clips. Storylines mitigate some of the shortcomings of existent spatio-temporal visualization techniques and, importantly, continue to highlight individual observer behavioral dynamics.
When solving problems, multi-person airline crews can choose whether to work together, or to address different aspects of a situation with a divide and conquer strategy. Knowing which of these strategies is most effective may help airlines develop better procedures and training. This paper concentrates on joint attention as a measure of crew coordination. We report results obtained by applying cross recurrence analysis to eye movement data from two-person crews, collected in a flight simulator experiment. The analysis shows that crews exhibit coordinated gaze roughly one sixth of the time, with a tendency for the captain to lead the first officers visual attention. The degree to which crews coordinate their gaze is not significantly correlated with performance ratings assigned by instructors; further research questions and approaches are discussed.
Prior research has shown that physicians’ medical decisions can be influenced by sequential context, particularly in cases where successive stimuli exhibit similar characteristics when analyzing medical images. This type of systematic error is known to psychophysicists as sequential context effect as it indicates that judgments are influenced by features of and decisions about the preceding case in the sequence of examined cases, rather than being based solely on the peculiarities unique to the present case. We determine if radiologists experience some form of context bias, using screening mammography as the use case. To this end, we explore correlations between previous perceptual behavior and diagnostic decisions and current decisions. We hypothesize that a radiologist’s visual search pattern and diagnostic decisions in previous cases are predictive of the radiologist’s current diagnostic decisions. To test our hypothesis, we tasked 10 radiologists of varied experience to conduct blind reviews of 100 four-view screening mammograms. Eye-tracking data and diagnostic decisions were collected from each radiologist under conditions mimicking clinical practice. Perceptual behavior was quantified using the fractal dimension of gaze scanpath, which was computed using the Minkowski–Bouligand box-counting method. To test the effect of previous behavior and decisions, we conducted a multifactor fixed-effects ANOVA. Further, to examine the predictive value of previous perceptual behavior and decisions, we trained and evaluated a predictive model for radiologists’ current diagnostic decisions. ANOVA tests showed that previous visual behavior, characterized by fractal analysis, previous diagnostic decisions, and image characteristics of previous cases are significant predictors of current diagnostic decisions. Additionally, predictive modeling of diagnostic decisions showed an overall improvement in prediction error when the model is trained on additional information about previous perceptual behavior and diagnostic decisions.
Transcranial direct current stimulation (tDCS) is a noninvasive form of brain stimulation that involves passing a weak electrical current between electrodes on the scalp to modulate underlying neural tissue. TDCS has been shown to modulate cognition in a variety of domains, including memory, attention, and visual processing. Prior work from our laboratory has shown positive effects of tDCS on learning to detect target objects hidden in complex naturalistic visual scenes and learn rules for categorizing images, though the mechanism for these benefits remains unknown. One possibility is that tDCS optimizes visual search by modulating visual attention or via the reduction in search errors. One method of quantifying visual attention is to use eye tracking to record search patterns to determine if and how visual search is adjusted under verum stimulation conditions. Eye tracking data allows classification of errors into error types, including sampling errors (failing to look in the relevant region), recognition errors (looking at the critical portion of a scene, but failing to recognize it as such as evidenced by visual fixation), and decision-making errors (fixating on the relevant portion of a scene, but making the wrong determination). Our results indicate that the benefit tDCS confers on visual search for targets stems from the reduction in decision-making errors when targets are present (Cohen’s d = 0.86). Also reported is a replication of previous findings showing a tDCS-dependent improvement in learning this task, learning score (Cohen’s d = 0.88); d’ (Cohen’s d = 1.00). This provides support for moving tDCS into the application space by pairing it with analysts who are concerned with the type of search error that is corrected via stimulation.