Analyst variability in labeling of unsupervised classifications

Abstract

Analyst variability in the labeling of unsupervised classifications is tested for Landsat 5 Thematic Mapper image products covering two test sites in southern California. The accuracy of results are tested using samples from a photo interpreted base map of the area. The significance of differences between analysts is indicated by comparing Kappa statistics derived from error matrices. Analyst variability is found to be statistically significant in most cases. Certain analysts provided consistently better results for a given study area or degree of training. This work demonstrates the potential influence of analyst bias on what would otherwise seem to be a fairly objective method and suggests that controls for this subjectivity should be factored into experimental designs.