Constructing Self-Labeled Materials Imaging Datasets from Open Access Scientific Journals with EXSCLAIM!
Due to recent improvements in image resolution and acquisition speeds, materials microscopy is experiencing an explosion in imaging data. Yet, despite the volume of images generated, the overall accessibility landscape is highly fragmented, as researchers who do release images to the public, often only do so as snapshots of their larger private dataset in context of scientific journal publications. The effort to automatically consolidate images and descriptive information from web-based platforms has garnered broad attention from the computer vision, language technologies, and chemistry/materials informatics communities. However, these methods are problematic for scientific figures because over 30% of figures are compound in nature, and it is the individual images themselves, paired with relevant context, that are necessary for construction a proper labeled dataset. To this end, we outline in this paper the design of a software pipeline for the automatic EXtraction, Separation, and Caption-based natural Language Annotation of IMages from scientific figures (EXSCLAIM!). Successful consolidation of materials imaging across literature sources will enhance navigation and searchability of materials microscopy images for both novice and experienced researchers, as well as establish the framework necessary for users to search by images, text, or some combination of both.