COE Task Order 4. Final Report: Automated Threat Recognition
Lawrence Livermore National Laboratory (LLNL) was tasked by the Department of Homeland Security (DHS) Science and Technology Directorate (S&T) with developing an automated target recognition algorithm for finding objects of interest in checked baggage at airports. The project used x-ray computed tomography (CT) data collected on a medical scanner of various simulated checked bags. Target objects of interest include clay, rubber, and saline in various concentrations. The final results are 93.9% probability of detection (P D ) with 11.9% probability of false alarm (PFA) on all targets and 100% detection for pseudo target sheets. This result is achieved by using a three-stage algorithm made up of segmentation, post-processing, and feature extraction and object classification. The segmentation proved to be the most important stage and it involves merging two distinct segmentation results followed by a set of algorithms for cleaning up the merged segmentations. The first segmentation algorithm operates on planar 10x10 voxel slabs (two- dimensional segments) in the three spatial planes. The segmenter calculates a probability that each slab in the image is a part of a target, determines if the probability was above a threshold, and then merges all connected slabs together. The second segmentation algorithm is an ensemble segmenter, which generates a set of possible segmentations through random permutations and computes an average behavior. The distinct properties of these segmenters results in a merged segmentation with superior results than either segmenter alone. The merged segmentation is “cleaned up” using post processing techniques to achieve the final labeled images. The final classification stage made use of a random forest classifier algorithm to discriminate targets from non-targets based on a set of features of the segmented objects partially from the voxel slabs and partially from the individual voxels. Features include standard statistics such as mean, median, and standard deviations of the density, features capturing some aspects of texture, and some features about the surrounding bag structure. In addition to the detection and false alarm results above, we show how the algorithms can be trained to “overfit” to the limited data set provided and achieve near perfect results (P D =100% and P FA =0%). This over-training is not recommended for long-term operation, but instead is meant to demonstrate how test design is an important factor in aviation security. Possible improvements to the algorithm and the testing methodology (including blind testing) are also presented.