Classification improvement by optimal dimensionality reduction when training sets are of small size

Abstract

A computer simulation was performed to test the conjecture that, when the sizes of the training sets are small, classification in a subspace of the original data space may give rise to a smaller probability of error than the classification in the data space itself; this is because the gain in the accuracy of estimation of the likelihood functions used in classification in the lower dimensional space (subspace) offsets the loss of information associated with dimensionality reduction (feature extraction). A number of pseudo-random training and data vectors were generated from two four-dimensional Gaussian classes. A special algorithm was used to create an optimal one-dimensional feature space on which to project the data. When the sizes of the training sets are small, classification of the data in the optimal one-dimensional space is found to yield lower error rates than the one in the original four-dimensional space.