Bloom filter de-identifier

Abstract

A bloom filter is a binary data structure that is designed to be memory efficient and able to determine whether an element is present in a set. The bloom filter is a probabilistic data structure. It can tell you whether an element, or set of elements, is likely to exist within a set. It can also tell you whether an element, or set of elements, does not exist within a set. So false positives are probable, but false negatives are not. Our novel approach utilizes the bloom filter to determine how many bigrams (two letter sequence) each value has in common. The resulting metric approximates the relative similarity of two text values.