The difference between “equivalent” and “not different”
Often, experimenters wish to establish that populations of units can be considered equivalent to each other, in order to leverage improved knowledge about one population for characterizing the new population, or to establish the comparability of items. Equivalence tests have existed for many years, but their use in industry seems to have been largely restricted to biomedical applications, such as for assessing the equivalence of two drugs or protocols. We present the fundamentals of equivalence tests, compare them to traditional two-sample and ANOVA tests that are better suited to establishing differences in populations, and propose the use of a graphical summary to compare p-values across different thresholds of practically important differences.