The difference between “equivalent” and “not different”

Experimenters frequently wish to establish that populations of units can be considered equivalent to each other, in order to leverage improved knowledge about one population for characterizing the new population or to establish the comparability of items. Equivalence tests have existed for many years, but their use in industry seems to have been largely restricted to biomedical applications, such as for assessing the equivalence of two drugs or protocols. We present the fundamentals of equivalence tests, compare them to traditional two-sample and analysis of variance (ANOVA) tests that are better suited to establishing differences in populations, and propose the use of a graphical summary to compare p values across different thresholds of practically important differences. The methods are illustrated using an example.