Using partial ranking information in the design of small-sample comparisons

Random assignment of experimental units to treatment and control groups is a conventional device to create unbiased comparisons. However, when sample sizes are small and the units differ considerably, there is a significant risk that randomization will create seriously unbalanced partitions of the units into treatment and control groups. We develop and evaluate an alternative to complete randomization for small-sample comparisons involving ordinal data with partial information on ranks of units. For instance, we might know that, of eight units, Rank(A) < Rank(C), Rank(A) < Rank(E) and Rank(D) < Rank(H). We develop an efficient computational procedure to use such information as the basis for restricted randomization of units to the treatment group. We compare our methods to complete randomization in the context of the Mann-Whitney test. With sufficient ranking information, the restricted randomization results in more powerful comparisons.