A comparison of chi-squared statistics for testing homogeneity of survey data: High School and Beyond survey

The performance of several test statistics for comparing vectors of proportions from certain survey data was compared. The statistics were used to analyze a subsample of data from the ‘High School and Beyond’ survey. These tests include the Wald test statistic and the modified Wald test statistic Fw, the chi-squared test statistic XRSB and its modification FRSB, a test X2DMB based on a probability model, and a method of moments approach, X2H. Data were also simulated based on two-stage cluster sampling design and the type I error level, and the power of these tests was obtained for selected combinations of parameter values. The statistics X2DMB, X2RSB, FRSB and X2H performed well both for a small number of clusters or a small number of units within clusters. The power performance of these tests is quite stable. Approximate intervals were constructed for design effect constants. Methods of estimating these constants based on a normality assumption worked best.