A Monte Carlo investigation of methods for controlling type I errors with specification searches in structural equation modeling

A standard strategy in structural equation modeling is to conduct multiple Lagrange multiplier (LM) tests after rejection of an initial model. Controlling for Type I error across these tests minimizes the likelihood of including unnecessary additional parameters in the model. Three methods for controlling Type I errors are evaluated using simulated data for factor analytic models: the standard approach which involves testing each parameter at the .05 level, a Bonferroni approach, and a simultaneous test procedure (STP). In the first part of the study, all samples were generated from a population in which all null hypotheses associated with the LM tests were correct. Three factors were manipulated: factor weights, sample size, and number of parameters in the specification search. The standard and the STP approaches yielded overly liberal and overly conservative familywise error rates, respectively, while the Bonferroni approach yielded error rates closer to the nominal level. In the second part of the study, data were generated in which one or more null hypotheses associated with the LM test were incorrect, and the number of parameters in the search was manipulated. Again the Bonferroni method was the best approach in controlling familywise error rate, particularly when the alpha level was adjusted for the number of parameters evaluated at each step.