Different evolution acceptance instruments lead to different research findings

Background: Despite widespread concern about the differential measurement of evolution acceptance among researchers, no one has systematically explored how instrument choice can impact research results and conclusions in evolution education studies. In this study, we administered six evolution acceptance instruments in a single survey to students in undergraduate biology courses at universities in Arizona, Colorado, and Utah. We conducted separate analyses with the same students for the six different evolution acceptance instruments to understand how different results and conclusions may arise based on different evolution acceptance instruments used. Results: We found statistically significant differences in levels of evolution acceptance across the three student populations when using a human evolution acceptance instrument, but not when using a microevolution acceptance instrument. Further, significance/effect sizes of variables associated with evolution acceptance differed beyond sampling variation depending on the evolution acceptance instrument used. The results of analyses using different evolution acceptance instruments were most often dissimilar when examining the effect of evolution understanding and identifying as Protestant/Mormon on evolution acceptance. Conclusions: We found that different instruments used to measure evolution acceptance sometimes led to different research results and conclusions. The extent to which variables predicted evolution acceptance was dependent on the instrument used to measure acceptance, which has the potential to explain over 30 years of conflicting research on the relationship between evolution acceptance and understanding. These results indicate that before researchers may be able to determine how to best improve evolution acceptance, the evolution education community may need to articulate a consistent definition of evolution acceptance and identify a singular valid and reliable instrument to quantify evolution acceptance so results can be compared across studies.