Assessing population structure: F_ST and related measures

Although F_ST is widely used as a measure of population structure, it has been criticized recently because of its dependency on within-population diversity. This dependency can lead to difficulties in interpretation and in the comparison of estimates among species or among loci and has led to the development of two replacement statistics, F'_ST and D. F'_ST is the normal F_ST standardized by the maximum value it can obtain, given the observed within-population diversity. D uses a multiplicative partitioning of diversity, based on the effective number of alleles rather than on the expected heterozygosity. In this study, we review the relationships between the three classes of statistics (F_ST, F'_ST and D), their estimation and their properties. We illustrate the relationships between the statistics using a data set of estimates from 84 species taken from the last 4 years of Molecular Ecology. As with F_ST, unbiased estimators are available for the two new statistics D and F'_ST. Here, we develop a new unbiased F'_ST estimator based on G_ST, which we call G''_ST. However, F'_ST can be calculated using any F_ST estimator for which the maximum value can be obtained. As all three statistics have their advantages and their drawbacks, we recommend continued use of F_ST in combination with either F'_ST or D. In most cases, F'_ST would be the best choice among the latter two as it is most suited for inferences of the influence of demographic processes such as genetic drift and migration on genetic population structure.