Genetic incompatibilities between mitochondria and nuclear genes: Effect on gene flow and speciation

The process of speciation is, according to the biological species concept, the reduction in gene flow between genetically diverging populations. Most of the previous theoretical studies analyzed the effect of nuclear genetic incompatibilities on gene flow. There is, however, an increasing number of empirical examples suggesting that cytoplasmically inherited genetic elements play an important role in speciation. Here, we present a theoretical analysis of mitochondrial driven speciation, in which genetic incompatibilities occur between mitochondrial haplotypes and nuclear alleles. Four population genetic models with mainland-island structure were analyzed that differ with respect to the type of incompatibility and the underlying genetics. Gene flow reduction was measured on selectively neutral alleles of an unlinked locus and quantified by the effective migration rate. Analytical formulae for the different scenarios were derived using the fitness graph method. For the models with haploid genetics, we found that mito-nuclear incompatibilities (MtNI) are as strong as nuclear-nuclear incompatibilities (NNI) in reducing gene flow at the unlinked locus, but only if males and females migrate in equal number. For models with diploid genetics, we found that MtNI reduce gene flow stronger than NNI when incompatibilities are recessive, but weaker when they are dominant. For both haploid and diploid MtNI, we found that gene flow reduction is stronger if females are the migrating sex, but weaker than NNI when males are the migrating sex. These results encourage further examination on the role of mitochondria on genetic divergence and speciation and point toward specific factors (e.g., migrating sex) that could be the focus of an empirical test.