D-statistics and the ABBA-BABA test, explained
The D-statistic, or ABBA-BABA test, detects gene flow from site patterns in four populations. How it works, how to read it, and its limits.
Updated 3 min read
The D-statistic tests whether four populations fit a tree, by comparing two kinds of shared alleles: ABBA and BABA. If the tree holds, the two patterns are equally common and D is zero. If one pattern is in excess, there was gene flow, or the tree is wrong.
The test is also called the ABBA-BABA test or Patterson's D. Green et al. 2010 used it to show that people outside Africa carry Neanderthal ancestry, and Durand et al. 2011 described its statistics in detail.
The four populations
D(P1, P2; P3, Outgroup) takes four populations. The usual setup is:
- P1 and P2: two closely related populations, for example two present-day groups.
- P3: a population that may have exchanged genes with one of them.
- Outgroup: a distant population that tells which allele is ancestral, for example chimpanzee or
Mbuti.
ABBA and BABA
At each SNP, write A for the outgroup's allele and B for the other allele. Look only at sites where P3 carries B.
- ABBA: P1 has A, P2 has B, P3 has B. P2 and P3 share the derived allele.
- BABA: P1 has B, P2 has A, P3 has B. P1 and P3 share the derived allele.
Under the tree ((P1, P2), P3), both patterns come only from incomplete lineage sorting, and they are equally likely. D is their normalised difference:
D = (ABBA − BABA) / (ABBA + BABA)
With allele frequencies in place of single genomes, the same formula becomes a normalised f4-statistic.
Reading D
- D ≈ 0 (|Z| < 3): P1 and P2 are symmetrically related to P3. The tree is consistent with the data.
- D significantly positive: P2 shares more derived alleles with P3 than P1 does. P2 (or its ancestors) and P3 exchanged genes.
- D significantly negative: P1 shares more with P3.
Signs depend on the order of populations, and different programs order them differently. Always state which population pair is "excess".
The Neanderthal example
Green et al. 2010 computed D with P1 and P2 as pairs of present-day humans, P3 as the Neanderthal genome and chimpanzee as the outgroup. When P1 was an African genome and P2 a European or East Asian genome, D was significantly different from zero: non-Africans share more derived alleles with Neanderthals. When both P1 and P2 were non-Africans, D was close to zero. The best explanation was interbreeding between Neanderthals and the ancestors of non-Africans.
What D cannot tell you
- Direction. D shows that P3 is closer to one of P1 and P2, but not which way genes moved.
- Amount. D is not an admixture proportion. Use an f4-ratio or qpAdm to estimate one.
- Structure versus gene flow. Old population structure in the ancestors can produce a non-zero D without later gene flow.
Running D-statistics
ADMIXTOOLS runs D-statistics with qpDstat, and ADMIXTOOLS 2 with qpdstat. Nexju runs single D-statistics on every plan and f-statistic tables on the Pro plan, on the AADR's ancient and modern populations or your own genome.
References
- Green R. E. et al. 2010. A draft sequence of the Neandertal genome. Science 328. doi:10.1126/science.1188021
- Durand E. Y. et al. 2011. Testing for ancient admixture between closely related populations. Molecular Biology and Evolution 28. doi:10.1093/molbev/msr048
- Patterson N. et al. 2012. Ancient admixture in human history. Genetics 192. doi:10.1534/genetics.112.145037