Choosing right populations (outgroups) for qpAdm

Right populations must relate differently to your sources, and none of them may share drift with the target that the sources lack. If the right populations cannot tell your sources apart, qpAdm cannot either, and many wrong models will pass.

The right populations (also called references or outgroups) are the viewpoints from which qpAdm compares the target and the sources. They are the part of a model that needs the most thought.

What the right populations do

qpAdm computes f4-statistics of the target and of each source against pairs of right populations. Each right population adds one dimension in which the sources can differ. Two sources that look the same from every right population are the same to qpAdm.

So a good right panel:

1. Has enough populations. You need at least one more right population than you have sources. In practice, use 6 to 12.
2. Separates the sources. For every pair of sources, some right population is closer to one than to the other.
3. Is not downstream of the target. No right population may have received gene flow from the target, or from a population close to the target, after the admixture you are modeling.

The base population (R0)

The first right population is the base. qpAdm measures all the other right populations against it. Use a population that is equally distant from all of your sources. For models of Eurasians, that is usually a deeply diverged African population such as Mbuti.

A starting panel for Eurasian models

A common starting panel for western Eurasian targets, with AADR labels:

• Mbuti: Central African; the base.
• Russia_UstIshim_IUP: an early Upper Paleolithic Siberian, about 45,000 years ago.
• Russia_Kostenki_UP: early Upper Paleolithic European hunter-gatherers.
• Russia_Malta_UP: Ancient North Eurasian, the Mal'ta boy.
• Georgia_KotiasKlde_Mesolithic: Caucasus hunter-gatherers.
• Iran_GanjDareh_N: Iranian Neolithic farmers.
• Israel_Natufian: Levantine hunter-gatherers.
• Han, Papuan, Karitiana: East Asian, Oceanian and Native American outgroups.

Change the panel to fit your sources. If two of your sources are both European farmers, add a population that one of them shares more drift with. If one of the right populations is a close relative of a source, remove it.

Rotation: letting the data choose

In the rotating approach, you start with a pool of candidate populations. Each model takes some as sources and puts all the others in the right list. A source that fits while its close relatives sit on the right is well supported. Many recent ancient DNA studies use rotation to avoid choosing the right panel by hand.

Nexju can rotate for you: guided rotation swaps one source slot at a time, and exhaustive rotation tries every combination from your pool. Nexju previews each rotation before it runs, so you can see how many models it will fit.

Common mistakes

• Using a population related to the target as a right population. For example, using modern French as a right population when the target is English. The shared recent drift makes models fail, or fit for the wrong reason.
• Too few right populations. With three sources and four right populations, the test has little power.
• A low-coverage right population. One genome with 50,000 SNPs adds noise and reduces the SNPs every statistic can use.
• Changing the panel until a model passes. Fix the panel before you look at the results, or use rotation.

References

• Harney É. et al. 2021. Assessing the performance of qpAdm. Genetics 217. doi:10.1093/genetics/iyaa045
• Lazaridis I. et al. 2014. Ancient human genomes suggest three ancestral populations for present-day Europeans. Nature 513. doi:10.1038/nature13673