What is FST? Measuring genetic distance between populations
FST is the share of genetic variation between populations. What FST values mean in humans, the Hudson estimator, and how FST differs from f2.
Updated 2 min read
FST is the share of genetic variation that lies between populations rather than within them. An FST of 0 means the populations have the same allele frequencies. An FST of 1 means they share no alleles at all. Human populations have low FST: a few percent between continents, and much less between neighbours.
FST is also called the fixation index. Sewall Wright defined it in the 1940s and 1950s, and it is still the most widely reported measure of genetic distance between populations.
What FST measures
At each SNP, compare the variation in allele frequency between two populations with the total variation. If the two populations have very different frequencies, a large share of the variation is between them, and FST is high. Averaged over many SNPs, FST reflects how much drift separates the populations since they split, and how little gene flow joined them.
Typical values in humans
- Between continents, for example West Africans and East Asians: about 0.1 to 0.2.
- Within a continent, for example between European countries: usually below 0.01, often below 0.005.
- Between isolated and neighbouring groups, for example Sardinians and mainland Europeans: higher than between mainland neighbours, because of the island's drift.
These values are small. Most human genetic variation is within populations, not between them.
The Hudson estimator
FST from sample data needs an estimator. Bhatia et al. 2013 recommended Hudson's estimator, computed as a ratio of averages over SNPs, because it does not depend on sample sizes and it behaves well with rare variants. EIGENSOFT's smartpca uses this estimator.
Ancient genomes are usually pseudohaploid: one allele is read at each site. ADMIXTOOLS 2 can adjust its statistics for pseudohaploid genomes, so a population of them can be compared with a modern, diploid one.
FST and f2
f2 is the average squared difference in allele frequency between two populations. FST divides a similar quantity by the total variation. Both grow with drift, and both rank pairs of populations similarly. f2 adds up along a tree, which makes it the building block of qpAdm and qpGraph; FST does not add up, but it is easier to compare across studies.
What FST cannot tell you
- Direction or timing. FST is the same for A and B as for B and A, and the same value can come from an old split with little drift or a recent split with much drift.
- Admixture. A population that is a mixture of A and B has an intermediate FST to both. Use f3 or qpAdm to test for mixture.
Running FST
EIGENSOFT reports FST with smartpca's fstonly option, and ADMIXTOOLS 2 with fst. Nexju runs pairwise FST between AADR populations on the Pro plan, as a table of every pair you choose.
References
- Bhatia G. et al. 2013. Estimating and interpreting FST: the impact of rare variants. Genome Research 23. doi:10.1101/gr.154831.113
- Patterson N. et al. 2012. Ancient admixture in human history. Genetics 192. doi:10.1534/genetics.112.145037