How to merge 23andMe or AncestryDNA raw data with the AADR

To use a consumer raw DNA file with ancient genomes, convert it to PLINK or EIGENSTRAT format, keep the SNPs it shares with the AADR, align its alleles to the AADR's, and merge. The steps below use PLINK 1.9 and ADMIXTOOLS' convertf. Nexju does the same steps for you when you upload a file.

• your raw data file, as downloaded from 23andMe, AncestryDNA, MyHeritage, FTDNA or LivingDNA;

• the AADR release in EIGENSTRAT or PACKEDANCESTRYMAP format: .geno, .snp and .ind, 1240K or Human Origins;

• PLINK 1.9 and convertf from ADMIXTOOLS (or EIGENSOFT).

Consumer files and the AADR both use the GRCh37 (hg19) reference, so positions agree. Most consumer SNPs carry rsIDs, as do most AADR SNPs.

PLINK reads the 23andMe format directly. AncestryDNA puts the two alleles in separate columns, so join them into one column first.

plink --23file genome.txt Me Me --snps-only just-acgt --make-bed --out me

--snps-only just-acgt drops insertions, deletions and no-calls such as --.

Write a convertf parameter file:

genotypename:  aadr_1240k.geno
snpname:       aadr_1240k.snp
indivname:     aadr_1240k.ind
outputformat:  PACKEDPED
genotypeoutname:  aadr.bed
snpoutname:       aadr.bim
indivoutname:     aadr.fam

Then run convertf -p par.convert. To save time and disk space, first keep only the populations you need with a poplistname: line.

awk '{print $2}' aadr.bim > aadr.snps
plink --bfile me --extract aadr.snps --make-bed --out me.shared

A consumer file of 600,000 to 700,000 SNPs typically keeps a few hundred thousand on the 1240K set. The exact number depends on the chip version.

plink --bfile aadr --bmerge me.shared --make-bed --out merged

If PLINK reports SNPs with more than two alleles, it stops and writes them to merged-merge.missnp. They are usually strand mismatches: the chip reported the other DNA strand. Flip them and merge again:

plink --bfile me.shared --flip merged-merge.missnp --make-bed --out me.flipped
plink --bfile aadr --bmerge me.flipped --make-bed --out merged

SNPs that still conflict are errors; remove them from me.flipped with --exclude and the new merged-merge.missnp, then merge once more. A/T and C/G SNPs look the same on both strands, so a strand error there cannot be detected. Many analysts drop them to be safe.

Convert merged back to EIGENSTRAT with convertf (outputformat: EIGENSTRAT), and give your genome its own group label in the .ind file. It now works as a target in qpAdm, f-statistics and PCA.

Your genome is diploid while most ancient genomes are pseudohaploid. qpAdm and f-statistics handle the mix. For PCA, compute the axes from present-day genomes and project the ancient ones, with smartpca's lsqproject: YES, which handles their missing data.

• Few shared SNPs. Check the chip version, and that the file is the original raw genotypes, not an imputed or reprocessed file.

• Many mismatched SNPs. The file may be on a different genome build. Positions must be on GRCh37.

• Duplicate SNP IDs. Some files list one rsID twice; remove the duplicates before merging.

• Wrong sample label. A genome whose .ind label matches an existing AADR group joins that group. Give yours a unique label.

When you upload a raw file to Nexju, its servers do these steps for you:

1. Each call in your file is matched by SNP identifier to the SNPs of the reference data.

2. The genotype is encoded against the reference alleles. If the alleles do not match, the complement, the other strand, is tried. A call that matches neither is set to missing.

3. The result is written in the same format as the reference data, ready for every analysis.

Nexju accepts 23andMe, AncestryDNA, MyHeritage, FTDNA, LivingDNA and tellmeGen files as .txt, .csv, .zip or .gz. On the Max plan you can also upload a whole-genome BAM or CRAM file aligned to GRCh37 by a public link. Your file is used only for the analyses you request, and it is visible only to you.

• Chang C. C. et al. 2015. Second-generation PLINK: rising to the challenge of larger and richer datasets. GigaScience 4. doi:10.1186/s13742-015-0047-8

• Patterson N. et al. 2012. Ancient admixture in human history. Genetics 192. doi:10.1534/genetics.112.145037

• Mallick S. et al. 2024. The Allen Ancient DNA Resource (AADR): a curated compendium of ancient human genomes. Scientific Data 11. doi:10.1038/s41597-024-03031-7