A recent article in Quartz with the headline “
in genetics research, and biomedical research in general.
It’s also an opportunity to address mischaracterizations of our science within the article, and motivations ascribed to our research that are just plain wrong. We’ve raised these issues with Quartz and feel strongly enough to share our thoughts more publicly to hopefully steer the discussion in an informative and worthwhile direction, since it is a conversation that should be had by the broader scientific community.
Genetics research has a diversity problem. This is widely recognized, but when you see the data it’s staggering. For example, a 2011 article cited data showing that only 4 percent (!) of all genome-wide association studies had been conducted using samples of non-European descent.
The Implications
There are very real repercussions when research is limited to just one group. The seriousness of this situation was demonstrated in a recent study showing that due to biased research, African Americans are more likely than whites to mistakenly be told that they carry a mutation putting them at risk for the heart condition known as hypertrophic cardiomyopathy, when in fact they do not.
This type of mistake has serious emotional and medical implications.
The lack of diversity in genetics research is reflective of the general lack of diversity in all aspects of health research. There are lots of reasons – historical, cultural, political. For good overviews we suggest reading this piece from Newsweek and this article from Oh, et al. These authors eloquently and forcefully explain the situation.
What does this have to do with 23andMe?
23andMe’s Reference Data
Though we do conduct our own research and are increasingly able to provide reports based on our own published studies, for the most part 23andMe is dependent on peer-reviewed scientific research published by others to fuel the reports in our product. Lack of diversity in what is studied by the research community at large therefore impacts the results we are able to provide to customers.
When it comes to our Ancestry Composition feature (which tells people the proportion of their DNA that comes from each of 31 populations), the results we provide to customers are derived by comparing their DNA to what are known as “reference datasets”.
We use publicly available data from academic initiatives that have specifically sought to characterize large swaths of the human genetic diversity of our planet: the Human Genome Diversity Project (HGDP), HapMap3 and the 1000 Genomes Project.
We augment this public data with information provided by a small percentage of our customers whose data meet our rigorous standards for being included in a reference dataset.
Is Bigger Better?
The author of the Quartz article focused in on the numbers of people in our most granular population references. She specifically questioned how we can claim to provide accurate results about her ancestry (she identifies as Korean, but her 23andMe results indicated Japanese and Chinese ancestry, as well as Korean) when there are data for “only” 76 Koreans in our reference data set. It’s an understandable worry – so much of what we hear about these days in science is the power of “big data”.
Bigger must be better, right? Not necessarily.
For Ancestry Composition, high accuracy can often be obtained with relatively modest sample sizes. Size matters more when it comes to granularity, not accuracy.
We’ve done a comprehensive evaluation of our algorithm and reference datasets, and are confident in the accuracy of the results we report for customers with Korean or any other ancestry.
If you want to get into the nitty gritty you can check out this white paper (in which we describe the multiple steps that go into producing accurate results and the metrics demonstrating how well our algorithm works), this blog post or this scientific poster.
Granularity
The gist is that we only go as granular as we can for a given confidence threshold. Customers can adjust their results from a 90 percent confidence level (“Conservative”) down to 50 percent (“Speculative”). If we say that a section of DNA is most likely Japanese or Chinese or Korean at one of our higher confidence thresholds, we stand behind it. If we weren’t sure that we could make the distinction, we’d go up a level and say that that DNA reflected “East Asian” ancestry.
The information would not be incorrect, simply less specific.
While the impact of adding more Korean individuals (or individuals of any of the groups we currently have represented in our reference datasets) is likely to be small in terms of improving the accuracy of our Ancestry Composition results, it could help us add more granularity to our results – i.e., groups of people from more specifically defined geographical areas. That way, for example, we might be able to distinguish between Cameroonian and Ghanaian ancestry, instead of lumping ancestry from these different locations under the label “West African.”
The author of the Quartz article correctly points out that we do have a lot of data and granularity for Europe (providing, for example, separate French & German and British & Irish groupings), but not for the rest of the world. She implies that this is driven by a lack of effort or concern on our part. The truth is that we have a great deal of customer data to enhance the public data available for Europeans because the vast majority of our customers have European ancestry, and several thousand of them are eligible to contribute to our reference dataset. This is not because of a “commercially driven ethnic bias,” and we have for several years been making in-roads to improve the diversity in our research and reference data.
This is where we come back to where we began – this is not just a 23andMe issue but an issue with diversity in genetics research in general.
23andMe’s Diversity Efforts
There’s only so much we can do as a single company, but we are trying. We funded the genotyping of the HGDP samples, data which are publicly available. Our Roots into the Future project enrolled 10,000 African Americans in our research program, and our African Ancestry Project helped contribute to our West African reference dataset. We recently collaborated to obtain data from for over 400 representatives of populations in the Democratic Republic of Congo. One of our researchers was awarded NIH funding to develop new methods for uncovering genetic variants associated with disease, especially in those with non-European ancestry. There are additional programs in the works, and we look forward to sharing more as we can.
We’re also always looking for more publicly available data that we can use to improve our product and inform our research. But it’s not as simple as just pulling studies for a given group or purchasing reference data. There are important scientific and ethical considerations that must be attended to when collecting population data and 23andMe is not willing to compromise our standards in the interest of appearing more diverse.
We can always do better. We want people – researchers, customers, journalists – to keep holding us accountable and pushing us to lead the way in consumer genetics and genetics research. But accusing us of racial bias and shoddy science is unfair. It reduces an important conversation to clickbait.
Please share your comments here but don’t limit it to our blog – discuss your ideas, raise awareness and spread the word that the lack of research diversity is a real problem. We want people to have informed opinions and conversations on this topic since the impact of genetics research will only continue to grow.