The Worrisome Limits of Online Genetic Test 'Results'

Too many false positives and false negatives to be reliable for medical research, if not for technology marketing. JL

Gina Kolata reports in the New York Times:

More Americans are embracing consumer genetic testing, but the Food and Drug Administration limits what major firms like 23andMe or Ancestry.com can say about a consumer’s health. So people are uploading the raw data for additional analysis. But these are not firms certified to provide medical diagnoses, nor are they sophisticated academic centers. The false-positive rate is 40%. (And experts) are increasingly worried about the flip side: false negatives that reassure consumers who should be worried.

Dr. Joshua Clayton, a 29-year-old radiology resident at Baylor University Medical Center in Dallas, wanted to learn about his ancestry. So he sent a sample of his saliva to 23andMe, the genetic testing company.

His report was pretty mundane — no new revelations. But then he sent the profile created by 23andMe to a separate company called Promethease, which promises to do a more in-depth analysis for genetic mutations that cause disease.

The news was not good. Dr. Clayton got back a report with a sinister red box at the top saying he had a mutation linked to Lynch syndrome, a frightening genetic disorder that leads to potentially deadly cancers at an early age.

In an interview, Dr. Clayton said he “knew, at least cursorily, that false positives were common with these tests.” But “that didn’t change the frightened feelings or concern, because I certainly couldn’t blow it off.”

After two weeks of panic and yet another genetic test at a company with expertise in medical diagnostics, he learned the red box result was wrong. He simply didn’t have the mutation.

More Americans are embracing consumer genetic testing, but the Food and Drug Administration limits what major firms like 23andMe or Ancestry.com can say about a consumer’s health.

So many people are taking an additional step: uploading all the raw data created by these companies to second companies for additional analysis. But these are not firms certified to provide medical diagnoses, nor are they sophisticated academic centers.

Companies like Promethease warn customers not to regard their results as medical diagnoses. Mostly, they are comparing a consumer’s raw data to gene variants reported to be linked to disease.

But it’s not at all clear that consumers understand how uncertain their results may be.

There are no systematic studies of how often the direct-to-consumer results and third-party analyses are wrong. In one small study, Ambry Genetics — a lab certified to do medical testing — looked at 49 samples sent in by physicians whose patients had been told that they had disease-causing mutations by third-party interpreters.

Ambry found that 40 percent were wrong. In addition, some genetic variations classified by second companies as threatening actually were benign.

This tiny sample doesn’t prove that the false-positive rate is 40 percent. But patients like Dr. Clayton are not uncommon, genetic counselors say. And they are increasingly worried about the flip side: false negatives that reassure consumers who actually should be worried.

Dr. Judy Garber, director of the Center for Cancer Genetics and Prevention at the Dana-Farber Cancer Institute in Boston, said one of her patients was told by a consumer testing company she had Li-Fraumeni syndrome, which greatly raises the risk of a number of rare cancers.

Further testing showed that diagnosis was wrong. “It makes you worry about the people who don’t come in,” said Dr. Garber.

“People think they are getting the same kind of genetic testing as they would get from a certified clinical laboratory,” said Stephany Tandy-Connor, a genetic counselor at Ambry. “Nothing could be further from the truth.”

Even some doctors are misled by the reports, she said.

Of course, companies like Ambry have an interest in making sure their business is not usurped by consumer testing firms. But it’s also true that the method used by consumer companies is very different from those used by certified clinical laboratories.

Consumer companies method look for changes in tiny segments of genes, rather than examining the entire gene and looking for alterations. That is cheap but not comprehensive.

And because they are not making medical diagnoses, these companies are not subject to the sort of quality controls as certified labs, which require extensive confirmation that results are free of errors.

Compounding the problem, the second companies rely on databases that may contain errors — so even if the genetic alteration found by the first company is correct, the analysis can be wrong. Ambry found such patients in its recent analysis.

Greg Lennon, a co-founder of Promethease, said that the company’s reports include a disclaimer saying the data are “not intended for medical or health purposes.” Customers are warned to seek out “an independent, clinically validated test” if they are told of a mutation and to seek out a genetic counselor.

When errors occur, they usually derive from mistakes in the raw DNA data, said Mike Cariaso, also a co-founder of Promethease.

23andMe offers limited medical testing — for example, looking for three of the most common mutations in breast cancer genes that occur mostly in Ashkenazi Jews. But there are thousands of other alterations in those genes that the company does not track.

The F.D.A. has given 23andMe approval to do similar testing for 10 disorders linked to genetic mutations, but not for the huge number of others, including Lynch syndrome, that companies like Promethease look for.

The variations in DNA in 23andMe’s raw data “are not for medical or diagnostic use,” said Shirley Wu, the company’s director of product science. “The data hasn’t undergone the same kind of quality control and validation as our variations in our health reports.”

Advertisement

The company warns customers of this before providing their raw data, Dr. Wu said.

For the consumer who expects validated diagnoses, it’s all a bit confusing.

Testing companies like 23andMe say they are not at fault, because they make it clear that their data are not meant to be used for medical diagnoses.

The companies that provide deeper analyses, like Promethease, say they take only raw genetic profiles created elsewhere and use publicly available data to scan for mutations that can increase disease risk. Promethease didn’t author any of it.

“So much of what I have seen in these D.T.C. tests are false positives and cannot be trusted,” said Dr. Robert Nussbaum, chief medical officer at Invitae, a medical testing lab.

Still, he added, false negatives are even more worrisome.

Dr. Clayton learned only after weeks of agonizing that his Lynch syndrome diagnosis wasn’t really a diagnosis at all. “My wife and I were talking about having children,” he recalled. “What do you do? Do you make that decision to pass this on to them?”

And what about disability insurance? If he got cancer, he might not be able to work. But could he even get insurance?

Dr. Clayton searched online for Lynch syndrome experts and found Dr. Theodora Ross, director of the cancer genetics program at the University of Texas Southwestern Medical Center in Dallas.

She had Dr. Clayton’s DNA tested at a lab certified to make the diagnosis. That was how he found out the report was wrong.

He was lucky, he said, that he had medical training and knew where to find help.

“I don’t think that applies to a lot of people,” he said.