The Reason A Covid Vaccine Is Unlikely To Be Approved Soon

Releasing a vaccine before full testing is complete is theoretically possible but a number of thresholds have to be reached beforehand. No vaccine currently in trial appears to be close to achieving any of those thresholds. JL

Matthew Herper reports in Stat:

“If you make a decision based on promising but not convincing data, and then you discontinue your randomization, you discontinue your evidence-generating process. You can never go back and generate your evidence. If you release an interim analysis that didnt cross the threshold, chaos can ensue. And I dont think people can use that information responsibly."

There is growing concern that the Food and Drug Administration, under political pressure, could approve a Covid-19 vaccine before it has robust safety and efficacy data.

The consequences of such a decision could be significant, particularly if the vaccine is ultimately shown to be less effective than early data suggest. But an approval before the completion of large, Phase 3 trials does not have to be problematic. Experts aren’t ruling out the possibility that a vaccine could be cleared this fall if it is very effective.

“There are mechanisms by which products that have a good amount of data can be made available in a controlled way,” said Natalie Dean, an assistant professor of biostatistics at the University of Florida who specializes in vaccine study design.

But Dean also sees risks. 

“If you make a decision based on promising but not convincing data, and then you discontinue your randomization,” she said, “you discontinue your evidence-generating process. You can never go backward. You can never go back and generate your evidence.”

Related: 

In the race for a Covid-19 vaccine, Pfizer turns to a scientist with a history of defying skeptics — and getting results

Here is how a fast approval might play out, if all goes well — and some warning signs that a science-based process is not being followed.

The FDA has laid out clear criteria for the full approval of a vaccine: It should reduce the rate of symptomatic Covid-19 disease by 50%. Equally important is that the data should suggest it’s highly unlikely that the vaccine could possibly be less than 30% effective. Any vaccine less  effective than that would be useless.

The agency also said that there should be safety data of a year or more for at least 3,000 patients. There’s no way to shorten that timeframe, and it is one of the reasons experts believe the FDA could grant a Covid-19 vaccine an emergency use authorization, rather than full approval. 

It’s also possible that the studies could be ended early based on an interim analysis of data. 

These early looks are handled by a data and safety monitoring board, or DSMB, that reviews the data from a study regularly to make sure that patients are not being endangered by side effects from the vaccine and that it’s still ethical to give a placebo. 

“At the end of the day, if things are done according to the tradition, the DSMB looks at the data intermittently and makes one of four determinations,” explained Anthony Fauci, the director of the National Institute for Allergy and Infectious Diseases, which is working with Moderna and AstraZeneca on vaccine trials.

A study can be stopped because a vaccine is clearly effective; it can be stopped because  clinicians are seeing troubling side effects. The DSMB can recommend changes to a study. Or, as usually happens, it can do nothing.

“It’s up to the DSMB, in their judgment, to balance the safety issue, the efficacy issue, and the duration of the trial issue,” Fauci said. “And that’s the reason why they’re an independent group. They are not the company because obviously the company is going to want to get their product approved as quickly as possible.”

But there are also pre-set rules as to when a trial can be stopped. “The earlier you stop it, the higher the bar,” Fauci said.

Technically, the final decision about what to do rests with the companies that are developing the vaccines, Fauci said. But, he added, “it would be unusual for the company to do something completely dissociated from or at odds with the DSMB.”

How could these interim analyses play out? Each of the Phase 3 trials is enrolling at least 30,000 volunteers. It’s expected that the trials could need about 150 cases of Covid-19 to tell whether or not the vaccine is preventing disease.

But the more effective a vaccine is, the more likely it is that the trial could be stopped at an interim analysis based on fewer cases. Because the studies are so big, this could happen very fast. Pfizer, which is conducting its study with partner BioNTech but not with NIAID, has said its first interim analysis will occur when 32 patients have developed Covid-19. That could happen as early as this month.

Is it likely the trial will be stopped that early? Probably only if the vaccine is extremely effective, with only a handful of patients in the vaccine group getting sick. 

Steven Nissen, a cardiologist at the Cleveland Clinic, thinks the trial shouldn’t be stopped so quickly, even if the DSMB has that option based on the efficacy data. He once ran a study of an obesity drug that was stopped — yielding unclear results — because the company funding the study improperly released an interim analysis.

“If I were designing this trial, and I knew you had to have full safety information, what is the value of the interim, exactly?” Nissen asked. “How do you help society with an interim if you can’t stop because you don’t know about safety?”

And if the DSMB thinks that it is ethical to keep patients on placebo, experts are firm that the results of these analyses should not be made public. 

“If you release an interim analysis that didn’t cross the threshold but the groups have started to diverge, chaos can [ensue],” said Dean. “People don’t know what to do with that information. And I don’t think people can use that information responsibly.” 

Side effects, she said, might be more apparent early on than many people expect, because with vaccines they usually occur within weeks of the administration of the shot. But she, too, hopes the DSMBs on these studies have the wherewithal not to rush.

“This is not a disease where you have to wait a decade to accrue the information you need,” Dean said. “You’d have to wait another month, or another week, and when you think about how long the rollout is going to take, that’s not going to be long.”

Luciana Borio, a former acting chief scientist at the FDA, said that in a public health emergency, “it may be reasonable” to deploy vaccines before all requirements for licensure have been met. She added:  “It would be completely unreasonable and wrong to prematurely deploy them before you have randomized controlled clinical trials and clear data that the vaccine is safe and effective.”

And, she said, the decisions must be based on science, and made by staff at the FDA, with input from its advisory committees. The design of the studies, she said, should be made public before results are released.

“I hope we see these decisions driven by the appropriate group of experts in a transparent fashion in our advisory committees, and not from the podium at the White House or in a phone call from the White House to the commissioner,” Borio said.