After pressure from scientists and public health experts, vaccine makers are releasing the detailed protocols for their Phase III vaccine trials. Moderna went first yes­ter­day morning and Pfizer followed in the afternoon. The others will come around.

These documents let outside experts — and the rest of us — see exactly how and when the vaccine makers will decide that their offerings are effective, and by what criteria they will decide to apply to the FDA for an emergency use authorization or for approval and lic­en­sing.

Here are the protocol documents that have been released so far: Moderna (135 pp. PDF), Pfizer (137 pp. PDF). One news source I read claimed that AstraZeneca had released their protocol, but I was unable to find any trace of it. Johnson & Johnson, whose Phase III trial is slated to start on September 21, has not released theirs. But Reuters has seen the latter two documents and provided some details, which I discuss below.

The New York Times posted an early report yesterday afternoon covering the Pfizer and Moderna protocols.

Pfizer / BioNTech, BNT162b2

This candidate is arguably in the lead. Pfizer’s CEO has said he expects to get an initial read­out on efficacy by late October. The company has expanded the initial 30,000-participant trial to 44,000 in order to include more minorities and a range of other volunteers, such as those with pre-existing conditions, the company says. As of last Wednesday Pfizer had signed up more than 30,000 participants, and I estimate that more than 10,000 of them have had their second injection (three weeks after the first).

The protocol document spells out when efficacy might be determined — Pfizer has set its end-point at 120 par­tici­pants who fall ill with Covid-19, with interim looks at the data at 92, 62, and 32 infected participants.

That 32 figure was what I had assumed in the October surprise post outlining the process a vaccine candidate nor­mally goes through. In that earlier post I was not totally clear on the math that would apply in the case of an early look at efficacy. The protocol document provides more clarity. If after 32 trial participants fall ill, only 6 of them had received the vaccine and 26 the placebo, Pfizer could determine that its candidate was 76.9% more effective than placebo in staving off Covid-19. That small sample of cases introduces inherent uncertainty, and after some more fancy math the company can conclude that, with only 2.5% probability of being wrong, their candidate vaccine is better than 50% effective. That is the bar the FDA has set to consider any application for authorization or approval.

Moderna, mRNA-1273

Moderna is in second place, time-wise. The company’s president has said he expects to get a grip on efficacy in November, but that October is a possibility. Their trial has signed up more than 25,000 volunteers, 28% of them from minority communities, and more than 10,000 have had their second dose (four weeks after the first).

Moderna has set tighter (more conservative) conditions than Pfizer on its determination of vaccine efficacy: its end-point number is 153 cases, with interims at 103 and 53. So it will probably take them a bit longer than Pfizer to get to the first point at which their Data and Safety Monitoring Board can have a look. The way I read the math, if 11 of the initial 53 participants diagnosed with Covid-19 had received the vaccine, and 42 the placebo, Moderna could declare 73.8% greater efficacy than placebo, and clear the FDA’s 50% efficacy bar with room to spare.

The company’s CEO told the New York Times in a phone interview, “If the interim readout is deemed by the inde­pendent safety committee as positive with 70 or 80 or 90% efficacy, we will indeed consider approval” (emphasis added). I believe he meant authorization, as in emergency use authorization. An application for full FDA approval is likely to wait for more safety data; I expect it could come in Q1 of next year.

AstraZeneca / Oxford, AZD1222

AstraZeneca’s Phase III trial remains paused in the US, but it has resumed in the UK, Brazil, South Africa, and India. So far, over 18,000 people worldwide have received the vaccine. Un­like the two candidates above, AZD1222 requires only a single dose.

As mentioned, AstraZeneca does not seem to have made its Phase III detailed protocol public yet. Reuters, whose reporters have seen the protocol, reports that the company’s first interim point comes after 40 cases of Covid-19 are recorded, a number in between those chosen by Pfizer and Moderna. To find efficacy at this first interim point, Astra­Zeneca would want to see that 8 of the first 40 participants who came down with Covid-19 had received the vaccine, and 32 the placebo. [See the update below dated 2020-09-21.] Reuters did not reveal other interim points for this study.

Johnson & Johnson / Janssen, AD26.COV2.S

As noted above, this Phase III trial is set to begin three days from now, and J&J wants to sign up 60,000 volunteers worldwide. Its detailed protocol has not been released, so again we rely on Reuters’ reporting for this data: J&J has set its first interim point at only 20 cases. Guess­ing at the math here, it seems that the company is looking for 4 of the initial 20 Covid-19 patients to have received the vaccine, and 16 the placebo. Again, Reuters did not report other interim points.

Can we trust it?

The most critical question in this whole accelerated process of vaccine development is going to be: can we trust the vaccine(s) that emerge(s)? Remember that if any of the above companies chooses to apply for an EUA at an interim point in their study, the scientific value of that study is at an end. With an EUA, the company could no longer recruit new subjects; they would have to offer the vaccine to all who had received the placebo; and they would be unable to refine the efficacy and safety data so far collected.

In vaccine development in times past, the developer has typically defined only a single interim point at which the study might terminate early (for efficacy or its opposite). Reuters quotes Eric Topol, director of the Scripps Re­search Translational Institute: “These interim analyses have a flashing sign of short cuts. You miss safety issues and you may very well exaggerate the benefits.” In this light Moderna’s two interim points, and Pfizer’s three, can inspire healthy skepticism — however justified they may seem in the rush of a pandemic.

One of the most hopeful indicators I have seen in this miasma of pandemic, science, politics, and uncertainty, came from Dr. Anthony Faucci. (As one of the TWiV regulars said when asked which of the developing vaccines he had most confidence in: “I’ll take whatever Tony Faucci is having.”) In this video clip of an interview yesterday on MSNBC, Dr. Faucci declared that if he has any suspicion that corners are being cut or that the process is less than fully trans­parent, he will say so to the American people and take the heat for doing it. “If in fact it looks like that vaccine is safe and effective, I can tell the American public that I will take the vaccine when it’s available to me, and I will recommend to my family that they take it.”

---

[ Note added 2020-09-21: ] AstraZeneca released its Phase III protocol (111 pp., PDF) on September 19. Contrary to the earlier report from Reuters, linked above, the protocol calls for an end-point of 150 cases of Covid-19 to determine efficacy, with a single interim point at “approximately 75” cases. At that point, the company will be looking for 17 of the cases to have taken the vaccine and 58 the placebo. The protocol does note: “A statistically significant finding at the interim analysis will not be considered a reason to stop the study, but instead will be interpreted as early assessment of efficacy.” This is the only such (encouraging) note I have seen in the released protocols.

---

[ Note added 2020-09-21: ] Derek Lowe has written about what the detailed protocols tell us. If you are interested in the subject you really should go read that now; Lowe makes clear the math behind analyzing the data for these Phase III studies. He gives no indication that any of the three companies is planning to cut short their studies at interim points, and seems to believe that all will run to completion — around 150 cases of Covid-19 for each study. I don’t see how that could happen in 2020 for any of the candidate vaccines. A commenter on Lowe’s post points out that Moderna has said it will release details of its interim analyses; Pfizer has said it will not unless it decides to stop the study for either overwhelming efficacy or its opposite.