Were bivalent boosters worth it?

The biggest questions about the newest Covid-19 vaccines are still unanswered.

In early September, the Centers for Disease Control and Prevention (CDC) recommended updated Covid-19 boosters for all Americans 12 and over.

The “update” was a broadening of the vaccines’ target: In addition to aiming for the initial strain of the virus, the new boosters would also take aim at the spike protein present on newer omicron variants of SARS-CoV-2, BA.4 and BA.5.

The hope was that the new boosters (called “bivalent,” for the two strains) would provide additional protection, priming the immune system to recognize a much-mutated virus that had grown better at evading the immune system and reinfecting people.

Back in the fall, that hope was based mostly on modest data showing that the updated boosters raised antibody levels in people who received them. But a big question remained unanswered: In the real world, would the updated boosters actually prevent more cases from progressing to severe disease than the original, monovalent boosters would?

Researchers weren’t sure, and were divided on whether it was worth the $5 billion price tag to update the boosters without better data.

Now we have real-world, human data on the bivalent boosters, along with more carefully controlled lab studies. They show that over the last few months, older adults who got the bivalent booster last fall were less likely to have a Covid-related hospitalization or death in the months since than those who didn’t.

But the studies still tell an incomplete story because of how their data was gathered. Scientists don’t yet have totally clear answers to all questions about the booster.

Here’s what the latest available data on the vaccine booster shows and what it doesn’t — and how future studies should change to more precisely chart a path forward.

The good news: The bivalent boosters have helped keep some people over age 65 alive and out of the hospital

The data that’s coming out now is promising, but it comes with big caveats.

Let’s start with these new studies’ outcomes:

A CDC study published in late December showed that between September 13 and November 18, there were 31 percent fewer emergency room or urgent care visits for Covid-19 among adults 18 and over who’d received a booster shot than those who’d received a monovalent vaccine series (with or without one to two additional monovalent booster doses) two to four months before the same time point.

(To note: Those who didn’t get a bivalent booster weren’t totally unprotected. But the longer they’d gone since their last dose of the original vaccine, the more their protection waned — and the more their risk grew, especially if they were over 65.)

A different recent CDC publication also showed the booster’s protective effect was especially strong in older adults. Americans 65 and over who had received a booster dose had 73 percent fewer Covid-19 hospitalizations in the fall than those who’d received at least two doses of the monovalent vaccine.

Another recently posted preprint study of nearly 700,000 Israeli adults 65 and over showed even stronger evidence of protection for older adults. In that study, between September 24 and December 12, the bivalent booster reduced hospitalizations by 81 percent and deaths by 86 percent in people who received it compared to those who’d gotten at least two doses of monovalent vaccine.

All of this suggests that the people who got the bivalent vaccine last fall did better than those who didn’t — especially if they were 65 or older.

Now, the big caveat that makes these studies hard to interpret:

All of this data comes from retrospective observational studies — meaning, studies that observe people outside of a controlled lab setting well after the intervention (in this case, the bivalent vaccine) has occurred. People receiving or not receiving the bivalent vaccine didn’t do so randomly — they self-selected to be in either group.

“The people who choose to get it are very different than the people who choose not to get it,” Shira Doron, an infectious diseases doctor at Tufts, said of the vaccine: They’re likelier to also do other protective things, like wear masks in crowded places and restrict travel or socializing when transmission is high.

In contrast, a randomized clinical trial would have arbitrarily assigned people to either get the bivalent vaccine or not, regardless of their preference. Because it more strongly controls for the lifestyle choices participants make, it’s the way Covid-19 vaccines were studied prior to their original rollout. It’s also the method scientists use to evaluate most medical interventions.

The lack of randomization is the first thing that makes this data messy. It means that at least some of the difference in outcomes these studies find between bivalent recipients and non-recipients could be due to lots of factors other than the vaccine itself.

A second thing that makes the data messy: People who got the bivalent vaccine aren’t being compared to people who got the original vaccine at the same time.

When the bivalent booster was made available, the older version of the vaccine was made unavailable. That means that in all of these studies, people who got the bivalent could only be compared to people who’d received the monovalent months earlier.

It’s not exactly apples to apples — people who got the older vaccine already had less protection than those who got the newer one because more time had passed since their last dose. Meanwhile, the SARS-CoV-2 virus has evolved. So the studies aren’t even necessarily comparing people who had exposure to the same subvariant.

More ideal would be an evaluation comparing people who got the bivalent vaccine to people who got the monovalent vaccine at the same time, said Walter Orenstein, an infectious diseases doctor who serves as associate director of the Emory Vaccine Center.

All of these caveats mean there are important questions the data can’t answer. Questions like:

• Is the bivalent vaccine truly better than the original formulation at preventing severe disease in any group of people?
• How much of the protection of the bivalent vaccines is due to the vaccine itself, and how much is due to lifestyle choices?
• Do people below 65 benefit from getting regular booster shots (bivalent or not)? Or how frequently should boosters be given now, and to whom?
• Are repeated boosters a good idea in the adolescent boys in whom mRNA vaccines are associated with higher rates of myocarditis?

The ambiguity of these questions has led to some big debates on what the boosting strategy should be going forward — and what the strategy should be for evaluating those boosters’ effectiveness, both before and after they’re rolled out.

More carefully controlled lab studies don’t show a clear way forward, either

There is some lab data to suggest the updated boosters were worth it. But it’s also hard to interpret.

At least eight lab studies have tried to answer the question of whether these new boosters lead to higher levels of antibody against newer strains of Covid-19 — including the latest XBB.1.5 variant, which is an omicron subvariant. In a recent edition of his newsletter, Eric Topol, a cardiologist and director of the Scripps Research Translational Institute, wrote that these studies “converged on the bivalent’s superior neutralizing antibody response to BA.5 ... but also against XBB.”

Translation: The bivalent appears to be better at provoking antibodies for the newest variants of SARS-CoV-2.

That’s promising, but it only really matters if you believe antibodies are as important a part of Covid-19 immunity as the protection that comes from other, harder-to-measure parts of the immune system, like T-cells — which is still an open question.

There’s also some concern that giving people updated boosters too frequently reduces their impact. Scientists increasingly suspect that the first strain of a germ one’s body encounters (whether from infection or vaccination) provides the most durable immunity. New research suggests that when later variants of the same germ come along too quickly, the immune system does not mount as robust a response targeted against them. This phenomenon is called “imprinting,” and it may mean diminishing immunological returns for boosters that are updated too often.

Antibodies are the first line of protection from infection. So for people who prioritize preventing even mild Covid-19 infections, antibody levels matter a lot. But as the pandemic has evolved, several prominent experts have said stopping mild infections is no longer a priority to them. That makes any vaccine’s ability to elicit an antibody response less valuable than it used to be.

One advocate of this view is Paul Offit, a prominent vaccinologist and vaccination advocate at the Children’s Hospital of Pennsylvania. In a recent New England Journal of Medicine editorial, Offit wrote that “we should stop trying to prevent all symptomatic infections in healthy, young people by boosting them with vaccines containing mRNA from strains that might disappear a few months later.”

Doron, the Tufts infectious disease doctor, agrees. “I don’t care about the data that shows that it prevents infection,” she said. “I want to see that a booster or a vaccine will prevent severe disease.”

“The most important thing” is thinking through what studies we’ll need to answer that question, said Doron. “From a financial standpoint, you do have to prove that the upgraded version is better. And they definitely have not done that,” she said.

“Why are we skipping the clinical trials?” asked Daniel Salmon, who directs the Institute for Vaccine Safety at Johns Hopkins University’s Bloomberg School of Public Health. Although they’re costly, time-consuming, and don’t represent real-world conditions, achieving randomization would be worthwhile given the importance of vaccine confidence, he said. Given how much the US government has already spent on Covid-19 vaccine development, “it’s really hard for me to hear an economic argument” against investing in this higher quality of evidence, he said.

Creating the kind of high-quality data that would more clearly prove the effectiveness of a revised vaccine is expensive. The US has already spent more than $30 billion to develop and roll out Covid-19 vaccines — and it’s unlikely future iterations of these vaccines will get the same level of funding.

The bottom line is that “following the science” doesn’t always tell us what to do when the science is messy. Even simple questions like “were the booster updates worth it” can evade easy answers. We’re going to have to find a way to create a higher level of evidence to support future vaccine revisions — or get more comfortable wading through the muck.