I tried to shake it. But I realized I was too ticked off about this opinion piece, and the tweeting I was seeing about it, to be able to concentrate on anything else. So here's my rebuttal to John Ioannidis' opinion piece about the coronavirus pandemic in Stat News, "A fiasco in the making?" By my analysis, there are 5 main arguments. I'll address each of them, and then where I think it leaves us.
Argument 1:
Better information is needed to guide decisions and actions of monumental significance and to monitor their impact.
This is a side of evidence-based medicine that is aggravating, to say the least: as though we have the luxury of not acting until there’s better evidence. Doing nothing for which there is no strong evidence is doing something: it’s withholding public health interventions that, on the balance of what we know, could save a lot of lives and trauma – including the lives of a lot of healthcare workers.
Secondly, the need for societies to be able to monitor the impact is an argument for putting more effort into monitoring. Weaknesses in that is not a reason to not act. The weaknesses are not the same everywhere, and may not be as weak 2 weeks from now.
Argument 2:
[S]hort-term extreme social distancing and lockdowns may be bearable...How long, though, should measures like these be continued if the pandemic churns across the globe unabated? How can policymakers tell if they are doing more good than harm?
It is not certain when extreme measures would ideally end, just as there is no universal certainty about exactly what should trigger them. (See for example an Australian evidence summary from last year.) That doesn’t mean that we can’t already begin to gather evidence that will help to narrow that degree of uncertainty. For example, when I complained that the Cochrane Collaboration was promoting a systematic review that was clearly wildly out of date, they replied they are undertaking a rapid update of it. Another example is one part of the evidence-based community coming together to mobilize rapid evidence summaries around the Oxford Centre for Evidence-Based Medicine.
That’s just a couple of examples within my circle of close contacts: there will be many amazingly impressive mobilizations. The same will be happening about gathering adequate data to inform the next decision, and the decision after that. The choice is not between having excellent evidence and acting, or having no excellent evidence and just sitting on our thumbs. The argument being made here isn’t evidence-based: it’s an ideological position, that sounds to me similar to the arguments people make for ignoring official pleas for them to evacuate in the face of incoming hurricanes or what we call bushfires in Australia (aka wild fires elsewhere).
Argument 3:
The data collected so far on how many people are infected and how the epidemic is evolving are utterly unreliable.
That’s simply not true. We don’t know how many people are infected, but the data is not “utterly unreliable” everywhere: the uncertainty surrounding estimates varies greatly. We do not know “the” case fatality rate, but that won’t be the same everywhere, dependent as it is on regional differences like health system capacity and levels of antibiotic resistance for secondary pneumonia. And while it means best and worst case scenarios are far apart, that does not of itself give best case scenarios greater weight.
I agree with Ioannidis that the uncertainties in the US are particularly high: but fortunately for all of us, including people living in the US, that’s not a universal situation. And deaths from causes such as pneumonia could be monitored: even if they don’t know how many of those people had Covid-19, increases and decreases may be informative.
This argument included this statement – emphasis mine: "The case fatality rate there [a particular cruise ship] was 1.0%, but this was a largely elderly population, in which the death rate from Covid-19 is much higher". Later he writes, in the worst case scenario he outlines, “The vast majority of this hecatomb would be people with limited life expectancies. That’s in contrast to 1918, when many young people died.”
Firstly, he provides no source for this data claim. In any event, a high death rate in elderly populations matters a lot. People in their 70s and older could be losing a lot of years life they would still like to live, their families and others would like to live with them, and from whose lives – and often ongoing work or social contributions – wider communities benefit. They are not the only ones who are at high risk though, even though they may not have been on that cruise. Yes, it would be far worse if more major age segments of the population were at risk, but in some countries at least, intensive care units are being crushed by the sheer absolute numbers of younger people falling seriously ill as well, even if they do have a lower prevalence of illness and risk of death.
In Italy, this week, it was reported that only 44% of 1,545 Covid-19 patients in intensive care were over 70, and in some parts of the country intensive care units are close to saturation. Alarming levels of infection in healthcare workers are being reported, too. This doesn’t mean that other countries face the same outcomes: Italy has relatively high rates of antibiotic resistance, for example, which could play a role with secondary infections. And the people who die because a health system is overloaded will include the young (as Ioannidis points out too later). More on this to follow.
Argument 4:
If we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population — a mid-range guess from my Diamond Princess analysis — and that 1% of the U.S. population gets infected (about 3.3 million people), this would translate to about 10,000 deaths… If we had not known about a new virus out there… [a]t most, we might have casually noted that flu this season seems to be a bit worse than average. The media coverage would have been less than for an NBA game between the two most indifferent teams.
I'll get into the numbers later, but this seems to me to be the second key element of Ioannidis’ argument: that public health experts globally are in the grip of a kind of hysteria, in effect, while he’s keeping a cool head. For him, we’re flying blind, because we don't “..know the current prevalence of the infection in a random sample of a population and to repeat this exercise at regular time intervals to estimate the incidence of new infections.”
But that’s not a luxury afforded to people who specialize in this area. They nevertheless have to become expert at not shouting fire in crowded cinemas until they are sure, and they are not in the habit of freaking us out over nothing. Those experts told people globally not to over-react about Ebola outbreaks, for example: they are not given to panic attacks. They kept their heads in the face of infectious hemorrhagic disease. Just because they deal with data of greater uncertainties than Ioannidis, and no doubt they, would prefer, doesn’t mean they lack the expertise to make calls here.
The situation is describing one of those epidemiological arguments that assumes everything is evenly spread – you know, the basis of those jokes of epidemiologists drowning walking across rivers an average of 3 feet deep. If the situation he is describing was the worst case scenario, though, he would have a good point. It is, however, his best case scenario.
[Update 19 March]: The best case scenario Ioannidis outlines involves what Lipsitch called on Twitter, "wishful fantasy": seasonal flu, he said, doesn't lead to overloaded intensive care units, and pathogens that are growing exponentially do not infect 1% of the population and then "dissipate". The worst case scenario? Neil Ferguson and colleagues from Imperial College estimated that without any control measures, 81% of the populations of the UK and USA could be infected, with over 4% requiring hospitalization and 1% of them dying: "[W]e would predict approximately 510,000 deaths in GB and 2.2 million in the US, not accounting for the potential negative effects of health systems being overwhelmed on mortality". The societal consequences of collapsing health systems and that many deaths in a short time are difficult to contemplate.
I posted an analysis of the fatality rate data in a follow-up "Fiasco" rebuttal postscript on 21 March.
Argument 5:
In the absence of data, prepare-for-the-worst reasoning leads to extreme measures of social distancing and lockdowns. Unfortunately, we do not know if these measures work… Flattening the curve to avoid overwhelming the health system is conceptually sound — in theory… If the level of the epidemic does overwhelm the health system and extreme measures have only modest effectiveness, then flattening the curve may make things worse: Instead of being overwhelmed during a short, acute phase, the health system will remain overwhelmed for a more protracted period.
In this argument, Ioannidis appears to be indicating he has not studied this question deeply. The evidence he links to, to support his claim that we do not know if these measures work, is the Cochrane review I referred to above as being so out-of-date. The date of its search strategy is over 10 years ago. Let that sink in - his argument is based on a lack of knowledge 10 years ago.
A while ago – feels like months, but it was only 5 March – I realized I had a blog post relying on that review. You can see the update I did there. I found on the questions I was looking at, there were far more trials since that review than there were in it, and the conclusions in the light of more recent systematic reviews were unreliable and outdated. Hence my challenge to Cochrane about its status. It took me less than an hour to determine it was an unreliable evidence source! Yet Ioannidis is prepared to throw his weight into making a high-profile call for inaction based on it. (That’s not to say everything in that review is similarly outdated – the situation with global travel restrictions for example, although I haven’t looked at that depth.)
Take for example the country I live in. Country-specific estimates from last year suggest that flattening the curve may only buy a short amount of time. But if that time is used gainfully, the healthcare system could be made more robust to the effects of a high rate of sick people (like delaying non-urgent procedures, and strengthening intensive care capacity). I haven't dug into that. But my conclusion: the evidence assessments and advice to government are coming from people vastly better informed on the specific situation we face than Ioannidis. He seems to be implying that unlike all the other times we’ve faced new infectious diseases spreading in recent years, this time, the leading public health experts globally, have succumbed to some kind of mass delusion, that could, with long-term draconian measures in place, he writes, lead us to this situation: “billions, not just millions, of lives may be eventually at stake”. (He provides no data for that claim.)
Could there be fiascos from over-reaction? Yes, there could, but several countries have introduced measures that are draconian, and they appear to have pegged outbreaks back. Could there be fiascos from under-reaction? Well, we already have some of those.
There isn’t unanimity on every possible course of action, in every region, that’s for sure. But there is broad consensus that this is a public health emergency, and we have to take action, not just sit there studying the situation and waiting for better information before acting. I think the stakes are too high to ignore the public health community urging us to act in favor of a “hot take” from someone who doesn’t seem to have done his homework. And if there are fiascos from that, I am sure they will advise a course correction, and the situation won't drag on like that for years.
[Update 19 March] Stat News has now published a response to the Ioannidis piece by infectious diseases epidemiologist and microbiologist, Marc Lipsitch, "We know enough now to act decisively".
Postscript: I published a follow-up to this post, on the case-fatality rate estimation, on 21 March.
Hilda Bastian
Originally posted on 18 March 2020
(Last updated, 21 March)
Disclosure: I am 59 years old. A close family member who is one of the people I care about most in the world is young and immuno-suppressed, and 2 of the others in the same category are high risk for other reasons. I wrote a blog post this week on the first-in-human trial of a Covid-19 vaccine in Seattle and associated processes and history.
Update: About an hour or so after I posted this, I added the sentence pointing out that there was no source provided for the cruise ship data claim. I hadn't fact-checked that - I'm indebted to Morten Grøftehauge's question to me on Twitter about that.
Updates on 19 March: Several are noted in the text. In addition, I changed the sentence in the last paragraph in response to several questions on Twitter about the risk of subsequent resurgence of the virus. It originally read: "Yes, there could, but several countries have introduced measures that are draconian, appear to have pegged outbreaks back, and are loosening the measures".
Updates on 21 March: Added links to the postscript. Corrected misspelling of Neil Ferguson's name - thanks to Robert McMullen for alerting me to the error.