No tracking. No cookies
Subscriptions
Subscriptions
In the first part of his investigation, Nafeez Ahmed looks at the serious flaws of scientific fatalism and inaccurate modelling.
On Thursday 12 March, the day after UK Health Minister Nadine Dorries tested positive for the Coronavirus, I accused the British Government of making flawed decisions which had endangered the public by accelerating the spread of infection, making the deaths of many vulnerable elderly and ill people inevitable.
At that time, the number of confirmed Coronavirus cases was 596. By 22 March, this had shot up to 5,682 cases and continues to rise – with new confirmed cases now roughly doubling every two to three days.
My back-of-the-envelope projections suggested that the Government’s refusal to attempt to slow and curtail the spread of the virus could lead to as many as 458,752 deaths – possibly as high as 1.6 million in an absolute worst-case ‘do nothing’ scenario.
From Complacency to Shut-Down
At that time, the Government’s abject failure to do anything beyond suggesting that British citizens should wash their hands and sing happy birthday twice while doing so was allowing the virus to spread exponentially.
But, on Monday 16 March, a scientific model published by Imperial College’s COVID-19 Response Team shocked the Government into rapidly changing course.
The study confirmed what many scientists around the world had already been saying, but this time with an epidemiological model. The team had found that the Government’s previous course of action was on course to massively exceed “the surge limits for both general ward and ICU [intensive care unit] beds… [by] at least eight-fold under the more optimistic scenario for critical care requirements”.
Even in a best-case scenario of all patients receiving treatment, the model projected some 250,000 deaths in the UK and some 1.1–1.2 million deaths in the US.
Whereas the Government’s approach up until then had recommended mild measures encouraging the self-isolation of symptomatic individuals, this new research prompted a rapid U-turn – in both the administrations of Boris Johnson and Donald Trump (the paper was also sent to the White House where it triggered the belated emergency response from the Federal Government). The U-turn has included more stringent social distancing strategies including bans on large gatherings, the quarantining of the elderly and, eventually, a decision to close schools and universities.
However, the Imperial College study did not stop there. Its “epidemic suppression” strategy – designed to “reverse” the growth of the outbreak, rather than simply slow it down – recommended that social distancing policies applied across the entire population would now need to be in place for potentially as long as 18 months or more – until a vaccine becomes available.
This offered a much-needed reality check on the disastrous strategy previously adopted by the UK Government. But many questions remained.
How could the scientific teams advising the Government have failed so dramatically to understand the scale of the crisis that was coming until only “a few days ago”, to quote the paper? Why did the Government previously adopt a strategy which would continue to allow the Coronavirus to spread through the population in a way that would inevitably cause a certain level of fatalities?
Why did the Imperial College team underestimate the escalating burden on the NHS despite a flurry of published data from Italy prompting scientists around the world to warn that the UK Government’s inaction was setting the country’s healthcare services up for catastrophe (“previous planning estimates assumed half the demand now estimated,” according to the paper)? Why did the modellers predict that virus “transmission will quickly rebound” if restrictions are relaxed?
The answer is simple: they were making the wrong assumptions.
Unscientific Fatalism
Any model is only as good as the assumptions that go into it and the empirical data it crunches. To get models to work, the assumptions need to be founded as much as possible on what is known. This also means ensuring that the data used to formulate assumptions is based on an appreciation of how multiple complex systems actually interact.
A scientific analysis by the New England Complex Systems Institute at New York University has found the Imperial College model to be deeply flawed. Although the authors praise the Imperial study for recognising the disastrous consequences of the Government’s previous “mitigation” approach, they identify a number of inexplicable failings.
One of the most egregious is the stubborn belief that the outbreak of the virus cannot be stopped. This belief can be traced partly to how the Government interpreted a range of modelling results produced by British scientists advising it, as part of its Scientific Advisory Group for Emergencies (SAGE).
Overlooking the Power of Tests and Tracing
Several models produced by SAGE scientists indicate that using contact tracing to contain the outbreak would be extremely difficult without sufficient speed, but they still emphasised that identifying people infected and isolating them systematically on a mass scale would at least help to reduce or control the epidemic.
Independent scientists who have reviewed the SAGE corpus say that it reveals a fatally incompetent scientific process. According to Professor Devi Sridhar, chair of global public health at the University of Edinburgh’s Medical School, the SAGE corpus inadvertently reveals why the UK Government “got it wrong… SAGE analysis was overcomplicated, too academic, relied on incomplete data, overlooked testing and health service capacity. Why didn’t we go fast down the path of test, isolate, trace while delaying spread for health services to prepare?”
According to the New York University complex systems scientists – Dr Chen Shen, Professor Nassim Nicholas Taleb and Dr Yaneer Bar Yam – early assumptions that the outbreak might inevitably grow beyond control in a way that would mute the point of mass testing and tracing, was completely unfounded. Dr Chen and his co-authors point out that the Imperial College scientists, along with their other colleagues and SAGE, have consistently overlooked data from China and elsewhere.
“They ignore standard contact tracing allowing isolation of infected prior to symptoms,” they state. “They also ignore door-to-door monitoring to identify cases with symptoms… They ignore the possibility of superspreader events in gatherings by not including the fat tail distribution of contagion in their model. This leads them to deny the importance of banning them, which has been shown to be incorrect, including in South Korea.”
The ‘fat tail’ refers to the existence of small probabilities leading to a much higher probability of extreme outcomes.
South Korea’s pursuit of the method of mass contact tracing has allowed the country to demonstrably contain and reduce the rate of infections without resorting to more draconian lockdowns.
Assuming an Inevitable Disease Rebound
The New York University paper also dismisses the Imperial College study’s insistence that a resurgence of the Coronavirus would be inevitable as soon as interventions are relaxed.
The authors state that the Imperial College team completely ignored the potential impact of widely available testing on empowering authorities to monitor outbreaks and keep the disease suppressed: “Their conclusions that there will be resurgent outbreaks are wrong. After a few weeks of lockdown, almost all infectious people are identified and their contacts are isolated prior to symptoms and cannot infect others. The outbreak can be stopped completely with no resurgence as in China, where new cases were down to one yesterday, after excluding imported international travellers that are quarantined.”
The assumption that the Coronavirus will definitively rise back in an even worse way in winter is also highly speculative and scientists admit that, in reality, right now they just don’t know.
The east Asian experience, the New York University paper points out, demonstrates that rapidly containing and stopping the spread of infection is entirely possible – within a period of six to eight weeks – through national scale public, private and community coordination. After this, selective and careful lifting of restrictions along with extensive monitoring and tracing may enable societies to slowly begin functioning again.
“Since lockdowns result in exponentially decreasing numbers of cases, a comparatively short amount of time can be sufficient to achieve pathogen extinction, after which relaxing restrictions can be done without resurgence,” the New York University analysts write. “If actions had been taken earlier, successful local lockdowns, as performed in China in Hubei province, would have been possible instead of national lockdowns.”
Inaccurate Modelling
Overall, the New York University study is deeply critical of the modelling approach used in the Imperial College study, for being too narrow and therefore “not well suited for incorporating real world conditions at fine or large scale”.
Important variables accounting for local conditions, travel restrictions, the impact of ‘superspreader’ events, and the result of changing social responses mean that the Imperial College model “is several degrees of abstraction away from what is warranted by the situation”.
I contacted Professor Neil Ferguson, the lead author of the Imperial College paper and a member of SAGE, about the problems identified with his paper and its underlying assumptions. He did not respond.
He did, however, surface on Twitter on 22 March, where he conceded that his model assumptions were written more than 13 years ago and based on the specific dynamics of a flu pandemic. In other words: the model was not calibrated for COVID-19 and failed to sufficiently incorporate new relevant data. According to Professor Sridhar, this is clear evidence of how the Government “messed up”.
Bill Gates, who predicted a Coronavirus-like pandemic in 2015 and funds the Institute for Disease Modelling, agrees that the model’s implication of the need for a 12-18 month minimum shut-down has little grounding in empirical reality.
“Fortunately it appears the parameters used in that model were too negative,” he said. “The experience in China is the most critical data we have. They did their ‘shut down’ and were able to reduce the number of cases. They are testing widely so they see rebounds immediately and so far there have not been a lot. They avoided widespread infection. The Imperial model does not match this experience. Models are only as good as the assumptions put into them. People are working on models that match what we are seeing more closely, and they will become a key tool.”