No tracking. No cookies
Subscriptions
Subscriptions
Nafeez Ahmed reveals how concerns around privacy and trust in the UK Government’s centralised COVID-19 tracing system are matched by doubts about its effectiveness.
An artificial intelligence firm previously hired by Dominic Cummings to work on the Vote Leave campaign has been intimately involved in R&D for the UK’s National Health Service (NHS) contact tracing app. The app, launched by NHSX, a NHS subsidiary focused on digital innovation, is currently being trialled in the Isle of Wight before national rollout across the UK.
On Friday 1 May, Faculty AI’s co-founding chief executive Marc Warner wrote in The Times that “despite claims by some, we are not working on the contact tracing app”.
However, a paper published by Oxford University’s Big Data Institute appears to contradict this flat denial, confirming that Faculty is directly involved in the modelling research that will “configure” and “optimise” the NHSX app.
The paper describing the Institute’s model, released on 16 April, identifies three Faculty employees as its co-authors: two Faculty senior data scientists, Ares Meroueh and Scott Stevenson, along with Faculty customer engineer, Bryn Mathias.
According to an official Oxford University press notice, the epidemiological model developed by the Big Data Institute team with Faculty’s support will “help configure a contact tracing app for Coronavirus. The model offers several safe configurations to introduce an app and a framework to optimise the app after it is released”.
The role of these three Faculty staffers reveals Faculty’s involvement in the design of the NHSX contact tracing app through a research backdoor.
Faculty refused to deny on the record that they are involved in development of the NHSX contact tracing app. But, according to Oxford University’s Coronavirus Fraser Group, “IBM UK and Faculty worked alongside each other and built the Python interface for the model which enables it to be used more widely by partners, such as NHSX… A team led by Dr Ilya Feige, Director of AI at Faculty also supported the development of the Python interface so that the model can easily be calibrated for different outbreaks. The Faculty team also prepared the repository for open sourcing, such as arranging the open source licence and helping to test the Python interface.”
Fears of Mission Creep
Marc Warner’s brother, Ben Warner, is currently a No. 10 advisor on digital solutions who as a former principal at Faculty (previously known as ASI Data Science), worked closely with Dominic Cummings on Vote Leave’s modelling. At the time this modelling was being overseen by Canadian data firm AggregateIQ (AIQ), which in turn was working for notorious data-mining firm Cambridge Analytica.
Last year, the final report of a Select Committee parliamentary inquiry concluded: “The work of AIQ highlights the fact that data has been and is still being used extensively by private companies to target people, often in a political context, in order to influence their decisions.”
Both the Warner brothers and Dominic Cummings have attended meetings of the Government’s Scientific Advisory Group on Emergencies (SAGE).
Prior to the COVID-19 pandemic, along with several other companies the firm had a contract with the Government to build a £250 million AI lab for NHSX. Over a period of 18 months, Faculty has had at least seven Government contracts worth nearly £1 million.
As part of the Government’s Coronavirus response, Faculty is also working with US big data giant Palantir in a massive data-mining exercise to process large volumes of confidential UK patient information in a centralised Government database.
A joint statement by dozens of UK privacy and security experts has raised further questions about privacy issues, noting that the current app design “records centrally the de-anonymised ID of someone who is infected and also the IDs of all those with whom the infected person has been in contact. This facility would enable (via mission creep) a form of surveillance.”
The letter especially warns against the creation of a ‘social graph’, which would enable tracking of people’s interactions. Matthew Gould, CEO of NHSX, has previously described this as one of the unit’s core goals. “With access to the social graph, a bad actor (state, private sector, or hacker) could spy on citizens’ real-world activities”, the statement says.
But Will It Work?
I spoke to clinical epidemiologist and statistical geneticist Dr Deepti Gurdasani, co-author of a new paper in The Lancet Global Health on how contact tracing could have suppressed the virus, about the modelling behind the NHSX app.
Dr Gurdasani, who is based at Barts and The London School of Medicine and Dentistry’s William Harvey Research Institute, Queen Mary University, and a former senior staff-scientist at the Wellcome Sanger Institute, welcomed the model as an important contribution to our understanding of the power of contact tracing. But she also pointed out some limitations.
“Of course the modelling itself doesn’t speak to the huge concerns about the way the app has been designed, the lack of transparency around what will be done with the data, and the leadership of this by individuals who were aligned with Vote Leave,” she said.
“Ultimately, success of the app is highly dependent on uptake, as most researchers agree, and uptake will likely be hit by lack of trust, if there isn’t transparency around who is developing the app, why they were given the tender, who data will be shared with, and what are the plans for destroying these data in the future.”
But perhaps the most critical limitation in the model co-developed with Faculty is that it “may not be completely valid in care home settings, and healthcare settings, where we are seeing a lot of the transmission occurring at the moment.”
According to Dr Gurdasani, the model is based partly on 2008 data which may not be generalisable to the whole UK population, and which could systematically underrepresent populations like elderly people in care homes, who are currently contributing to a substantial and increasing proportion of all deaths in the UK.
The model also overlooks hospital interactions – where patients infect other patients, infecting healthcare workers, who infect other patients and their families; and assumes that people over 70 years old are successfully quarantined, despite emerging evidence that in many cases such shielding is not really possible (care home settings being a prime example).
Even if privacy concerns can be resolved, she said, the bottom line is that the NHSX app alone “isn’t going to be sufficient to control outbreaks in several important settings, such as care homes. We definitely need manual contact tracing, very much at the local and regional level, carried out by people who understand local communities, their structures, and can engage with them with trust to do this.”
The headline was corrected to reflect the statement from Oxford University that the involvement of Faculty AI was historic.