Steven Barnett, Professor of Communications at Westminster University, on why broadcasters, academics and members of the public should be careful to trust newspaper coverage of polls and call out bogus claims about public opinion.
Opinion polls exude an aura of scientific truth.
Those numbers and percentages are so reassuringly solid, especially when generated by one of the well-known names of the polling world, that for many people they represent hard evidence of the state of British public opinion.
So when the Daily Telegraph announces in a stark front page headline that ‘Public Backs Johnson to Shut Down Parliament for Brexit’, majority public support for such a huge constitutional risk seems assured. And when this headline is reiterated by Newsnight’s Emma Barnett – who rarely lets anything remotely contentious slip by – without so much as a raised eyebrow, it receives an implicit nod of authority from the nation’s impartial public broadcaster.
In fact, it is complete nonsense.
At best, a misleading interpretation of a simplistic poll attempting to tap public feelings on a complex and controversial issue; at worst, a deliberate and malign distortion of a poorly designed poll, with a headline calculated to set the news agenda and serve the interests of the Prime Minister and his strategy sidekick Dominic Cummings.
Let’s take the poll first.
Anyone who understands questionnaire design will tell you that both question wording and question order can easily bias responses if not carefully drafted.
There was little sign of careful drafting. The headline was based on a question which invited respondents to agree or disagree with the statement “Boris needs to deliver Brexit by any means, including suspending parliament if necessary, in order to prevent MPs from stopping it”. It doesn’t take an expert in research methods to understand that there are at least four problems here.
First, there is “Boris”: not Mr Johnson or the Prime Minister, but that nice uncle “Boris” with whom we are all intimately familiar. Who wouldn’t hurt a fly.
Second, there is the double question – breaking all the usual rules of simplicity, which buries the suspension of Parliament inside a statement about delivering Brexit “by any means”.
Third, there is what academics grandly call “acquiescence bias” which simply means we are all more prone to agreeing with statements than disagreeing with them.
Fourth, and most egregious of all, there is the rider that this is all about preventing MPs from stopping Brexit rather than allowing MPs to prevent a ‘no deal’ Brexit (which would almost certainly command a higher level of support). That rider is even more problematic given the order of questions because it followed almost immediately after two bluntly anti-Parliament agree/disagree statements. One read: “On Brexit, most MPs seem to ignore the wishes of voters and push their own agendas”. The other: “Parliament is out of touch with the British public”.
In the art of questionnaire design, respondent-priming is a well-known phenomenon and generally avoided. By the time they got to the uncle Boris question, respondents would hardly be well disposed to MPs. Had the Parliament suspension question come first, unqualified by cuddly Boris or those dastardly MPs, the results would almost certainly have been very different.
Then there was the Telegraph reporting, which deliberately distorted weak data to produce the headline it desperately wanted in pursuit of its unadulterated pro-Johnson agenda.
First, there is “Boris”: not Mr Johnson or the Prime Minister, but that nice uncle “Boris” with whom we are all intimately familiar.
That suspending Parliament has “the support of more than half of the public” is a straightforward lie, derived from ignoring the one in five “don’t knows”. In fact, 44% agreed with the dodgy statement, 37% disagreed, and 19% said they did not know. However much you try to slice up the statistical cake, 44% is not half of the British public.
There has been at least one accuracy complaint to the press regulator IPSO which, if past behaviour is any guide, will now take several months to determine that everything is absolutely fine; or that perhaps a tiny correction might be warranted sometime next decade at the bottom of page 17. By which time either Brexit or Boris Johnson – and quite possibly both – will be ancient history.
If this was a single lapse – either by polling company or by publisher – it might be less worrying. But, our Brexit-loving press will be using every conceivable device over the next 10 weeks to persuade MPs that a Prime Minister with no democratic mandate of his own is reflecting the ‘will of the people’ in his determination to crash us out of the EU.
Not only will they be exploiting polling companies who really ought to know better, they will also be exploiting broadcasters who routinely feature the next day’s press headlines in their programmes, as if these come from trusted bi-partisan seekers after truth rather than propaganda machines for Johnson and his band of hardline Brexiteers.
With declining circulations, the Telegraph, Sun, Daily Mail and Express should be less powerful than during their halcyon days of last century’s elections. But, their ability to influence agendas in other news organisations and to provide content for television programmes that are cash-strapped and seemingly addicted to newspaper headlines, whatever their circulations, remain as potent as ever.
It will therefore be down to the producers of those programmes on BBC News and Sky News, as well as Newsnight, The Andrew Marr Show, Peston and all the other political programmes that still love to feature our daily papers, to treat every poll and every statement of “public opinion” with a huge dose of scepticism. They need to remember that there are newspaper journalists and editors who will be seeking to spread deliberate untruths under the cloak of “objective” opinion research.
As for the pollsters, it is time they took responsibility for how their work is reported as well as conducted. The Market Research Society used to insist that its members take steps “to check and where necessary amend any client-prepared materials prior to publication”. But many, including ComRes which conducted the Telegraph poll, do not belong to the MRS and therefore allow themselves to become accomplices to newspaper fraud.
They, along with responsible broadcasters, academics and ordinary members of the public, should be watching carefully and calling out bogus claims about public opinion.