Race Report: Sewell Commission Couldn’t Find Something It Wasn’t Looking For

Race ReportSewell Commission Couldn’t Find Something It Wasn’t Looking For

The methodology used by the Commission on Racial and Ethnic Disparities excluded the possibility of finding that differences in outcomes are the result of race, says Jonathan Portes

Fifteen years ago, when I was Chief Economist at the Department for Work and Pensions, I proposed that it fund a ‘field experiment’ test for racial discrimination in hiring, where researchers examine whether otherwise identical (fictional) CVs will get different response rates from employers depending on the ethnic identity of the name attached to the CV.

Two ministers refused to sign-off on the research, on the grounds of the political sensitivity both of the methodology and the potential findings. A third approved it, observing “I’m going to regret this, aren’t I?”. 

The research – a chart from which features in the recent report of the Commission on Racial and Ethnic Disparities – found clear and incontrovertible evidence of high levels of direct discrimination against applicants from ethnic minority backgrounds, whether deliberate or unconscious. Subsequent research, some as recent as a year ago, has confirmed its findings.

The politicisation of research relating to racism in the UK – and alongside that, attempts to distort or minimise it – is nothing new. A lot of ink has been spilled over the Commission’s report, in particular its supposed conclusion that “the UK is not institutionally racist”. In fact, not only did the report itself not say that, it is not even clear what such a claim would even mean: a country is not an institution. 

This led to some amusing contradictions. Legatum Institute researcher and regular Daily Mail columnist Matthew Goodwin, eager to hype the report, joyfully tweeted that the “UK is not institutionally racist: New evidence-based report”. He then repeated this obviously nonsensical claim, only to be left high and dry by Samir Shah, one of the Commission’s members, who wrote: “It was determined that the Commission had said institutional racism did not exist… Did these people bother to read the report?”.  Evidently not Goodwin or other excitable commentators.

But this somewhat confected dispute is a distraction from the more substantive problems with the report. These relate to its argument on the crucial issue it was set up to address: what causes “racial disparities”, or differences in outcomes, in terms of health, education, employment and more between people of different ethnicities?  

It is here that the Commission, in its effort to avoid one statistical trap, falls directly into several others.

Correlation is not causation: different outcomes for different groups do not, in themselves, reveal that group membership is the key or only driver. For example, ethnic minorities are younger, on average, than white Britons. It is not surprising, therefore, that their crude mortality rates are lower. 

The Commission is therefore correct that worse outcomes for an ethnic group do not in themselves prove direct discrimination against that group. One cannot, for example, look at the substantially elevated risk of dying from COVID-19 for many ethnic minority groups and conclude simply from the raw data that this is the result of structural racism in the provision of healthcare in the UK.  

But the Commission’s approach in dealing with this is profoundly flawed, conceptually and methodologically.  First, it divides observed disparities into two categories: “explained” and “unexplained”.  It states:

So disparities are either explained by factors other than racism – or there is no evidence so they are unexplained. Thus, apparently, while racism does exist (and the Commission goes on to set out its own definitions of systemic, structural and institutional racism), there is no way, within its framework, to demonstrate, through the use of evidence or analysis, that racism or discrimination, indirect or direct, is actually causing the observed disparities in outcomes.  

Therefore, it is not that the Commission did not find any evidence that disparities are the result of race or racism – it excluded the possibility ex ante.  

Once this Catch-22 is understood, it is easy to understand why supporters and critics of the Commission were taken in – and alternatively exhilarated or enraged – by the claims of it’s chair, Tony Sewell (not repeated in the report itself), that it had found “no evidence” of institutional racism. 

It is hard to find something if it is not being looked for. And it’s even harder to do so if a methodology is used which excludes, by definition, even the possibility of finding it.

---

A Basic Statistical Error

Even worse than this definitional sleight of hand is the Commission’s approach to the data when it comes to ‘explaining’ racial disparities.  

In practice, this is done through regression analysis, where instead of looking at the simple correlation between ethnicity and outcomes, outcomes ‘controlling’ for other variables are looked at.

As Professor Eric Kaufmann, another of the report’s cheerleaders, put it: “Yet to discern whether racism is involved requires controlling for minorities’ disproportionate employment in frontline services and their multi-generational housing and higher rate of pre-existing health conditions. Instead of crude racial aggregates, the commissioners seek a new scientific emphasis on multivariate analysis of individual-level data to isolate discrimination from confounding factors such as poverty or urbanity.”

The report is better viewed as ‘rhetoric-based evidence making’

Based on this approach, the Commission concluded: “The evidence shows that geography, family influence, socio-economic background, culture and religion have more significant impact on life chances than the existence of racism.”

But this is not ‘scientific’. Instead, it is a very basic statistical error – indeed one that every undergraduate learning about quantitative methods in social science should recognise – described as ‘conditioning on a post-treatment variable’. The impact of someone’s race on their health cannot be dismissed by saying ‘well, actually, poverty is the “real” cause’, if poverty and race are – as they are in the UK – inextricably linked. 

This isn’t hard to see. It is true that, controlling for one’s university degree, one’s ‘A’ Level results probably don’t make much difference to future earnings. But that doesn’t mean one shouldn’t bother studying for them. Equally, poverty and urbanity are not “confounding factors” when it comes to assessing the impact of race on outcomes; they are potential channels. 

Structural inequality is a complex interplay of causes and outcomes – and one variable can be both at once. Sticking as many variables as possible on one side of a regression and claiming you’ve ‘explained’ away race and racism (or sexism or poverty) is not a credible analysis. Yet, it is precisely what this report has tried to do.

The more one attempts to unpick this, the more circular it becomes. The disparities the Commission is trying to ‘explain’ – education, employment, etc – are complex and multi-dimensional economic and social outcomes. Of course they are not directly ’caused’ by the colour of someone’s skin, but by other, intermediate channels and drivers, themselves mostly social and economic. Showing the influence of those channels is not about explaining away the impact of race, but about explaining how race (and other factors) relate to outcomes. 

This does not mean that race or racism drives everything – that would be equally reductive. What it does mean is that, if racial (or other) disparities are to be understood, they have to be unpicked and what is driving what needs to be thought about. But ‘explaining them away’ – and hence giving up on doing any serious or credible analysis – is precisely what the Commission has sought to do. 

---

Comprehensive Incomprehension

In my specialist area – the labour market – the Commission presents some basic statistical analysis of both pay and employment ‘gaps’ and claims that it shows an “overall convergence story on both employment and pay”. But it can only do this by some fairly crude sleight of hand.

For example, it analyses pay adjusted for various control variables, which reduce the size of observed gaps – but then looks at unadjusted employment rates, presumably because adjustment would increase the size of the gaps. In any case, there is no attempt to justify this inconsistency. As Alan Manning and Rebecca Rose observe: “It is clear there is no evidence for pay gaps being smaller for ethnic minorities now than they were 25 years ago, contrary to the impression given by the Sewell Report… the overall impression is statis.”

It gets even worse when it comes to the field discrimination tests. The Commission’s conclusion is that “we know that discrimination occurs, but these experiments cannot be relied upon to provide clarity on the extent that it happens in everyday life… it is unclear if this effect is about race, class or perceived foreign culture”. As Manning and Rose observe, this is garbled at best: nobody who had actually read or understood the research could have written this. 

Similar examples litter the report. Indeed, it is almost impossible to find a topic on which someone with acknowledged expertise has not demolished the report’s analysis or complained about the distortion or misrepresentation of their own research – from globally-respected public health expert Sir Michael Marmot, Professor Alex Stevens on drugs and crime to Colin Angus on COVID-19 deaths and Kenan Malik on family structure (and this is only a partial list). 

In 35 years of both producing and consuming government reports, I don’t think I have ever seen one where the evidence and analysis has been so comprehensively discredited so quickly and completely.

So where does this leave us? Some people were quick to accuse the Commission of ‘policy-based evidence making’ (instead of ‘evidence-based policy making’). I think this gives it too much credit as the Commission’s policy proposals are mostly sensible, unoriginal and inoffensive. 

Instead, the report is better viewed as ‘rhetoric-based evidence making’: the point was not to distort the evidence to provide a basis for specific policies, but to justify the sort of vitriolic hyperbole deployed by Professor Kaufmann when he says that it is a “major blow against institutional wokeness… the sacralisation of historically disadvantaged racial, sexual and gender minorities”. 

Does that mean that the facts, data and evidence – and their misuse – don’t really matter? No, quite the contrary. By commissioning and endorsing such an egregiously and obviously flawed report, those who would like to take us backwards have over-played their hand. Responding with generalisations or personal attacks serves their agenda, allowing them to ignore the issues and turn this into yet another battle in the ‘War on Woke’.  Instead, as Sunder Katwala argues, the focus should turn from rhetoric to action. 

Jonathan Portes is Professor of Economics and Public Policy at the School of Politics and Economics at King’s College London