Lies, Damned Lies and Diversity Statistics

On Wednesday the Creative Diversity Network (CDN) published the diversity figures for the UK television industry collected by Diamond and my Twitter feed went into melt down.

Here is a sample of the Twitter reaction:

"the publication of this diabolical @tweetCDN report proves why we need far better Diversity reporting & data analysis because this is not fooling anyone.” @ScreenNation

"The whole thing is a ridiculous farce.” @BlaakRichardson

"This is a blatant lie” @Shabir1

"Failing us. One report at a time.” @courttianewland

The outrage extended beyond the “usual suspects” of people interested in TV diversity.

Uzma Mir-Young a board member of Glasgow Film tweeted “Wow. I thought Diamond was a good idea, but this is NOT my experience at all.” @Uzma MY

Respected BBC journalist and presenter, Rajan Datar, responded to the report by saying: “That is very worrying indeed. This cannot be allowed to give the impression that things are going swimmingly” @rajandatar

And the actor and senior member of Equity Daniel York tweeted; “This is misleading at best, mendacious at worst.” @danielfyork

So what is all the fuss about?

First a little background.

The Creative Diversity Network (CDN) is an organization supported by all 5 major British broadcasters, BBC, ITV, Channel 4, Viacom/C5 and Sky, with the task of coordinating efforts to increase diversity in the industry and to “share, discuss and profile the work that the UK broadcasters are doing around the diversity agenda. Both independently and collectively”.

Key to this is Diamond, (formerly called Project Diamond).

Diamond is an online system where television productions can report their diversity figures, both on-screen and behind the camera, of every programme they make for one of the five major broadcasters.

If you believe the old adage that “sunlight is the best disinfectant” Diamond should be a game changer for TV diversity as it should reveal what is happening on TV productions and enable appropriate action to be taken.

But the recently published report seems to be far from the “sunlight” that most people have been hoping for.

For example the report said 16.4% of senior TV commissioners are from a Black, Asian or Minority Ethnic background (BAME). Anyone who works in UK television knows that reporting such a figure is laughable, and that is putting it politely.

The report also said 12.8% of people working behind the camera are BAME. And that BAMEs are over-represented behind the camera in News & Current Affairs, Entertainment, Music, Education and Factual Entertainment programmes. Again anyone who works in television knows this is simply ridiculous.

To put these figures in context the number of BAME commissioners at the BBC is so low that in their annual diversity report they have replaced the number with an asterisks because in the words of the BBC “Data has been replaced with an asterisk where figures are below sample size”. For BBC Studios, the department that makes TV programmes, only 9.6% of its staff are BAME and 7.7% of its freelancers are BAME. If the Diamond report is right that would mean the rest of the industry are roughly twice as good at employing BAME people than the BBC. Even the harshest BBC critic doesn’t believe that.

So how did Diamond arrive at these figures?

The report acknowledges that only 25.2% of productions actually reported their diversity figures to Diamond.

Therefore in the words of the National Centre for Social Research, the body who reviewed the report,"there is the inevitable possibility of reporting bias due to sample self-selection as a consequence of the low response rate, and this caveat must be included in the methodology, and results interpreted with a degree of caution."

For the non-statisticians and social scientists amongst us that one sentence basically means; “these figures are meaningless as in all likelihood only the best students bothered to hand in their homework”.

So it is a bad report and the figures are terrible – but why are so many people angry?

People are angry because that one sentence admitting that the figures need to be “interpreted with a degree of caution” is buried on page 24 of the 28 page report (not counting the appendix).

Most politicians, journalists and even industry figures just read the executive summary and won’t read the important sentence 4 pages from the end.

There is the very real risk that people will read the report by the major industry body and supported by all five major UK broadcasters and think that TV does not have a diversity problem. And if they think that they definitely won’t commit resources & institute policies to address it.

Far from helping to address diversity problems in the industry this report this report could set things back – and that is enough to make a lot of people angry.

TV has a diversity problem and CDN publishing the Diamond data in its current form does not help. Diamond is a great idea and the truth is the response rate of TV productions of reporting their diversity data was marginally better this year than last. Eventually I hope it will be a useful tool in tackling diversity.

But we are not there yet.