I was unable to see any Fringe shows this year due to a crippling bout of agoraphobia . I was miserable to be missing out, and by my third month of hermit-like existence I was desperate to participate in some way, so idly decided to tally up a few star ratings to see if men and women were rated equally . It’ll kill an hour or so, I thought, and in any case, I was running short on episodes of Undercover Boss.
What followed turned into a two week full-time project, during which I had to teach myself first Excel and then (in a late realisation) basic maths.
What follows are my findings. It’s a long read, but don’t worry, there’s pictures.
Review stars, eh? What a wonderful, unbiased way of summarising the artistic originality and accomplishment of an act. In the Edinburgh sea of nearly 4000 comedy shows, the 1*- 5* rating system is an easy way of separating the wheat from the chaff, and letting punters know the best shows out there.
Or is it?
During August, I’ve lurked like a benign spectre on various Facebook groups for performers and have seen the same themes crop up again and again. Gender bias. This publication only awards 1*s to women. That reviewer only rates men. This reviewer reckons a woman discussing her fertility is somehow too niche for public consumption: two stars.
Of course, it would be unnatural if there weren’t a small punnet of sour grapes attached to some of these remarks; a comic probably won’t ask a reviewer to their wedding if they’ve said their show’s a hackneyed pile of garbage.
But individual beefs aside, I wanted to know if there was any truth in the rumours of gender bias in the awarding of stars, so I decided to crunch the numbers to ascertain if bias really existed, and who the worst offenders were.
In all, I looked at 1530 reviews across 12 publications: Fest, The Skinny, Broadway Baby, Ed Fest Mag, The Wee Review, The Scotsman, One 4 Review, Chortle, Fringe Guru, The List, The Telegraph and The Guardian. Although the latter two had small sample sizes, their ratings mean a lot so were worthy of inclusion. (I’m sure I missed some important publications, and it would help me greatly if you could let me know in the comments, just in case I decide to embark upon this foolhardy enterprise again next year).
There are significant limitations to my research. The data were collected on 24 August before all publications has submitted their reviews, and is therefore not entirely complete (this was the only way to analyse the data before the Fringe was a distant memory). I had to make not-wholly-scientific decisions about who I included; although I only analysed the ‘comedy’ category, I had to make judgement calls about large mixed-gender troupes (I left them out) and sketch groups where the ratio is two male one female and a female director (thanks Geins) or vice versa. A serious criticism of the research is that it operates on the gender binary – I had to guess gender from the artist’s name; failing that from their photo; and if I was really stuck then the pronouns used in the show blurb, so anyone who chose to identify as non-binary would likely have been noticed but the research doesn’t have the capacity to account for this. Finally, the research ignores intersectional axes of oppression such as disability, sexuality, race, religion and class. Although these are big issues and shouldn’t be ignored, it would have been impossible and foolish to try and make judgement calls about these signifiers based on the little information I had.
One factor I was really shocked by is that the Edinburgh Fringe don’t collate any information whatsoever about the demographic of participants – and this would be so easy to do, since performers must complete some sort of registration – how hard would it be to include an equal opportunities monitoring form? When I spoke to the press office to see if they held data on the gender split of comics at the Fringe, my question was blithely laughed off with the claim that ‘the Fringe is open to everyone’.
Yes, but so is Eton. Oh, unless you’re a woman (amirite Lou Sanders?). It seems to me that the Fringe is missing a big trick here, and as it becomes increasingly crowded with performers and expensive to play, it seems likely that the most privileged groups will dominate and elitism will prevail. Equalities monitoring wouldn’t stop that, but it would give a jumping off point to understand how structural inequalities are preventing certain groups from participating in this ostensibly ‘open’ event.
Notwithstanding the limitations of the data, here’s what I discovered.
There are more male than female comics performing so you can’t compare like for like. The publications I looked at reviewed an average of 69% men to 31% women, so star ratings were weighted to reflect this. Here’s how all stars awarded looked across all publications.
Looks pretty unbiased, right? With the exception of the male weighting towards 1* reviews (which is a little misleading as the sample size of 1* reviews is tiny – many publications don’t give any at all).
But although the overall picture looks benign, once I began to break down star ratings to each publication, a different picture emerged.
In order to ascertain how fairly stars were awarded to men and women, I calculated the average gender split reviewed by each publication, and then worked out what percentage of stars were awarded to men and women in each ratings category. So, as an example, The Wee Review reviewed 66% men and 34% women, yet awarded 93% of their five star ratings to men and the remaining 7% to women, so it’s clear to see that there is some bias at play here. However, the same publication may not replicate the same level of bias across all star categories – for instance, in the case of this publication the 4* awards were tipped in favour of women, and the 3* awards were fairly even.
It’s not an exact science.
I’d gone in, somewhat naively, expecting bias to be screaming itself loud and clear from the rooftops, and yet the findings were nowhere near that simple.
There was only one publication – The Scotsman – that indicated bias in favour of men across the board, awarding them far more 5* and 4* ratings and reserving the 1*s and 2*s for the women. I spoke to the Arts Editor of this magazine for comment, however he simply informed me that his reviewers were all unbiased and highly experienced thank-you-very-much and assured me my remarks would be borne in mind at next year’s festival. So much, so revolutionary.
By contrast, I spoke to Ben Venables, editor of The Skinny, which was much more favourable to women in awarding star ratings across the board. Ben explained there was a clear ethos in place about criteria for reviewing in general; schedules were assigned in advance and at random so reviewers could not choose ‘favourite’ performers – a reviewer from a different publication (who asked not to be named) told me that it’s commonplace for people to choose who they review. The Skinny placed an emphasis on seeing acts debuting the Fringe for the first time, and there was a strict criteria for allocation of stars, with very few being given out at both ends of the spectrum. Even more hearteningly, Ben showed an awareness that even where they were getting it right on areas like gender, their reviewers still tended to meet a certain age, race and class demographic. He recognised the imperfections in their system, which is at least a start. He also told me that they vet potential reviewers. Anyone with the attitude that ‘women aren’t funny’ never gets through the door.
Of the other publications I reviewed, it was hard to draw any concrete conclusions as each one painted a unique picture. It did seem as if more 5* awards were going to men with the notable exception of Edinburgh Comedy Award winner Hannah Gadsby. I was left wondering had she not performed the festival this year, if women would have been represented much at all at the 5* mark.
Here’s the rest of the publications broken down.
Why is any of this relevant? As each performer tells themselves for three weeks, ‘reviews don’t matter.’ Except they do. The comic Athena Kugblenu explained to me how she hates the review system, yet ‘I want the 4* and 5* as much as the next comic, not for the endorsement, but for the marketing and PR.’ And as Edinburgh stalwart Kate Smurthwaite explains, ‘Anything less than a 4/5 star is of no use cos you can’t put 3* on your posters.’ I spoke to one experienced reviewer who hypothesised that reviewers simply aren’t going to enough female shows. Proportional representation is all very well but because female comics are still in the minority, their work is still viewed as marginal which means reviewers are less likely to award the more prestigious star ratings. As encapsulated in the comment that Sara Pascoe’s material was ‘too tamponny’; when male comics talk about their lives it’s comedy. When female comics talk about their lives, their shows are described as focusing only on ‘women’s issues’ or pigeonholed as feminist, as if feminism was somehow a bad thing rather than just a wish for gender equality and an end to young girls having their genitals mutilated. Something one hopes we would all strive for, whatever our gender.
Although the project above focused on numbers rather than qualitative analysis, there’s no doubt that the use of language in male vs female reviews is an area ripe for research. As a little test I decided to peek at the content of the reviews of Sara Pascoe’s acclaimed show Lads Lads Lads and the 2017 Comedy Award winner John Robins’ show The Darkness of Robins. I chose these shows because I’ve seen them both, and although the content of the shows isn’t identical, they were both written on the back of their breakup in late 2016.
The reviews I looked at were from the Guardian, the Standard, the Skinny, the Scotsman and Chortle, and all awarded 4*. Of the five reviews of Pascoe’s show, she was referred to as endearing twice, vulnerable once and ‘oversharing about sex’ – presumably because she had the temerity to mention masturbation – a theme that’s rarely absent from most male standups’ staple material. Robin’s show, despite containing several depictions of sex with his ex partner and a whole bit about women’s pubic hair, did not attract any comment on the topic of sex whatsoever. Adjectives attached to Robins’ show included ‘extremely forthright’, ‘high-powered, front foot delivery’ and ‘an eloquent howl of rage’. The assertive adjectives attached to Robins’ reviews are ironic if only because in both Pascoe and Robins’ depiction of their break-up, she is undeniably the one who is taking it better.
The most telling observation, however, came from the three out of five reviewers who commented on Pascoe’s stage attire – a sparkly bodysuit and fishnets. This might be understandable were it not for the fact the John Robins went on stage every night donning a tailor-made bright yellow leather replica Freddie Mercury jacket. Not. A. Mention. [Edit – apparently John ditched the jacket after the first two previews so this isn’t an entirely fair comparison. Thanks to commenter Pete for pointing this out].
Kate Smurthwaite provides some examples of gendered reviews from her own experience.
My first Chortle review call[ed] me “too alpha female”. My second says sometime I make a political point instead of a joke which is TRUE but that’s the style of my comedy and no-one complains when Mark Steel does it.
We also need to take into account the structural bias behind the scenes before shows even make it to the review stage. Smurthwaite claims that bias amongst venue managers means women end up with less favourable rooms and slots, and a poor slot with a barely warmed up or hungover audience, taking in their first show of the day, can have a devastating effect on reviews.
Mark Twain was right when he talked about lies, damn lies, and statistics. I embarked upon this project looking for easy (or at least clear-cut) answers, and aside from the fairly unequivocal findings of the Scotsman, the results turned out to be a little more complicated.
It has certainly given me food for thought about how we can build on this information in years to come, and I hope that comics of all genders will feed in with their own experiences. I’ve set up a Facebook group for this purpose: Edinburgh Fringe Ratings.
Female (and many male) comics don’t need to wade through a 2000 word blog to know there’s gender bias in comedy. I just fancied drawing a few graphs to illustrate it, and as Stewart Lee’s apocryphal taxi driver once remarked “Well. You can prove anything with facts”.
For the full report and datasets, visit http://wp.me/p8JV3i-dU.
I welcome your comments and notification of any errors via this blog. Please bear in mind I’m an amateur unpaid blogger, so by all means tell me I’m wrong but please do so with respect. If you like this article you can help by sharing it on your social media.
Edit: Quite a few commenters are asking why I didn’t take into account the gender of the researcher. Here’s my response:
I consciously didn’t analyse the gender, ethnicity, sexuality, age or class demographic of the researchers because i thought it was a red herring. There’s no obligation for women to support women and women are just as subject to the patriarchy as men. They’ve seen more male comics in more prestigious venues/media slots etc. They’re sitting in an audience of people who may well have the embedded beliefs that men will automatically be funny, women have to prove they will be, and god forbid they should hear a mention of ovaries. This of course impacts on the performance, audience reaction and subsequent impression left upon the reviewer.
I do think there is space for [analysis of the reviewers], perhaps as part of a larger piece of research that looks at organisational culture. I was struck by the comments from the Scotsman that their reviewers were well established and experienced, and the Skinny that they tended to have a high turnover of younger reviewers. I think these factors surely also have a bearing as much as the researcher’s gender.