On readers' reviews
Normally, in this column [this article was originally a Prompt Corner editorial column in Theatre Record magazine in 2006] I take my cue from the shows covered (or,
sometimes, conspicuously not covered) during the period in question. I very seldom set about collecting material
for this page with malice aforethought, so to speak. But I have to admit that for most of
September – as far as pressures of time and house move permitted – I have been
keeping watch in one particular quarter, with a view to reporting my
conclusions.
Distorted
It’s often been said – usually without much thought behind it – that
reviewers offer a distorted view of the plays they write about: we get in free,
so we’re not beholden to the show, and we’re paid to write about what we think
rather than how everybody else in the audience seemed to respond. To which I say: on the first count, damned
right, we shouldn’t be obliged to productions; and on the second, any
conscientious reviewer will note audience mood as well, and will consider his
or her own response in relation to it.
Nevertheless, it’s sometimes suggested that getting ordinary punters to
write reviews will somehow give a “truer” picture.
Well, this was tried in the early 1990s by the late, lamented Plays
& Players magazine. Indeed, it
was one of the steps on the road to P&P’s demise: by that point in
their decline they simply couldn’t afford to pay pros, so tried to make a
virtue out of necessity, with results that were patchy at best. And for the past few months, the Independent
has run a spot entitled You write the reviews… (underlining and
ellipsis marks included in the title), in which “civilians” review plays,
concerts etc. Out of curiosity, I
started keeping tabs on these reviews throughout September. I had a hunch...
Mathematical
Now, whatever the rights and wrongs of including star ratings on reviews
(and as you know, Theatre Record does not include such ratings when it
reprints reviews), when considered in bulk they can serve as a handy, albeit
rough, arithmetical indicator of critical mood.
This is despite the fact that different titles, and even reviewers, have
their own methods of calculating star ratings.
When they were introduced in the Financial Times, I put forward a
rather recondite mathematical argument that, assuming a statistically normal
distribution around the mid-point of the scale (i.e. three stars on a scale of
1 to 5, or two and a half on a scale of 0 to 5), one should expect around half
or even more of the shows one sees to merit that middle mark – three stars,
say, with maybe 20% each gathering two or four stars and only the occasional
one or five. Basically, extremes are
rare; far more often you’ll see stuff that’s not especially distinguished one
way or the other.
So, if that theory is valid, if you take a sizeable sample of reviews
and average their ratings, the mean value should be around 3, or a little
under. The Independent’s
September theatre reviews bore out that hypothesis. The punter-written reviews, however, averaged
out at over four stars.
Self-selecting
There are two main reasons for this, I think. The first is fairly obvious: as with any
newspaper, TV or radio poll or debate where the public writes or phones in, the
data is based on what market researchers call a self-selecting sample – people
record their opinions because they feel strongly enough to do so. Few but the most dedicated are going to feel
driven to go to the effort of saying something is not really worth getting
excited about either way. So, unlike my
statistical theory, the input from reader reviews will tend towards the
extremes rather than the middle, the opposite of the standard of work on view
overall. The second reason is less
conspicuous, but still simple. Paying
punters have, well, paid. And they want
to get their money’s worth. They will
have bought their tickets in the first place because they expect to like what
they see, and having laid out money, they will be even more keen to like
it. And so they do. If I had the time (and were sad enough) to
conduct further and deeper study of the subject, I wouldn’t be surprised if
average star rating from paying audience members rose more or less in
proportion to average price paid.
In any case, the conclusion is plain.
The ordinary reader may feel more inclined to trust a similar reader’s
opinion, as being the same species of viewer rather than the exotic, plumed criticus
opinionatus gittus… but in fact the opposite is the case. Detachment is a positive advantage in a
reviewer. It should go without saying,
really, that an impartial judgement is more reliable than a biased one; but
strangely, it’s an idea that simply won’t take root.
Misrepresentation
One other thing about the Independent’s reader review column: on
one or two occasions, a reader has reviewed a show which is still in preview
and therefore on which the paper’s own reviewer has not yet been allowed to
pass an opinion. Quite apart from the
way this marginalises the paper’s own writer
(as I wrote in 2004 with regard to the Guardian’s coverage of David
Hare’s Stuff Happens and the ludicrous position into which Michael
Billington was thrust), there is other fallout. Possibly that particular column’s editor
doesn’t know about these embargos and is breaking them through ignorance rather
than active disregard, but the end result is the same: that paper gets to steal
a march on the competition. I’m sure
that if the editors have noticed this, they’re not against it. Then, of course, if it’s a conspicuously
positive review (as most of them are, remember), the producers of the show can
paste up quotations as being from the Independent. There’s a “gentlemen’s agreement” in force
between the Critics’ Circle and the Society of London Theatre that, whenever a
quote from a paper is not from its main critic, the writer’s name should be
displayed as well as that of the publication.
Sadly, not all producers are gentlemen (or -women). I’m not by any means suggesting that the Indie
is tacitly conniving at such misrepresentation, but it would be nicer still if
the opportunity didn’t even arise.
Reviewing works best when it’s done by reviewers: do the maths!
—Written for Theatre Record, issue 19/2006.