Are Game Magazines Too Soft On Indies?
A criticism that you hear all the time on gaming forums is that the mainstream publications are too soft in their review scoring of AAA titles from major publishers. From a distance, it appears to be editorial policy at certain magazines that games from certain publishers can't be scored any lower than an "8" or an "80%", regardless of how poorly received they are in any and all other quarters.
There's two major reasons for this. On an individual level, a lot of "games journalists" don't regard or carry themselves as actual journalists. This probably began with the game magazines recruiting more from people who built popular gaming websites and forums (people who wanted to be "in the industry" and to "play games for a living" so bad that they were willing to accept the low wages and horrid working conditions that experienced professional writers would not) than from candidates with actual journalism degrees and backgrounds in the field. Anyone with even basic journalism training would have been exposed along the way to a code of professional ethics that makes clear that taking paid product or letting a company pay your airfare, hotel and
food to cover an event is strictly off limits. Recruiting noobs without that level of professional discipline, however, allows ample room for this sort of thing:
And when the public started calling them on it, the response was to institutionalize it instead of cleaning it up. "Oh, see, we're not *real* journalism", responded many of the magazines, "we're the "enthusiast press." It's totally different. Junkets and free merch are totally OK here, Because Of Reasons."
That's the "micro" reason that gaming journalism is Soft On The Big Boys. The "macro" reason is that gaming journalism is almost entirely dependent on The Big Boys for their economic survival, since they're often the sole source of advertising revenue for a publication.
So that's why some people feel that you can't trust the big game rags as far as you can kick them, at least when it comes to AAA gaming. But what about indies? There's no economic incentive to inflate the score of an indie game. As well as they've done for themselves lately, Klei and Galactic Cafe still probably can't afford to pay anyone off. I get the sense a junket hosted by Edmund McMillen would be in his basement, with him saying nothing more than "There are sandwiches" before retreating to a dark corner and spending the rest of the event uncomfortably staring at everyone without speaking. Notch would ... er, well, maybe this premise isn't always true
. But with the exception of the one indie outfit that sold 50 million copies and thus probably shouldn't be considered "indie" anymore, even the most popular indie studios still don't have a fraction of the finances or the clout that the AAA companies do.
So why inflate the indies? There's a few possibilities.
1) An admirable (but misplaced) desire to help out the struggling little guy
2) An equally misplaced sense of doing penance or balancing the karmic scales for any AAA whoresmanship
they've engaged in by helping said little guy out when they get anything at least partially right
3) Desperation to reach the same level of regard and credibility that film and music criticism enjoy with the general public
4) Following the VICE magazine playbook -- cultivating a bogus sort of hipster, counterculture "cred" that can be packaged and sold right back to all the pretentious hipsters and hipster wanna-bes with lots of disposable income (applies more to the "arthouse" scene, but I feel like some of these games like Meat Boy and VVVVVV at least partially fall into this dynamic as well)
But at this point, we've been dealing almost entirely in broad generalities, and those aren't real helpful. So let's drag this back into the world of the tangible with some examples.
Now, fair warning -- we're about to paddle into the murky waters of Subjectivity, where everyone's opinion about everything is equally valid because they say so and because they'll put their fingers in their ears and go NA NA NA NANANA whenever anyone tries to say otherwise. And (at least to a point), varying subjective enjoyment is a perfectly valid phenomenon for the end user. I may not understand why people want to play the shittest bits of NES platformers cut out and lined up one after another, but over a million people have voted with their dollars to indicate that yes, the modern take on poorly designed 8-bit platformers is in fact what they want. It's their right to supply that demand and entrepreneurial designers have just as much of a right to go out and meet that demand with their product. And good for everyone involved.
But we're focusing on games journalism right now. And even if true objectivity is impossible, anything that calls itself "journalism" is supposed to at least be making one of the better attempts at it.
One objective measure that is available to us is statistical sampling. And while we all (often rightfully) shit on review aggregator sites like Gamerankings and Metacritic, they're actually kind of useful in this regard. Let me use my favorite example here to illustrate:
This is an old meme image, but I've checked Metacritic for updates (as of early July 2014) and the situation hasn't really changed - a range of about 89 to 91 for the critics, contrasted to a rating of 6.3 to 7.3 from actual end users (depending on the platform).
Gone Home is an even more glaring example. The consensus among 1415 players is that it deserves a barely-better-than-average score, but only two out of 55 critics are even anywhere near that neighborhood (and they're still ten full points above).
I picked 15 more indie titles off the top of my head, from among the most popular and recognizable names. I found that most were critically reviewed at between 5 to 10 points above the user median score. Two of them were evenly matched or very close to it, at least for a couple of their platforms, but I couldn't find even one example where the user consensus exceeded that of the critics. This is markedly worse with iOS and "casual" reviewers, where it's not uncommon to see a 20-point difference. So the public is clearly identifying problems that the supposed "expert" reviewers aren't.
Now, there are problems with this analysis, not the least of which that it's not at all scientific. You can also argue the "vulgar popular taste" deviates significantly from the perspective of professionals who have long experience in the field, and while that argument is certainly near and dear to my own heart, it doesn't work here. In fact, it bolsters the opposite argument, because this established pattern of the general public promoting mediocre talent well beyond what it probably deserves (in a truly just universe) should actually give a boost to problematic games that only manage to do one or two things well or just look really nice. If anything, lower general standards should result in a user ratings boost for critically overrated indies. You should at least be able to find ONE example of an indie outscoring the critics with the public if this was the dynamic commonly in play.
Methodological problems aside, I think this does at least demonstrate a trend. Indie games DO tend to be critically overrated -- at least a little -- across the board, as compared to how a more generalized audience takes them. The 4 reasons I cited before are likely the most probable culprits, plus a general melange of other factors I'm too lazy to thoroughly account for here. And it's going to vary by writer, publisher and game in question, I'm sure.
The more pertinent question is, "Why is this such a problem?" And that's much easier to answer. Aside from the fact that we'd just generally like to have mainstream game journalism that we can trust, it's also because indie gaming is pretty much the last refuge of development for all the things that AAA won't take financial chances on any more. Which is pretty much anything that's not a Dudebro shooter, sports game, or bloated fantasy RPG. Reviewers not being properly critical -- for whichever reason or reasons it might be -- sends the message to indie designers that the same lazy patterns that AAA devs get away with are OK for them too. You wonder why we're drowning in a swamp of samey puzzle-platformers, "retro" pixel art and "emotional" linear arthouse games that want desperately to be taken as literature even though they don't actually have a compelling or even coherent story to tell? Because they keep getting good reviews, they keep getting awards, and they keep selling. Aside from diverting talent down the path of lesser resistance that might otherwise be pressured into creating something really interesting , they promote a culture of mediocrity and open a door for hustlers to flood the market with quickly-made garbage just to turn a quick buck. They also take money out of people's pockets when they blithely give a game that's made for a niche (like a ridiculously hard platformer) an overly glowing review for a general audience.
As much as we all like to shit on magazine reviews for being untrustworthy and incompetent, the fact is that they're still very influential and they still drive sales. And a lot of the people driving those sales already had serious problems doing their job properly when dealing only with major publishers, so it's no surprise that they continue to struggle with indies introduced into the mix. As reviewers should serve their function in the greater ecosystem by pressuring designers to get better, the public should be pressuring reviewers to do better too. While the "AAA dependency" problem is one that's not so easy to solve, poor reviewing of indie games is one that actually can be addressed with appropriate pressure. And this becomes increasingly more important as the almost totally unregulated indie market more and more begins to resemble the catalog of the Atari 2600 in the days just before the bottom fell out of it.
- C. M0use