By John Keefer
Let’s put it on the table. I don’t like game score aggregation sites.
GameRankings and MetaCritic were thorns in my side when I was editorial director at GameSpy, particularly when those sites take a 5-star system and rigidly convert it to a percentage (“3 stars is NOT 60%”) . Then try having a conversation with the game’s publisher who only looks at the score on those sites:
Publisher: “You gave our game a 70!”
Me: “No we didn’t. We gave it 3½ stars. That means it is a good game. Did you read the review?”
Publisher: “No, but I saw the score.”
Me: “Where did you see it as a 70?”
Publisher: “GameRankings said you gave it a 70.”
When publishers and even readers put so much emphasis on scores and don’t even bother to read the words associated with the review, then something is wrong with the system. And I believe that the aggregate sites are at the core of the problem. It’s all about the numbers and not about the meaning or the context.
When I started the ill-fated Crispy Gamer, I wanted to try to get around that. When a reader looks at a review, what do they what to know? They want to know if they should spend their money on it. If you are creating a site for the reader and want to be useful, then get them talking about the review and not the score. Crispy’s Buy/Try/Fry system was a simple guideline that was clear on what the site thought of the game. No arguments about a 72 versus a 77 and how the reviewer arbitrarily came to that number.
And I really didn’t give a rat’s ass whether the aggregate sites included us in their metric.
In the end, it was probably a bit idealistic. It’s always about the eyeballs but, while the site generated a fair share of traffic and chatter, we couldn’t put enough of a dent in the existing mindset.
I still believe that the scoring system is broken and top-heavy with scores from 7-9 (70% to 90% for the aggregate impaired) and it is the emphasis on the aggregate sites by fans AND publishers that keep it from being fixed.
Unfortunately it will take more than a few isolated voices to change it.
(Originally published as a sidebar in EGM magazine, 2010)