How the press grades games
Gamers and developers alike pay a lot of attention to the opinions expressed in the specialized press. Now, everyone is used to have games evaluated in terms of graphics, sound, originality or whatever. This hasn’t always been the case, though.
Originally, the press just welcomed every game with great enthusiasm. Came the 90s, and that sycophantry led place to heightened criticism. In those times of great experimentation, every game had to bring something special, something new, to push the technical limits forward. And they were scored accordingly: if a game was the first to feature some advance in programming and graphics, that very bit was warmly appreciated, to the expense of what made games, well, games and not programming tour de force.
Another overrated aspect of games in the reviews was that of game duration, a value expressed in hours of gameplay. Fortunately, that criterion, which was one of the most important over the nineties, is now absent from most scoring systems. Often, it has been replaced by gameplay, originality or something like this, which seems to be the new mantra.
The game press, today, still carries that heritage. It still applauses the use of some new lighting technique, the number of polygons used in the main character, and some old-school journalists never hesitate to underline the long time necessary to beat a game, while younger writers boast how the gameplay mechanisms of such or such title are unlike anything done before.
This is fallacious for two reasons.
First, it is wrong to assume that a game is good because some aspects of it deserve a good grade. No game was ever successful, or at least, genuinely appreciated by the gaming audience, thanks to its technical features, its duration or the originality of its gameplay. Sure, some great games are also technical wonders, entertain gamers for ages and bring a shining new contribution to the gaming tradition. But these are rather by-products or consequences of their greatness, than causes. There are, however, legions of average games which in their time were technically advanced, very long or very different, and which, predictably, never met any success. Sadly, the majority of journalists, and virtually every gamer, have a very limited knowledge of the science of game-making, although game reviews are often full of technical considerations, often off the point.
Second, it is wrong to think that such grades are a good proxy for commercial success. In my tenure in the game industry, I have produced a game which received the accolades of the press - it got no less than a 90% in the rather unforgiving French magazine Joystick, and the "golden joystick" distinction, usually handled with great parsimony. The game, however, sold less than 1500 boxes in Europe - not exactly the million-seller that it great scores could have led to think it was.
So are game reviews useless? certainly not.
Game reviews are usually in tune with the opinions of the gamers at large. Journalists explain and justify what they do not like in a game, and that is unvaluable.
The press is also better than the average game developer at detecting what it is that gamers truly want, which is always a good thing.
But, from an objective point of view, the scores that games receive are actually of no practical use.
other grading systems
I have read in a post-mortem (Freedom Fighters? Can’t find the damn Game Developer issue :( ) about the following evaluation system: after the development of a game, one was to consider: how much the development team liked the game, the relationship with the publisher, how well the game sold, what gamers thought of it and, optionally, the reaction of the press. This system is interesting as it allows to take a couple of steps back and instead of attempting to measure how well a game did, it focuses on how much it was liked, which is always a good idea. However, this is still a reactive, a posteriori, evaluation method. It is well suited for a post mortem exercise, as it highlights what could have been done better to improve one criterion or the other, which all have their importance in the final success of the game, but it cannot be used to detect whether the game will be good or not.