Kotaku’s Jason Schreier, whom I had the pleasure of meeting at last month’s PAX East convention, published his feature story about Metacritic today, and it’s well worth reading all the way through. In it, he speaks to a number of publishers and developers about how Metacritic is used from a game producer’s perspective, and weighs both the risks and rewards for an industry that’s become so heavily reliant upon the aggregation website.
I’ve questioned the need for a site such as Metacritic for a number of years, but I’ve never really dug into just how deeply it’s roots intertwine with that of the gaming industry. While the site collects aggregate scores for all types of media, video game publishers in particular have developed a near-obsession with the site over the past few years. At this point, not only has Metacritic become the standard by which a developer’s game will ultimately be judged, but publishers now use overall scores and leverage when negotiating post-release bonuses for developers.
What’s that, developers? You want more money? Well you better get really chummy with this logo…
In today’s industry, which is so focused on copycat formulas and generic multiplayer shooters, it’s hard enough pitching a unique idea. I’ve discussed before, to no small degree, how publishers have backed themselves into a corner this generation: First by getting into an arms race with competing publishers to see who can develop the biggest AAA spectacle, and then expecting that every gamer will purchase every game that’s released so that the multi-million dollar blockbusters can, at best, break even. It’s gotten to the point where the industry can barely keep itself afloat. Add on top of this, our industry’s reliance on Metacritic to determine the value of a particular development studio, and we have a recipe for total stagnation and unoriginality. Allow me to explain.
In Schreier’s article, he explains that when developers go in and pitch game ideas to publishers, a few things are considered before a final decision is made. The publishers want to know what the studio has previously worked on, what those games averaged on Metacritic, and how the developer’s themselves are ranked. These scores can either make or break a meeting, and there are more than a few cases resulting in publishers turning down a developer’s idea simply because their Metacritic score on the last game they worked on wasn’t impressive enough.
If Metacritic means anything, Gearbox and TimeGate Studios will never work in this town again!
Now, consider smaller development studios like thatgamecompany (makers of such titles as Flower and Journey) or Supergiant Games (creators of Bastion and the currently-in-development Transistor). Consider that the very pillars of their design philosophies are centered around creating out-of-the-box experiences with unique game mechanics; whose only goal is to make a quality game that, it just so happens, ends up finding a very niche audience. Smaller scale titles such as Flower may not get as much review coverage as a AAA title. Combine that with the fact that such a unique title will certainly only appeal to a select number of reviewers, and you have a nightmare scenario where one or two poor reviews from folks who “just didn’t get it” or feel that “it just wasn’t for me” can absolutely obliterate a Metacritic score. This creates a situation where a small studio looking for publisher backing so they can present their ideas on a larger scale is rejected because, based on their track record according to Metacritic, their games aren’t very popular or well received. In this example, publishers are fostering an environment were it’s becoming harder and harder to pitch creative (i.e. risky) ideas, because anyone who’s previously taken a chance and been burned by a Metacritic ranking will no longer want to risk both publisher support and their high-rank incentive bonuses.
Looking at it from the consumer side of the coin, I still don’t see much value in Metacritic. I can understand that some folks who don’t have time to read through or watch a video review might want to, at a glance, see how well a game is performing overall. But is that one simple aggregate score really enough to help someone determine whether or not a game is worth purchasing? Is a single number, by itself, all someone needs to make a purchase decision? It’s certainly not enough for me. Ideally, folks who seriously consider the value of a game before buying a brand-new release should read two or three reviews from critics they admire or respect before feeling confident they’re familiar enough with a title to decide one way or the other. Now, I would never tell you ladies and gents where you should be getting your reviews from, but I’ll admit that I have three people in particular whose opinions I trust above anyone else; and I often feel just a bit of validation when I’ve finished completing my own review and find that those folks had similar experiences to mine. If I’m being totally honest, I might even love it more when we totally disagree on a title, because those are the instances when I have the opportunity to see a different point of view and gain a bit more perspective. Either way, the point is that Metacritic shouldn’t be used as anything more than a quick reference to see how games are being received at launch.
But even then, the site runs into a few pitfalls of it’s own design, because as Schreier points out, Metacritic doesn’t take into consideration sites that use adjustable review scales. Instead, the first score a site posts is the one Metacritic uses, and if a game ends up getting a higher, or lower, score a few days later based on things like server stability or what-have-you, Metacritic won’t reflect those changes. Sticking with the example of a consumer who only has time to glance at a Metacritic score before deciding whether or not to pick up a game, a title like SimCity would’ve seemed like a sure-fire purchase at launch. Yet, once the game came out and was plagued by server issues, resulting on some sites such as Polygon drastically lowering the game’s final review score, Metacritic’s refusal to update new scores would lead that consumer to purchase a game that was completely unplayable. So, obviously, the system isn’t exactly ideal.
Oh boy! I can’t wait to play my single-player, umm, wait, what’s this?
At the end of the day, it’s always a matter of perspective. If a developer chooses to use Metacritic scores as motivation to create the best game possible, who am I to say that’s a bad idea? But when it becomes a matter of money; when it gets to the point where a review score could either help pay off a family’s mortgage or keep food off the table, now there’s a serious problem. One that potentially stifles creativity while simultaneously giving publishers a bit too much leverage over developers (as if they didn’t have enough already.) Ultimately, Metacritic seems to be doing more harm than good. To be clear, it’s not Metacritic’s fault; the site is simply doing a job. The folks that are to blame are whoever is saying that Metacritic is the be-all and end-all determinate of quality development. I’m not saying we need to get rid of it, but I do believe we need to stop taking it so seriously. Giving one aggregation site so much power can never be a good idea.
Do you guys and gals have any thoughts on Metacritic? Do any of you frequent the site? What would you say to publishers who place such a high premium on top-rated Metacritic scores? let us know in the comments below.
