The Problem with Rating Systems
Countless readers and writers love rating systems. They are a simple way to mark a film in terms of quality and worth, require little effort, and put forward a simple message that anyone can understand. While there are many writers who put emphasis on the actual review rather than the rating, too many place importance on their final stamp. This has led to a culture in which audiences use a simple number or metric as a way of assigning value to a film. Many will assume a film is better or worse than another film based on the rating, and even worse, many will jump to conclusions regarding the quality of a film having only seen the rating. Ratings tell you very little about a film’s quality, and more critics should take care in how they craft and use their own rating systems.
A Simple Experiment
Many years ago, I crafted an experiment to see if rating systems had some unclear coordination. The experiment works like this: take any two rating systems, convert them into percentages, convert them into letter grades, and then convert them into integers on a 1-10 scale. Once you have done that, check to see if each rating is equivalent.
I will use the grade level system, and the star system for this example.
A typical grade level system has thirteen different ratings, three ratings per letter for letters D-A and one rating for the letter F. A star system has either five different ratings, one for each star, or ten different ratings if the system uses half of a star. For the sake of simplicity, I will use the former.
I will then take the two grades from each system that wind up exactly in the middle. Since the grade level system has thirteen different ratings, the middle of the system is around a C+. The middle of the star system is 3/5 stars.
A C+ in percent form in around 78%, while 3/5 stars is sitting at 60%. In letter grades, a C+ reads as a C+ while a 3/5 stars reads as a D-. On a 1-10 scale a C+ is a 7.8/10 while a 3/5 stars is a 6/10.
Yet on its own, a 3/5 star rating might make the reader believe the film is worth seeing while a C+ might be seen as a subpar experience. This is because the rating systems have been arbitrarily made up by independent people, and since a reader might be using many reviews as a guideline, these ratings can lead to fuzzy understandings.
The Real World
Let’s now take this out of the theoretical and into the real world.
Take IGN for example, they say that an 8/10 score is ‘great’. But convert that into the grade level system, and you wind up with a B-, which is the cutoff between a positive and negative review for popular YouTube critic Chris Stuckmann. The cutoff between a positive and negative review on Rotten Tomatoes is 60%, which translates to a D on the grade level system.
When it comes down to it, each individual rating, even to people who use the same system, has a different meaning to everyone. Some people have written reviews with the final grade of ‘C’ that were fairly positive, while others have written reviews with an 8/10 stating the film was just okay.
Since we are not using the same grading system as if we were in school, not only does each rating have a different meaning to each critic, but it has a different meaning to each reader. Two people who see 3/5 stars from the same critic might walk away with completely different impressions. One might think “That sounds good, I think I’ll check that out.” while the other might think “3/5? That’s like a D, I’ll skip it.”
When you get right down to it, any given rating doesn’t mean anything on its own. 8/10 has different meanings to everyone who sees it, it is almost subjective, and all of this gets really interesting when you look at one of the largest rating systems on the internet: Rotten Tomatoes.
First, I want to dispel some common misconceptions regarding Rotten Tomatoes. The percent that their site displays is not the average score that critics and audiences gave the movie, instead it is the percent of critics and audiences that gave it a positive review. For example, if every single critic gives a film a 6/10 the film would display a critics score of 100%, with the average review score in a small font below it.
While it may be tempting to raise your fists in fury against Rotten Tomatoes upon this discovery, let’s sit down for a minute and think about what we have just discussed.
Every review score is essentially meaningless on its own, because very few rating systems have concrete values and definitions for each rating. The only thing most review scores can tell you is whether the film was good or bad (and this differs also, especially on the grade letter system), so why should we expect Rotten Tomatoes to treat them any differently?
The average review score that is posted below the overall percent is meaningless as well, because not every critic has a rating system, and each rating can differ when being placed on a 1-10 scale.
Essentially, the over-reliance of both readers and writers on loose rating systems has led to a tangled web of fundamentally fuzzy perceptions. What this translates to is sites like Rotten Tomatoes doing their best to synthesize every major review on the internet, but because ratings are not always specifically defined, they provide, at best, a vague understanding of a film’s quality.
So, what is the solution to this problem?
As big of a problem as rating systems can be at times, the problem they have sparked has a refreshingly simple solution: read the review. Find a group of critics or movie fans on the internet who you usually agree with, and read or listen to their reviews. Hear the weaknesses and the strengths of the film, and think to yourself: “does this sound like something I would like to see?”
Using sites like Rotten Tomatoes is fine, so long as you understand that it doesn’t provide a definite, synthesized score based on the reviews of critics and audiences.
We may all fall victim to laziness and want to see the rating and then walk away, but I encourage you to fully understand the writer’s perspective and opinion, and not just look at their stamp.
Critics, on the other hand, should take more care in how they define their rating system. Don’t just say something like “5/7 galo-goobs” without providing a concrete explanation as to what that means. I would also advise that critics pick a rating system that they actually understand. I once used the 1-10 scale system to rate films before I began writing here, but I often found myself arbitrarily selecting a rating, while using another film as a vague guideline. If you are going to use a rating system, make sure you know what each possible rating says about a film.
With these two solutions in mind, readers and writers will have a more solid understanding of quality.
End Note: If you want to fully understand my rating system, go to the About Me page.