Saturday, May 30, 2009

Going all Bill James on Movies - CEVR

I love comparative statistics. Can't get enough of them. There's something about discovering how something is better than another thing that I just find fascinating. Maybe that's why I enjoy fantasy sports so much. But that's another story. Tonight (and tomorrow) I want to delve into something else that has statistics readily available online - movies. Friday's release and subsequent warm reception of the Sam Raimi-directed Drag Me to Hell is what got me really thinking specifically about the 3 main sites which track statistics on how good a movie is judged to be: Metacritic, Rotten Tomatoes, and IMDb.

Drag Me to Hell will certainly not be in the running for any major best-of awards at the end of the year - yet it is among the year's best-reviewed movies to date, holding a score of 82 on Metacritic, a 94% and "Fresh" rating on Rotten Tomatoes, and a user rating of 8.2 on IMDb. For those unfamiliar with how these sites arrive at these scores, here's a brief summary.

1) Metacritic takes the average numerical scores given to a movie in a review (and when no score is given, it's approximated)
2) Rotten Tomatoes decides if a review is merely positive and negative - the percent denotes how many reviews are positive
3) IMDb allows users to rate the movie themselves
*For more info, visit the respective websites*

So what do I find wrong - well, not wrong. Maybe lacking is a better word. Let's go with lacking. What do I find lacking in each of these sites' methods? The fact that none of them are comprehensive. Metacritic allows for the possibility of outliers to rule the score should not many reviews be given, Rotten Tomatoes doesn't take into account that a "positive" score makes a "eh, it was okay" review equatable to a "near-perfect movie" review, and IMDb leaves it too much up to people who more often than not aren't great critical evaluators of movies.

And yet - the people who watch movies more often than not are not looking to view great art. They just want to be entertained. It seems to me that a better way to measure fully how good a movie is is to take into account both its critical and entertainment value. So I've come up with a way to measure that - what I call CEVR, or Critical and Entertainment Value Rating. It takes the 3 scores from Metacritic, Rotten Tomatoes, and IMDb, adds them up, and then averages them to find what a movie's true rating is. Under this, Drag Me to Hell earns a CEVR of 86 - a good indicator of its average "entertainment" score, average "critical" score, and percent of positive reviews all in one fell swoop. Despite Drag Me to Hell not being a classic or an awards contender, it still is a movie that scores well with both critics and audiences - a surefire sign of a succesful movie. This algorithm then can be used to determine easier the answer to the endless debate - What is the #1 movie of all time? This is the question I'll take up tomorrow (well, later today), but will give you a little taste of right now. I've taken the top 13 (where a tie was broken on Metacritic, the first site I went to) movies listed on each site and made them into one uber-list. Granted, there were so many movies with 100% on Rotten Tomatoes that I would've needed a Top 100 to fit them all in - and thus, this list is not comprehensive. It's merely a demonstration and an experiment. So without further ado, the individual lists. I should note, however, that out of the 34 movies on the list, only 4 were repeated at least twice (The Godfather, Dr. Strangelove, Pulp Fiction, and Return of the King), and only The Godfather was on all 3. NOW without further ado, the lists:

Metacritic
The Godfather – 100
(T) Superman II – 99
Army of Shadows – 99
Pan’s Labyrinth – 98
4 Months, 3 Weeks and 2 Days – 97
(T) Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb – 96
Ratatouille – 96
(T) The Manchurian Candidate (1962) – 94
Pulp Fiction – 94
Killer of Sheep – 94
Spirited Away – 94
Lord of the Rings: The Return of the King – 94
Sideways – 94

Rotten Tomatoes
Man on Wire – 100%
Toy Story 2 – 100%
Taxi to the Dark Side – 100%
Bus 174 – 100%
A Hard Day’s Night – 100%
Deliver Us From Evil – 100%
The Wizard of Oz – 100%
The Godfather – 100%
The Taste of Others – 100%
Rear Window – 100%
The Third Man – 100%
Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb – 100%
The Sweet Hereafter – 100%

IMDb
(T) The Shawshank Redemption – 9.1
The Godfather - 9.1
The Godfather: Part II – 9.0
(T) Il buono, il brutto, il cattivo – 8.9
Pulp Fiction – 8.9
Schindler’s List – 8.9
(T) The Dark Knight – 8.8
One Flew Over the Cuckoo’s Nest – 8.8
12 Angry Men – 8.8
Star Wars: Episode V – The Empire Strikes Back – 8.8
Casablanca – 8.8
Star Wars – 8.8
The Lord of the Rings: The Return of the King – 8.8

No comments: