Thursday, January 26, 2023

The subjectivity of star ratings

What do you consider to be a five-star book (on a rating of 1-5 stars)? Or a one star book?

Is the former a book that you couldn’t put down? Or that covers its subject comprehensively and in impeccable detail? Or that is an acknowledged classic (but you didn’t necessarily enjoy)?

Is the latter a book that is badly written? Or maybe you could not finish? Or is it in a genre in which you have zero interest? Or, perhaps its even an acknowledged classic that you could not comprehend, and threw down in frustration.

It could be any one of these, which is why star ratings are so subjective as to be almost useless.

I used to rate books on a scale of 1-5 stars but eventually stopped the practice because I didn’t have a firm set of criteria that measured quality. For example, I once thought only a handful of the very best all-time books (The Lord of the Rings, Watership Down) etc. could earn a 5-star rating, which meant that other awesome books or stories could only earn a lesser rating. In short, I was grading on a curve. But this doesn’t really work; how can you possibly compare The Rise of the Fall of the Third Reich (a 5-star history if there ever was one) to The Lord of the Rings. If I found the latter more enjoyable to read, can the former only ever a achieve a 4 or 4.5 rating? Kind of nonsense.

So, I abandoned star ratings.

And yet, for some unexplainable reason I brought back a star rating for my recent review of Blood of the Serpent. But as I evaluate my 3.5 stars for that book, I realized my rating was not based on any objective measure of quality, but purely on whether the book met my expectations for a prose relaunch of Conan. It didn’t quite, hence the 3.5. But you might find it did for you, and give it a 5 (or didn’t at all, and rate it a 2).

If you look at Amazon or Goodreads reviews you will find that they do not correlate highly with quality, but what I do think they represent is the expectations the reader brings to the book.

For example, Moby Dick rates at a 4.4 while the Da Vinci Code is a 4.6. Does that make the latter better? Perhaps, but it probably means some bored high school students took out their frustration on the former (they wanted Jaws or The Meg, but their expectations were unmet). Poul Anderson’s The Broken Sword rates a 4.2, below the sixth volume of 50 Shades of Gray (which rates a whopping 4.8/5). But most likely that’s because its readers got the S&M they wanted in 50 Shades of Gray, as opposed to readers who came to The Broken Sword expecting Brandon Sanderson length epic fantasy. And rated it lower. 

This is a roundabout way of saying that thoughtful individual reviews are superior than the aggregate of a thousand star ratings. 


Matthew said...

I once ran across a low review of Day of the Jackal because of some very minor error Forsythe made. I actually went to the lady's reviews on Amazon all she seemed to like was books on chess.

Reviewing things is hard because it is so complicated. I don't know if it is really subjective but there are a lot of things to consider. Jackal for example has no real characterization but it has a thrilling plot. So you have to take things on Amazon with a grain of salt.

Paul R. McNamee said...


I've started using the star rating as PERSONAL evaluation. For instance, if you notice at goodreads, the start ratings from 1 to 5 are "didn't like it," "it was okay," "liked it," "really liked it," and "it was amazing"!

I like that, for me. But as a general aggregation it's awful. It is all about personal taste.

I mean, THE GREAT GATSBY maybe from a technical critique should get five stars. But I don't really care for that story, so for me it's a low ranking.

Matthew said...

Well, Gatsby was a book with pretty prose but the story is not that great. How many people have you known who created a fake identity to get back with a girl he dated like once or so? People don't do that. There are other problems too.

I've been thinking to day about the differences in taste. Largely, because I got a rejection for a submission. The editor's problem was that I don't put enough about the main characters thoughts and emotions in the story. I, at least, tried to imply emotion through the actions and speech. (Whether I succeeded is debatable.) A lot of stories that tell you the emotions and thoughts seem to be too on the nose and kind of annoying to me. I'm not sure either my view or the editor's is "right" or "wrong."

John said...

Yes, whether based on the reviewer's knowledge, education or erudition, reviews are ultimately at the mercy of subjectivity. I've watched 4-star-rated movies on Prime that I thought should have had half of those, etc. I think it's best just to read what the reviewer has to say and then you can see what they're basing their opinion on, whether it's movies books or other subjects. Basing your decisions on star ratings, especially when there are lots of reviews, can be a pitfall and end up a waste of time.