Why I stopped scoring shows, and why I decided to start again

Outside of the realms of scoring a test, where there are predetermined correct or incorrect answers, or criterion to grade a piece of work, scoring can be a veritable minefield. There seems to be a lot of different opinions on how we should value or compare a piece of work to other similar pieces of work, whether this is books, films, games, or anime.

I think that when it comes to rating things online, there seems to be a ‘more is more’ type philosophy to giving scores, where in spite of many sites using a 1-10 scale (or 5 stars with half stars which is still 1-10 in increments), it is rare to see scores below a 7, with actively bad titles perhaps receiving as low as a 5. There are exceptions to this rule, but as someone who is mathematically minded, this has in the past been a huge source of frustration to me. My rationale being ‘why would you only use 3 points to show how above average something is when you leave 6 points to show how below average a show is?’. I suppose that this is different from actual tests where while 50% may be a pass still means that you have a lot of room for improvement, which is perhaps the mindset that people have when assigning scores. Or they merely have the thought that higher numbers are inherently more appealing which can ring rather hollow when all the numbers are high. Scoring media is not a clicker game (it’s rather off topic for this blog, but I love clicker games, they tap into a part of me which feels satisfied merely by my having minimal agency over the numbers going up). There are no predefined criteria for rating anime, and while it’s possible to break something down in terms of its technical features such as visuals, sound or story, they’re still mostly rated on a comparative basis (is what you’ve seen better or worse than other things you’ve seen, or did you like it more or less than other things you’ve seen?).

I enjoy using my MAL as a way of tracking what I’ve seen, and also for noting shows I may like to watch in the future which others have suggested to me or have caught my interest, but for some time as I didn’t feel right about the prevailing scoring system, I decided to not score my shows. Besides, I felt that the numbers were meaningless alone. A number by itself tells you nothing about how someone perceived a show or whether it was good or not. There are people who rate on enjoyment alone, and those who rate based on technical aspects, and those who may use a combination of the two factors, and so this makes numbers even more confusing! I felt that my numbers were meaningless in this inflated system, and meaningless by themselves anyway.

What I have come to realise over time though is that while a number is useless on its own, it can potentially be useful in combination with other numbers. A score, whether you give an average show a 7 or a 3 is still useful when with a lot of scores from the same person. You can see which shows they preferred when compared to other shows, as a show with a higher score was obviously either more enjoyable or technically better than a show with a lower score in their eyes. By looking at the scores as a collection, you can get a feeling for a person’s taste, even if it is useless to compare the numbers alone to other’s scores. In addition to this, if you remember that MAL has a very high average score, it’s quite easy to see what the public opinion considers to be good or poor, by remembering that 7.00 is basically an average show. I can’t help but feel that perhaps this is still wasting the better part of the lower half of the 1-10 scoring system, but people don’t need to score in line with my ideals in order for the information to be useful.

The other thing which I’ve come to realise is that it’s been quite a long time since I’ve watched some anime, and the details are not as clear in my head as I would like. This is perhaps only reasonable as it’s been over 10 years since I’ve watched some titles. I believe that that at least giving a number to compare the shows to other shows which I have seen will help me remember my feelings on this particular title, as I’m sure going forward I am going to watch even more anime, and while I have revisited certain titles and there are other titles which I would like to revisit in the future, there are still titles which I wouldn’t plan to rewatch but I can have some sort easy prompt about my feelings on the anime. Scores can act as a sort of revision flashcard for me in order to help conjure up personal memories.

Generally, I’ve eased up a bit on my feeling that ‘someone is wrong on the internet’, and have come to realise that in spite of the limitations of assigning a single number to a show, it is of some benefit, especially on a personal level. People are free to score how they wish, and this includes myself.

What does this mean for me? I’m still not going to give scores on my blog accompanying reviews, as I think that the words I write are far more informative than boiling my opinion down to a single number. I’ll stick with making recommendations as to what sort of person I think may enjoy a show, or if it’s a show that I feel that no-one would enjoy. However, I have decided to start scoring shows on my MAL in order to provide a basis of comparison primarily for myself, and perhaps other people.

I personally think that it’s most meaningful to score titles on a comparative basis, and think that ideally the distribution of scores should closely approach a ‘Student’s T-distribution’ (like a normal distribution or bell curve which perhaps people may be more familiar with, but with more open ends). When scoring shows, I do consider it to be possible to get scores in the extreme range (although I’ve never sought out or found a show I’d consider to be a 1 in that it is unreservedly poor in every aspect), which a true normal distribution would effectively preclude with a 1-10 scale.

t-distribution for reference

I consider that most titles will be fairly average (in the 4-6 range), and fewer titles will earn more extreme scores as being exceptionally poor or excellent. I also think that there will be a slight skew towards higher scores, as realistically, I personally and people in general are less likely to select titles which they are not interested in or actively bad, and if they do find them are less likely to actually complete them. Unless you were completing some sort of challenge where you just selected random titles from a large database (like putting Crunchyroll on random), it’s normal to want to watch things which seem like they will suit your interests and are well regarded. I do think with watching more seasonal anime and being uncertain of the outcome or overall opinion of a particular show when starting to watch, my skew may even out slightly over time, but I’d still expect to have a mean score closer to 6 than to 5.

As such, shows which I score a 5 I would consider to be of similar quality and enjoyment to most other shows I have watched. Shows which I score a 4 I would consider to be slightly worse than most shows I’ve watched, shows which I score a 6 I would consider to be slightly better than most shows I have watched and so on. It’s not a perfect system, and I may find the need to revise my ideas in the future, but I’m happy to have a system. You don’t need to have this system, but as a mathematically minded person it feels good to create a rationale behind the numbers I assign to a particular show.

Scores are always up for review, and if I rewatch a show or I view a lot more of a particular genre and my opinion of a particular show changes, I’m quite happy to revise the numbers.

As readers, do you like to score shows you watch, or would you rather not assign a number to your experiences? If you score, do you have a particular schema in mind, or have you not given the whole thing too much thought?