Why I stopped scoring shows, and why I decided to start again

Outside of the realms of scoring a test, where there are predetermined correct or incorrect answers, or criterion to grade a piece of work, scoring can be a veritable minefield. There seems to be a lot of different opinions on how we should value or compare a piece of work to other similar pieces of work, whether this is books, films, games, or anime.

azumangadaiou10I think that when it comes to rating things online, there seems to be a ‘more is more’ type philosophy to giving scores, where in spite of many sites using a 1-10 scale (or 5 stars with half stars which is still 1-10 in increments), it is rare to see scores below a 7, with actively bad titles perhaps receiving as low as a 5. There are exceptions to this rule, but as someone who is mathematically minded, this has in the past been a huge source of frustration to me. My rationale being ‘why would you only use 3 points to show how above average something is when you leave 6 points to show how below average a show is?’. I suppose that this is different from actual tests where while 50% may be a pass still means that you have a lot of room for improvement, which is perhaps the mindset that people have when assigning scores. Or they merely have the thought that higher numbers are inherently more appealing which can ring rather hollow when all the numbers are high. Scoring media is not a clicker game (it’s rather off topic for this blog, but I love clicker games, they tap into a part of me which feels satisfied merely by my having minimal agency over the numbers going up). There are no predefined criteria for rating anime, and while it’s possible to break something down in terms of its technical features such as visuals, sound or story, they’re still mostly rated on a comparative basis (is what you’ve seen better or worse than other things you’ve seen, or did you like it more or less than other things you’ve seen?).

I enjoy using my MAL as a way of tracking what I’ve seen, and also for noting shows I may like to watch in the future which others have suggested to me or have caught my interest, but for some time as I didn’t feel right about the prevailing scoring system, I decided to not score my shows. Besides, I felt that the numbers were meaningless alone. A number by itself tells you nothing about how someone perceived a show or whether it was good or not. There are people who rate on enjoyment alone, and those who rate based on technical aspects, and those who may use a combination of the two factors, and so this makes numbers even more confusing! I felt that my numbers were meaningless in this inflated system, and meaningless by themselves anyway.

azumangadaiou14What I have come to realise over time though is that while a number is useless on its own, it can potentially be useful in combination with other numbers. A score, whether you give an average show a 7 or a 3 is still useful when with a lot of scores from the same person. You can see which shows they preferred when compared to other shows, as a show with a higher score was obviously either more enjoyable or technically better than a show with a lower score in their eyes. By looking at the scores as a collection, you can get a feeling for a person’s taste, even if it is useless to compare the numbers alone to other’s scores. In addition to this, if you remember that MAL has a very high average score, it’s quite easy to see what the public opinion considers to be good or poor, by remembering that 7.00 is basically an average show. I can’t help but feel that perhaps this is still wasting the better part of the lower half of the 1-10 scoring system, but people don’t need to score in line with my ideals in order for the information to be useful.

The other thing which I’ve come to realise is that it’s been quite a long time since I’ve watched some anime, and the details are not as clear in my head as I would like. This is perhaps only reasonable as it’s been over 10 years since I’ve watched some titles. I believe that that at least giving a number to compare the shows to other shows which I have seen will help me remember my feelings on this particular title, as I’m sure going forward I am going to watch even more anime, and while I have revisited certain titles and there are other titles which I would like to revisit in the future, there are still titles which I wouldn’t plan to rewatch but I can have some sort easy prompt about my feelings on the anime. Scores can act as a sort of revision flashcard for me in order to help conjure up personal memories.

Generally, I’ve eased up a bit on my feeling that ‘someone is wrong on the internet’, and have come to realise that in spite of the limitations of assigning a single number to a show, it is of some benefit, especially on a personal level. People are free to score how they wish, and this includes myself.

What does this mean for me? I’m still not going to give scores on my blog accompanying reviews, as I think that the words I write are far more informative than boiling my opinion down to a single number. I’ll stick with making recommendations as to what sort of person I think may enjoy a show, or if it’s a show that I feel that no-one would enjoy. However, I have decided to start scoring shows on my MAL in order to provide a basis of comparison primarily for myself, and perhaps other people.

I personally think that it’s most meaningful to score titles on a comparative basis, and think that ideally the distribution of scores should closely approach a ‘Student’s T-distribution’ (like a normal distribution or bell curve which perhaps people may be more familiar with, but with more open ends). When scoring shows, I do consider it to be possible to get scores in the extreme range (although I’ve never sought out or found a show I’d consider to be a 1 in that it is unreservedly poor in every aspect), which a true normal distribution would effectively preclude with a 1-10 scale.

325px-student_t_pdf-svg

t-distribution for reference

I consider that most titles will be fairly average (in the 4-6 range), and fewer titles will earn more extreme scores as being exceptionally poor or excellent. I also think that there will be a slight skew towards higher scores, as realistically, I personally and people in general are less likely to select titles which they are not interested in or actively bad, and if they do find them are less likely to actually complete them. Unless you were completing some sort of challenge where you just selected random titles from a large database (like putting Crunchyroll on random), it’s normal to want to watch things which seem like they will suit your interests and are well regarded. I do think with watching more seasonal anime and being uncertain of the outcome or overall opinion of a particular show when starting to watch, my skew may even out slightly over time, but I’d still expect to have a mean score closer to 6 than to 5.

As such, shows which I score a 5 I would consider to be of similar quality and enjoyment to most other shows I have watched. Shows which I score a 4 I would consider to be slightly worse than most shows I’ve watched, shows which I score a 6 I would consider to be slightly better than most shows I have watched and so on. It’s not a perfect system, and I may find the need to revise my ideas in the future, but I’m happy to have a system. You don’t need to have this system, but as a mathematically minded person it feels good to create a rationale behind the numbers I assign to a particular show.

AzumangaDaiou15.jpgScores are always up for review, and if I rewatch a show or I view a lot more of a particular genre and my opinion of a particular show changes, I’m quite happy to revise the numbers.

As readers, do you like to score shows you watch, or would you rather not assign a number to your experiences? If you score, do you have a particular schema in mind, or have you not given the whole thing too much thought?

Advertisements

16 thoughts on “Why I stopped scoring shows, and why I decided to start again

  1. Great read into the context of your experiences. I’m very comfortable with rating things from a more objective scale than personal enjoyment, but nothing is exactly set in stone as to what each number means. Sometimes a 5 can be enjoyable, but had a lot of nagging issues that I couldn’t stand with it. Alternatively, a 7 can be objectively good, but personally dry. Somewhat like my recent viewing of Emma, which I gave a 6 because it had a nice atmosphere, relatable(ish) characters, and decent attempts at creating an immersive story and setting. However, I was more than often a little bored while watching, and was distracted from time to time by other things. It really all depends on the circumstances, which I hope to elaborate on with my blog posts.

    I look forward to seeing more of these kinds of posts from you.

    Liked by 1 person

    • Thank you! Ideas-based pieces like this are somewhat more challenging to write as it takes more time for me to collect and refine ideas on an interesting topic than to just be able to speak about what happened in a particular series or episode, but I certainly plan to do more of them!

      There’s definitely no set-in-stone criteria for getting a particular number for me either. Some shows which are a 4 for instance are highly personally enjoyable but I recognise that they are technically poor (issues like frequent off-model animation or major plot holes), but others may be more technically sound, but just didn’t have the capacity to hold my interest. I think that words are really necessary to elucidate a show’s strengths or weaknesses to others, but I get the feeling that you, like me, use the score as personal comparison or reference for the shows you’ve seen.

      Liked by 1 person

  2. I stopped scoring anime on Mal as people just shrug over certain shows if it’s low. I use mal to mainly keep track of where I am and use for blog referencing. I agree that there’s nothing wrong with scoring shows I Use to base mine on enjoyment factor I gave to many tens as I enjoy so many but after that phase became a bit more aware of my enjoyment level of the show. This was a refreshing read honestly. Especially if there have been shows you haven’t watched. Revisiting them might make you score it different. I’d still rather write out my opinion on a series than score it.

    Liked by 1 person

    • I know that the community opinion on MAL is a bit weird. I hope to not let it get me down though! We haven’t been given specific rating criterion in advance, so it’s not like you’re ‘doing it wrong’ regardless of what score you give a show. It’s not like people are giving different marks to an identical maths test.

      I actually think that if you score on enjoyment, it makes sense that the first few things you watch will have very high scores. I experienced, and I think it’s common to experience a feeling of fresh uniqueness with the first few anime series I watched. I suspect this is for two reasons; firstly because at first you usually take recommendations from others who will pick the best of the best or titles which they think will strongly appeal to your tastes, and once you start branching out and trying titles yourself, they don’t come with this sort of pedigree. Secondly, when you’ve seen very little of anime in general or a particular genre, everything seems very fresh, new and exciting in a way that series may not after you’ve seen a lot. It doesn’t mean that you won’t still find things you like (I’ve watched old and newly aired series which have become personal favourites in the last 12 months in spite of having been watching anime for over 10 years now), but the feeling of novelty has definitely worn off somewhat.

      Scoring is very much something just for me! I’m still planning to write about the things I watch and share them on my blog, as I think that words are far more meaningful than a single number when conveying information about a show to others.

      Liked by 1 person

  3. I don’t score and I don’t really pay attention to the scores other people give. I read their impressions and reasoning because I feel that is more informative than a numerical value on a subjective experience.

    Liked by 1 person

  4. I am always willing to give ratings to titles- these are determined by a mix between the enjoyment factor and the objective issues. If I enjoy an anime, it automatically gets a 4/10- then I start building up the score on animation, art, plot, etc. I don’t score comparatively because I just haven’t watched enough titles and also that there will times where I have to compare between two shows of different genres.
    Anyways, nice post.

    Liked by 1 person

    • I’d never thought of starting off from a base score and adding points as wanted from different categories, but it’s an interesting system! There’s so many different ways to score shows out there.

      I’m not sure how much anime you’ve watched, but I’ve suddenly got a different idea of ‘a lot of anime’ after getting into the ani-blogging community and seeing people who have over 1,000 watched titles. Some people have been watching a lot of anime for a long time! We’re all anime fans though whether you’ve watched less than 10 titles or over 1,000 if you’re someone who enjoys anime.

      Thanks for sharing your thoughts!

      Liked by 1 person

    • I couldn’t find a neat way of doing it (other than just superimposing a t-distribution with a sensible amount of degrees of freedom on my Graph Anime graph in GIMP or something, which I didn’t think really counted). Maybe there’s a way to export the stats as a .csv or something useful? I didn’t delve deeply enough into this to see if it was viable.

      At present, I can tell from the shape of my distribution that I do have a skew towards higher scores, I think both because of the natural skew which I mentioned in my article, and also because in the big scheme of things, I don’t have that much data. I’m at slightly under 200 completed entries, and some of these are films/OVAs which I don’t remember at all, and thus did not score. Maybe it would be interesting (at least on a personal level, this isn’t a maths blog so I may hold off on sharing it with the world except perhaps as an inclusion in a different article with other substance) to compare my stats now with my stats in 12 months or so. I know with choosing to pick up more shows as they’re airing, I’m going to end up watching more duds, or things which I personally don’t enjoy as much.

      Liked by 1 person

      • You can play a bit with Excel, having the data from GraphAnime and making a corresponding graph for the T-distribution.
        Yeah, a skew is expected, but I don’t think that ~200 entries is a small population to deal with. I think you’d need to watch a ton of random anime to reverse it.
        True, digging too much into too technical and personal stuff might not make the most widely appreciated posts, but from time to time at least for me it’d be interesting. Even for the sake of mixing two subjects that look completely unrelated – math and anime.

        Liked by 1 person

        • I’m pretty good at picking shows I like. Even with this new season coming out, the shows I earmarked as being interested in, I like them, and that’s without being someone who checks out source material. I have to accept that my ability to find anime that I like and enjoy will affect my pursuit of ‘ideal’ statistics.

          I may see if I can get some things to work in Excel, even if it’s just for personal enjoyment! It would be interesting to see how closely my current stats fit the distribution. It’s nice to talk to someone else who is obviously interested in maths and anime.

          Liked by 1 person

          • Well, it just may be that your distribution is identical to the theoretical, just the average value is not 5. It’s actually quite praiseworthy that you are able to guess which shows you’ll like. The amount of time saved and energy wasted watching something you don’t like should be more than enough to compensate for the lack of perfectness in the statistical department.
            I wouldn’t call myself a tough maths geek, but maths certainly has some neat and interesting things to offer which I usually don’t shy away from.

            Liked by 1 person

  5. This is not too far off how I feel about scores. They are really only important to the person giving them, although you can still get a general read of someone else’s tastes by skimming over their scores. Using the whole rating scale is not essential for most people but I feel you can distinguish titles from one another much better if you do.

    Funnily enough, while my scores do tend to average out in the T-shape I haven’t actually designed it that comparatively. Five is basically a rough baseline that then fluctuates over the course of a series until I settle on a final score in the end. Typically this gets me thinking a lot more about the show when I have to justify lowering/raising the score on an episode by episode basis. In the past I found it too easy to reach the end of a show or film and then slap a score on it for future reference without much thought. Nowadays I don’t make it that simple, which is probably what makes extreme scores so rare for me, since I have to actually rationalize each score change I make.

    Numbers still won’t be relevant when going for in-depth discussion but as a personal system it works well enough.

    Good post!

    Liked by 1 person

    • Your article as part of your 12 days of aniblogging was definitely part of the reference material I used when putting this together. It was really interesting!

      I think that final episodes definitely can leave a strong impression and affect how you may rate a show overall, and it’s interesting that you’ve created a system to counteract that to some extent. I figure that the final impression is going to shape your final thoughts, but at the same time, I’m not really someone to rage rate a show (or other piece of media), and also like a lot of fluffy comedy where there’s not even a decisive ending but just a simple closed door on the characters (or a door left open for a possible additional season), so perhaps this isn’t as much of a concern as it would be for people who enjoy things like thrillers, the horror genre, or other genres where a satisfying conclusion really matters to have a satisfying show. I’d never really seen the benefits of rating a show mid-watch previously, but you’ve made a compelling point regarding some possible benefits. Thanks for sharing!

      Liked by 1 person

  6. I have a 5 star rating system that I use. I save the 1 star for things I can’t finish, if I have to stop reading a book or stop watching a series or episode then that’s when I use the one. If I finish it but really don’t like it I give it 2 and try to provide the reasons why. 3 is average, it means I enjoyed it but I have reasons why it couldn’t be higher. 4 means I would recommend it but I have an issue or two. 5 means perfect, I’d recommend it and cannot give it any major faults. I don’t do half stars.

    I am scoring episodes as I watch them this season (It’s the first season I’ve watched anime) and my intention is for the seasons I episode review to have an overall season score that will take into account each of the individual scores. For things that I watch or read in my own time and don’t think about in terms of score after it’s finished I just have a text review. I want to try and avoid my season reviews being ‘tainted’ by a good finish if it had a very bad start (or vice versa) and unless I’m judging it week by week I can’t be sure that’s what I’m doing.

    I find rating systems help me align what I like with what the reviewer likes. If something I like is receiving low score and things I don’t like are being praised then I know that, from that reviewer, I can dismiss anything they rave about. Similarly that’s why I score my own reviews so those that do find my blog can maybe find new things if they like similar shows.

    Liked by 1 person

    • I think that you’re right in that having scores can help you compare your taste to another person (although sometimes just comparing someone’s feelings on a thing to your own can be helpful), as people’s tastes can differ! There’s a few people who I know with blogs whose taste in anime differs from mine to the point where certain types of shows they describe as boring or poor I know I’ll enjoy, or other shows which they describe as enjoyable, I know that I’d find uninteresting. This isn’t a bad thing, it’s just something to bear in mind.

      Judging week by week will help you to hopefully give a show a more balanced score, although I’m at least partially of the opinion that the first episodes are what are designed to hook the viewer in, and the final episodes are the ones which are meant to leave a strong impression, and for people who are not approaching anime solely from the purpose of criticism, this is what they’ll think about. Of course, I do think some people can go a bit far in this direction and ‘rage score’ a series if the ending wasn’t to their taste, and your method would definitely avoid that particular pitfall.

      Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s