Proposal: 5★ Ratings and Reviews

RateBeer Feedback
1

Legend: Proposed versions of 5★ Scores

  • Average Rating - Flat average of both ratings (ticks) and reviews.

  • Review Average - Flat average of reviews only.

  • Weighted Average

Intro

Thanks for being patient with this topic everyone. We’ve taken everyone’s feedback onboard and have aimed to come up with a way forward that addressed not only the areas of concern, but also opportunities that we could look into. Our main goals when approaching this were to:

  • Retain the Overall Score and Style Score.
    These remain as they are, there are NO changes being made to the algorithm here.

  • Surface our users 5★ ratings (Currently known as Ticks).
    The ratings will now be visible by the public.†

  • Phase out the use of the word ‘tick’.
    Our users should be able to easily give our beers a 5★ rating and thus be able to reference it as such.

  • Appreciate the score weighting but simplify the visual application.
    It’s clear that there isn’t a definitive consensus on what the current 5★ rating is showing or should be showing. We felt the best way forward is to RETAIN all attributes but show one by DEFAULT. Our preference is to show the ‘Average Rating’ as it is easier to understand, matches new user expectations and highlights all contributions.

  • Phase out the conception of ‘private reviews’.
    This was an incorrect use of the term ‘private’. The mechanics behind this ARE NOT changing, but the way in which we talk about it should. We should now say something more along the lines of ‘Your description/review of this beer will only be visible once you have written 75 or more characters’.

Here are some proposed examples on how these changes will take effect.

On the Beer Search Result screen, users will see:
Overall Score and Style Score. And now the ‘Average Rating and the total ratings given’ vs ‘Weighted Average and total reviews given’.

On the Beer Profile Screen, we have exploded the scoring module to now show more detail. Users will now see:
Overall Score and Style Score. And now the ‘Average Rating and both the total ratings/reviews given’ vs ‘Weighted Average and total reviews given’.

RatingsConcept1%402x
RatingsConcept1%402x1224×816 156 KB

Improving the way we read the scores beers are given.

Previously the intended way to read the scores was:
AleSmith Speedway Stout has an Overall Score: 100, Style Score: 99 and a Weighted Average of 4.29 based on 3,177 reviews.

A lot of new users were unfortunately reading it incorrectly as:
AleSmith Speedway Stout has an Overall Score: 100, Style Score: 99 and an Average Rating of 4.29 based on 3,177 ratings.

Now we’ve hopefully simplified this and enabled it to be read in this way.
AleSmith Speedway Stout has an Overall Score: 100, Style Score: 99 and an Average Rating of 4.32 based on 3,972 ratings. Out of those 3,972 ratings, 3,177 people have also written a review.

Ensuring we retain existing scores and wrapping some informative content around it.

You will see in the design that the users now also have 2x new call to actions in the scoring module.

  1. They can read our ‘What do these scores mean’ FAQ.

  2. They can pull up more in-depth statistics (where the Review Average is now stored along with Weighted Average).

RatingsConcept2%402x
RatingsConcept2%402x1224×816 162 KB

How ratings and reviews are shown.

By default we will only continue to show reviews. Ratings without any descriptions will ONLY be surfaced to a user once they decisively toggle it on through a filter. It is important to note that we are relying on our filters to gradually improve over time to give our users individual and customised control over how they wish to visualise and consume the content that is most important/relevant to them. This is different for everyone and the flexibility of filters enable us to provide a better experience.

† To finish off, we wanted to re-iterate that an ‘incomplete’ rating/review will have its’ description hidden if it does not have 75 or more characters. This is the current behavior and is NOT changing.

RatingsConcept3%402x
RatingsConcept3%402x1224×816 166 KB

4 Likes
2

Reserving post here.

3

Right now, I’m using 5/5 ratings private scores as a beer cellar option since it’s been missing (the legacy option is still available, but there’s no way to see if a beer is in our cellar at first sight, like if we already rated a beer ie). I know l other users who do the same. Could you do something about it?

(at the same time, a WHAT THESE SCORES MEAN is needed for Places too)

“We felt the best way forward is to RETAIN all attributes but show one by DEFAULT . Our preference is to show the ‘ Average Rating ’ as it is easier to understand, matches new user expectations and highlights all contributions.”

What do you mean about that? We won’t be able to see others Attribute ratings?

@aww

5

This seems like an obvious concern. There are a lot of beers that will show up as 1* average rating because the only rating on the site is fonefan saying that something is in the backlog.

Why does Speedway have a weighted average of 4.29 but an average review of 4.30? It has 3177 full reviews, so surely the Bayesian weighting shouldn’t have any impact. Is there an error in one of the algorithms? Or is this some mad coincidence where the weighted average is 4.29499… and the real average is 4.2950001… and so the tiny impact of the Bayesian weighting just happens to look important.

Similarly, why is the style score of Speedway only 99? It’s the 9th highest rated beer in the style. Do we really only have 900 of them so that it’s not in the top 1% (I guess that could be 1800 and 0.5%, I’m not actually sure where the rounding to 100% is)? Given that I have almost 200 rates of the style it seems that there should be thousands of them. Is there an error in that algorithm?

The first 99 for hazy IIPA is the no.7 rated. Surely we have more than 700 (or 1400) hazy IIPAs on the site? I fixed score so it can’t be that. The no.50 also is a 99, so I guess this doesn’t make sense unless something only gets a 100 style score if it’s in the top 0.05% or so. For reference

#7: https://www.ratebeer.com/beer/tree-house-juice-machine-2015-/385710/#
#50: https://www.ratebeer.com/beer/trillium-scaled-way-up/409752/#

So which one appears where? I guess review average is used for top 50 pages? But which appears on brewery pages, and which on search results?

1 Like
6

I also thought this… but in one of the screenshots in the OP it reads “These scores are calculated using an algorithm and are weighted using a Bayesian formula; it takes into account both the reviews given by each user and the total number of reviews for the beer.”
So it seems that if you have fewer ratings (especially when around the 10-50 beer range), it will be weighted less than someone with 2000+ ratings. Either that or if your average rating given is very different from the average beer rating.
I think.

7

I think default, where available should be still the weighted full review score. When beers don’t have 50 ratings or more, then you can put the average up there, but perhaps in red or some color that indicates this isn’t equal. As in, so a new beer with 11 ratings at 4.5 doesn’t appear equal to Westy 12’s 4.5 if that makes sense.

Or better yet, let users toggle which one they want to see first like we can do on the app. On the app, I can see average, style score, or overall score. Put the ball in user’s court

But definitely definitely we should still use full review scores to give RB awards, and to compile top 50 etc. (though having a top fifty from ticks as another stats option is fine with me too, as long as it’s separate).

1 Like
8

It would be nice to have the types of scores shown be a toggle box / filter of some kind on a customizable Top 50 similar to how the # of ratings is currently.

1 Like
9

Will your reviews and ticks (5*Ratings) still be displayed under separate categories in your profile once this change goes live or will they be rolled into a single category?

10

I think we could use all ratings for the RB awards / top lists, as long as the Beer qualify to have a weighted average score, not beers without enough ratings.

Will weighted average become the weighted average of both 5-star ratings (ticks) and reviews?

11

I just think that if we do this, it will make it easier to abuse. It’s easier to give a million fake 5-star reviews to amplify the beer you just brewed than to write reviews on a million beers. Or maybe not. Maybe it will be obvious. We can also address that if and when it becomes a problem, I guess.

12

If that is true, then it’s a change in policy that has not been communicated to us.

13

This matter should be addressed more IMO by increasing the minimal number of ratings a single user must have before its counts for scoring. If a user is cheating, it will be more obvious.

14

I think he just didn’t understood the way it is written. The sentence was about the number of ratings the Beer have, not the user…

1 Like
15

@aww
I don’t know if it could be possible but, next to the 5-Star Tick bar, if you could add a ADD TO CELLAR option.
Then in the beer entries, it could be displayed as CELLAR (similar to the 5-star block) to quickly see a user have one already. This would remove the need for user to use 5-star system for this matter.