OUR SCORES page Bugs and Feedback

BUG and FEEDBACK for OUR SCORES / WHAT THESE SCORE MEAN page (last updated 2020-09-27)

https://www.ratebeer.com/our-scores

Here’s what should be the correct definitions:


Rating beers on RateBeer is easy section

Overall: Your overall personal experience of the product, which will dictate your final 5 STAR RATING.

Appearance and Mouthfeel are each scored out of 5. Aroma and Taste are scored out of 10. While Overall is scored out of 20. These all combine to give the product a total score out of 50, which is then divided and displayed as a score out of 5 for each rating.

example
Product #1: Aroma 7/10 Appearance 3/5 Taste 7/10 Mouthfeel 3/5 Overall 15/20
STAR RATING: 3.5/5
Product #2: Aroma 7/10 Appearance 3/5 Taste 8/10 Mouthfeel 2/5 Overall 16/20
STAR RATING: 3.6/5
Despite the 4 sensory attributes giving the same total, the higher Overall rating on Product #2 (3.6 star rating) reflects that user liked it more than Product #1 (3.5 star rating).


RateBeer Score Calculation Guide section

  1. STAR SCORE: An real average score out of 5 (not Star Rating)

Add this section before the OVERALL SCORE section

STAR SCORE section

Next to the star score, you will notice the number of ratings and the number of reviews. RATINGS that are accompanied with a written review of 75 or more characters are considered as REVIEWS.

RATING AVERAGE SCORE
Score based on the real average of all Ratings. This is the score displayed as the 5 STAR SCORE on the product page.

REVIEW AVERAGE SCORE
Score based on the real average of Reviews only. This score is displayed in the STATISTICS of the product page.

WEIGHTED AVERAGE SCORE
move the graph pic there
RateBeer uses an algorithm when calculating the weighted average score, considering both the reviews given by each user and the total number of reviews for the product.
Our scores are calculated with the same true Bayesian estimate formula used by the Internet Movie Database for calculating their average ratings (with 2.75 as the midpoint of the scale). On RateBeer, this means that as a product accumulates more ratings, the Weighted Average Score in turn becomes more accurate.
The Weighted average Score is based on REVIEWS only, not all Ratings.
TOP LISTS are made using the WEIGHTED AVERAGE SCORE, not the RATING AVERAGE SCORE.


Overall Score section

Remove everything and change to this only:

A product’s overall score reflects the rating given by RateBeer users and how this beer compares to all other products (no matter the kind) on RateBeer.

The OVERALL SCORE is a score based on 100 (Highest being 100) obtained by comparing the Weighted Rating Score of a product to all the products (no matter the kind) listed on the website


STYLE SCORE section
The style score is helpful in illustrating how a product stacks up against other beers listed within the same style category – it’s like comparing apples to apples and oranges to oranges (or Imperial Stouts to Imperial Stouts and Premium Lagers to Premium Lagers).

The STYLE SCORE is a score based on 100 (Highest being 100) obtained by comparing the Weighted Rating Score of a product to all of the products of the same style.


Quality of our scores

The framed section isn’t true anymore since the star rating includes all Ratings, not just reviews…so delete it

Add this instead there:
For increased accuracy, a beer must also receive a minimum number of ratings before the OVERALL SCORE and STYLE SCORE are visible.
To ensure the validity of all our scores and to combat abuse, ratings from new users on RateBeer do not impact the WEIGHTED AVERAGE SCORE until they have given several ratings.


PLACE SCORES

Also, could you add how PLACE SCORES are calculated too in this page:

There is no definition for each of the 6 points used to calculated the score. (and why 2 out of them don’t count towards the total)

There is no mention about if the 5-STAR PLACE SCORE uses Bayesian formula or not.

There is no mention as well if the GLOBAL PLACE SCORE is based against all places in the world vs country vs city as well as there’s no mention if it’s based against all places in the world vs types of places OR if it’s simply a percentile score of the average 5-star score…

@joet @services

3 Likes

That’s good feedback. Thank you!

Thanks, Viper. This is good feedback, especially the section on Places. Honestly, there have been so many changes there and so many places on the site that present Place data that it’s difficult for even me to have a complete catalog of what is presented where.

That’s thankfully incorrect. The 5-star average as shown is a strict real mean. The Bayesian, or weighted average, reflects confidence based on a number of ratings and not relative rank.

I’ve requested an update to the language on this page. For one thing Aroma is out of 10 and not out of 5. A couple other small updates as well.

Place score explanations will come at some future update, hopefully right after we’ve cataloged everything and/or have made decisions on how we’d like to universally apply them. This is good feedback. Thank you.

Right now The 5 star score shown is changing from page to page :
On the search result page, it’s the weighted average score

On the beer page, it’s the real average score

On the Brewer page, it’s the weighted average score

On the best beers page, it’s the Overall score (percentile) which is based on the weighted average score , not the real average score.

Both the Overall Score and Style Score are percentile scores calculated by comparing weighted average scores, not real average…

@joet @aww @joet

Right now, is the OVERALL SCORE:
the equivalent of the Weighted Average Score expressed in %
or
a percentile rank ( the percentage of scores in its frequency distribution that are equal to or lower than it) of that Weighted Average Score compared to all products.

?

And now, it is even worse to follow the scores and reviews:

On the Main page / Top Beers page and on the Brewer page, it’s the WEIGHTED AVERAGE SCORE FOR REVIEWS ONLY that is displayed (followed by the NUMBER OF REVIEWS ONLY in brackets)

On the BEER SEARCH RESULT and the BEER PAGE
The REAL AVERAGE SCORE FOR ALL RATINGS that is displayed (but still followed by the NUMBER OF REVIEWS ONLY in brackets)

And all of them listed with the OVERALL and STYLE SCORES that are based on the WEIGHTED AVERAGE SCORE, no matter which /5 score listed…

So people right now are seeing the Best beer in the world as a 3.5/5 100% 100% beer based on 158 users.

What a mess…

@joet @aww @joet

why did you cut the number of TOTAL RATING vs REVIEWS on the Stat pop up?..it was there on the proposed project

There seems to also have differences in scores between the App and website.

Seems like RATINGS now means QUICK RATING and REVIEWS still means FULL REVIEWS
Before RATINGS meant QUICK RATINGS + REVIEWS

Why not showing this way: STAR SCORES = RATINGS + REVIEWS and explain it correctly in the OUR SCORES / What these scores mean page

Now Beer score seems to be an average of RATINGS ONLY, not REVIEWS
Now Beer with REVIEWS ONLY don’t even have a score…

Even worse, Number of Reviews listed on the beer description section doesn’t fit now with the number of reviews listed in the Review section.

Check the picture…

Lion Bleu Couteau got 0 ratings and 17 Reviews (without any 5-star score)on top, opposed to 17 Reviews on the reviews section, but 22 if we click on SHOW PRIVATE REVIEWS .

Toppling Goliath got 1037 Ratings and 158 Reviews (with a 3.50 score) on Top opposed to 165 Reviews on the reviews section but 1037 if we click on SHOW PRIVATE REVIEWS .

@aww @services @joet

updated

@services

at the same time, can you confirm this please?