RATINGS/DECODED

Explainers / READING

How many reviews does a business need before its rating means something?

There is no magic number, but the uncertainty shrinks with the square root of the count. With a typical spread of ratings, 10 reviews fix the average to within about half a star either way, 100 reviews to within about 0.18, and 1,000 to within about 0.06. Below roughly 30, the first decimal place is mostly noise.

Shoppers are bad at this, and it has been measured. In a study covering 1,272 products, researchers found that average ratings "are often based on insufficient sample sizes which limits their informativeness", and that consumers "fail to moderate their reliance on the average user rating as a function of sample size sufficiency".11 de Langhe, Fernbach and Lichtenstein, Journal of Consumer Research 42(6), 2016, Navigating by the Stars: Investigating the Actual and Perceived Validity of Online User Ratings People see 4.9 and stop reading, whether it comes from 9 reviews or 9,000.

The square-root rule

Each review is one noisy reading of how the business performs. Averaging readings cancels noise, but slowly: to halve the uncertainty you need four times the reviews. The margin of error on an average is roughly

1.96 × spread ÷ √n

where spread is the standard deviation of the individual ratings and n is the count. For a business in the mid-fours with the usual scatter of a few low reviews, a spread near 0.9 stars is realistic. That gives:

ReviewsMargin of error, starsA displayed 4.6 could plausibly be
50.793.8 to 5.0
100.564.0 to 5.0
250.354.25 to 4.95
500.254.35 to 4.85
1000.184.4 to 4.8
2500.114.5 to 4.7
5000.084.5 to 4.7
1,0000.064.55 to 4.65
Margin of error on an average rating, by number of reviews00.20.40.60.80.7950.56100.35250.25500.181000.112500.085000.061000Number of reviewsPlus or minus, stars
Our calculation: 1.96 × 0.9 ÷ √n. The assumed spread of 0.9 stars is illustrative; a business with more divided reviews has wider margins at every count.

Three thresholds fall out of the table.

Under 10: almost nothing. The interval is over a star wide. A 4.6 and a 4.0 are statistically the same business.

Around 30: the first decimal starts to hold. The margin is about a third of a star. You can tell a 4.7 from a 4.0, though not from a 4.4.

Around 250 and up: one decimal place is real. The margin is about a tenth. Differences of 0.2 between two businesses probably reflect something.

Notice the flattening. Going from 10 to 100 reviews cuts the margin by 0.38 stars. Going from 500 to 1,000 cuts it by 0.02. Past a few hundred, more reviews add little statistical precision, and other problems, like who is being asked and what is being filtered, dominate.

What sales research found about the first few

The same diminishing return shows up in buyer behavior. Northwestern's Spiegel Research Center found that the purchase likelihood for a product with five reviews is 270% greater than for a product with none, and that "the marginal benefit of additional reviews begins diminishing rapidly after the first five reviews".22 Medill Spiegel Research Center, Northwestern University, How Online Reviews Influence Sales

Buyers, in other words, respond most strongly to the stretch from zero to five reviews, which is the stretch where the average is least reliable. The gap between persuasive and informative is widest at the start.

Sample size cannot fix a biased sample

Everything above assumes reviews are a fair draw from customers. They are not. Reviews come from people who chose to write, and a 25-platform study found that this self-selection, with extreme opinions over-represented, both produces polarized distributions and "reduce[s] the informativeness of online reviews".33 Schoenmueller, Netzer and Stahl, Journal of Marketing Research 57(5), 2020, The Polarity of Online Reviews: Prevalence, Drivers and Implications

More reviews shrink random error. They do nothing about systematic error. Ten thousand reviews collected by asking only delighted customers give you a very precise estimate of how delighted customers feel.

So the count answers one question, how much is this number bouncing around by chance, and leaves another open: is this number aimed at the right target.

Platforms know about small samples

Several build in protections.

Google's stated method has no such adjustment. One review at five stars displays as 5.0.99 Google Business Profile Help, Understand review scores for local places and businesses

Three reviews as a qualifying bar, used by two home-services platforms, is worth a second look. At n = 3 the margin of error is about a full star. Those thresholds screen out the clearly bad; they cannot distinguish good from very good.

A rule of thumb for comparing two businesses

When choosing between two local businesses, compare the uncertainty ranges, not the averages.

  1. Find each margin from the table using the review count.
  2. Subtract it from the displayed average to get a cautious floor.
  3. Prefer the higher floor.

A 4.9 from 8 reviews has a floor near 4.3. A 4.6 from 300 has a floor near 4.5. The second business is the safer choice on the evidence, although the first may well be excellent; you simply cannot tell yet. Ranking by the cautious floor is the idea behind the Wilson score method that Evan Miller popularized for sorting rated items.1010 Evan Miller, How Not To Sort By Average Rating (2009)

Questions readers ask

Is 4.5 stars from 20 reviews good?

It is consistent with a true average anywhere from about 4.1 to 4.9. That range includes excellent and merely decent. Read the reviews themselves and check their dates before leaning on the number.

Do more reviews mean a business is better?

No. A high count shows volume of customers or an active review-request program. It makes the average more precise without making it higher. Tripadvisor says explicitly that having more reviews does not by itself mean a higher rank.66 Tripadvisor Insights, Everything you need to know about the Tripadvisor Popularity Ranking

Why do new businesses often have perfect ratings?

Because with three or four reviews, all five-star is a common outcome even for an ordinary business, and early reviewers are often friends or enthusiasts. See is a perfect 5.0 suspicious.

Method The interval widths are our own calculations using the standard error of a mean with an assumed standard deviation of 0.9 stars, which matches the 900-review example in our math section. The research findings are quoted from the cited papers. See how we source and check.

Sources

Every source below was opened and read for this page. Links go to the publisher's own copy.

  1. de Langhe, Fernbach and Lichtenstein, Journal of Consumer Research 42(6), 2016. Navigating by the Stars: Investigating the Actual and Perceived Validity of Online User Ratings
  2. Medill Spiegel Research Center, Northwestern University. How Online Reviews Influence Sales
  3. Schoenmueller, Netzer and Stahl, Journal of Marketing Research 57(5), 2020. The Polarity of Online Reviews: Prevalence, Drivers and Implications
  4. Trustpilot Help Center. TrustScore and star rating explained
  5. Meta Business Help Center. About business ratings and reviews
  6. Tripadvisor Insights. Everything you need to know about the Tripadvisor Popularity Ranking
  7. Thumbtack Help. How to improve my rank in search results
  8. Angi. Frequently Asked Questions
  9. Google Business Profile Help. Understand review scores for local places and businesses
  10. Evan Miller. How Not To Sort By Average Rating (2009)