RATINGS/DECODED

Math / METHOD

A simple confidence-interval explainer for star ratings

A confidence interval is a range that would contain the long-run average in about 95 of 100 repeated samples. For star averages, take the mean plus or minus about two standard errors. For thumbs-up or percent-recommend scores, use the Wilson interval, which behaves properly with small counts.

Every rating is an estimate. A confidence interval is the honest way of writing one: not "4.6", but "4.6, give or take 0.2". This page shows two versions, one for star averages and one for yes-or-no scores.

Version 1: an average of star ratings

Three ingredients.

  • n, the number of ratings
  • mean, the average
  • s, the standard deviation of the individual ratings

The 95% interval is

mean ± 1.96 × s ÷ √n

The quantity s ÷ √n is the standard error. The multiplier 1.96 comes from the normal distribution and is safe for samples of 30 or more. Below that, use a slightly larger t multiplier: 2.26 for n = 10, 2.09 for n = 20.

Worked example

A dentist has 60 Google ratings: 45 fives, 8 fours, 3 threes, 1 two and 3 ones.

  1. Mean = (225 + 32 + 9 + 2 + 3) ÷ 60 = 271 ÷ 60 = 4.52
  2. Standard deviation = 1.03
  3. Standard error = 1.03 ÷ √60 = 1.03 ÷ 7.75 = 0.133
  4. Interval = 4.52 ± 1.96 × 0.133 = 4.52 ± 0.26

Result: 4.26 to 4.78. The display says 4.5. The data say somewhere from the low fours to the high fours.

How width changes with count

For a fixed spread of 0.9 stars, the half-width of the interval at different counts:

Half-width of the 95% interval for an average, spread 0.9 stars00.20.40.60.80.7950.56100.35250.25500.181000.112500.085000.061000Number of ratingsPlus or minus, stars
Our calculation of 1.96 × 0.9 ÷ √n. The horizontal spacing is by category, not to scale.

The curve falls fast and then flattens. Quadrupling the count halves the width, every time.

If you only have the average and the count

Most platforms show the mean and n but hide the full breakdown. You can still estimate. Ratings in the mid-fours with a visible minority of low reviews have a spread around 0.9 to 1.0 in the worked examples on this site. Using 0.9 gives a reasonable margin. For profiles that are nearly all fives, the spread can be as low as 0.5; for sharply divided ones, as high as 1.4.

Version 2: percent positive

Some scores are proportions: percent recommended, thumbs up versus down, share of four and five star reviews. The textbook interval for a proportion, p ± 1.96 × √(p(1−p) ÷ n), fails badly with small counts. If 8 of 8 ratings are positive it returns 100% ± 0, claiming perfect certainty from eight data points.

Statisticians have known this for a long time. Brown, Cai and DasGupta showed that the standard interval's coverage is erratic and that common textbook advice about when it is safe "cannot be trusted", and they recommended the Wilson interval for small samples.11 Brown, Cai and DasGupta, Statistical Science 16(2), 2001, Interval Estimation for a Binomial Proportion Agresti and Coull likewise found that Wilson's method gives coverage "close to nominal confidence levels, even for very small sample sizes".22 Agresti and Coull, The American Statistician 52(2), 1998, Approximate is Better than "Exact" for Interval Estimation of Binomial Proportions Wilson published it in 1927.33 Wilson, Journal of the American Statistical Association 22(158), 1927, Probable Inference, the Law of Succession, and Statistical Inference

The Wilson interval, with p the observed share positive, n the count and z = 1.96:

center = (p + z²/2n) ÷ (1 + z²/n)

half-width = z × √( p(1−p)/n + z²/4n² ) ÷ (1 + z²/n)

Worked table

PositiveTotalObservedWilson 95% interval
22100%34% to 100%
91090%60% to 98%
1212100%76% to 100%
192095%76% to 99%
485096%87% to 99%
9510095%89% to 98%
47550095%93% to 97%
82890092%90% to 94%
0.30.420.530.650.770.8812 of 20.34 to 1.012 of 120.76 to 1.019 of 200.76 to 0.9995 of 1000.89 to 0.98828 of 9000.90 to 0.94Wilson 95% interval for the share of positive ratings
Dots show the observed share. A perfect 12 of 12 has a lower bound of 76%, below the lower bound of 828 of 900.

Two perfect scores sit at the top of that table with lower bounds of 34% and 76%. The last row, a merely good 92%, has a lower bound of 90%. Sorting by the lower bound puts the 900-rating business first, which is the argument Evan Miller made in a well-known 2009 essay on ranking rated items.44 Evan Miller, How Not To Sort By Average Rating (2009)

The plus-four shortcut

Agresti and Coull observed that the 95% Wilson interval behaves much like the ordinary formula after "adding two successes and two failures" to the data.22 Agresti and Coull, The American Statistician 52(2), 1998, Approximate is Better than "Exact" for Interval Estimation of Binomial Proportions So for a quick mental estimate: add 2 to the positives and 4 to the total, then proceed as usual.

Eight of eight becomes 10 of 12, or 83%, with a margin of about 21 points. That is far more sensible than 100% ± 0.

What an interval does not cover

A confidence interval describes sampling error only: the wobble you would get from drawing a different random set of reviewers from the same pool. It assumes the reviews are a fair sample of that pool.

It says nothing about:

  • Selection. Who chose to review, and who was asked.
  • Filtering. Which reviews the platform removed or set aside.
  • Fraud. Reviews that describe no real experience.
  • Change over time. A business that was better or worse three years ago.

These are usually larger than the sampling error once a profile has more than a few hundred reviews. The interval is the floor on your uncertainty, not the whole of it.

When to bother

For two businesses with thousands of reviews each, do not. The intervals are tiny and other issues dominate. The calculation earns its keep when at least one option has fewer than about 100 reviews, which describes most local service businesses most people compare.

Questions readers ask

What does "95% confidence" actually mean?

If you repeated the sampling many times and built an interval each time, about 95% of those intervals would contain the true long-run value. It is a statement about the method's reliability, not a 95% probability for any single interval.

Can a confidence interval go above 5 stars?

The formula for a mean can produce an upper limit above 5 when ratings are bunched at the top. Truncate at 5. The Wilson interval for proportions never leaves the 0 to 100% range.

Is there a quick rule without any formula?

Divide 1.8 by the square root of the review count. That is the approximate margin in stars for a typical profile: about 0.6 at 10 reviews, 0.18 at 100, 0.06 at 1,000.

Method Formulas follow the statistical papers cited. All worked figures are our own calculations and can be reproduced with the calculator in this section. See how we source and check.

Sources

Every source below was opened and read for this page. Links go to the publisher's own copy.

  1. Brown, Cai and DasGupta, Statistical Science 16(2), 2001. Interval Estimation for a Binomial Proportion
  2. Agresti and Coull, The American Statistician 52(2), 1998. Approximate is Better than "Exact" for Interval Estimation of Binomial Proportions
  3. Wilson, Journal of the American Statistical Association 22(158), 1927. Probable Inference, the Law of Succession, and Statistical Inference
  4. Evan Miller. How Not To Sort By Average Rating (2009)