Lesson 1.3: Summary statistics: centre, spread and shape
Learn the mean, median, mode, standard deviation, quartiles and skew, when each one misleads, and how to describe a column in three numbers.
By the end of this lesson you will be able to describe any column of numbers in three parts: where its centre is, how spread out it is, and what shape it takes. You will know when the mean is a fair summary and when the median is the honest one, how to compute and read a standard deviation and a set of quartiles, how to spot skew and two-peaked data, and why an average of averages can be badly wrong. You should have read Lessons 1.1 and 1.2, so that you can read a table and have a clean column to work with. The mathematics here is arithmetic and one square root, and every formula is worked by hand on a small table before it is used on a large one.
A table with 10,000 rows is too much to read. A summary statistic is a single number that stands in for the whole column, such as the average order value or the typical delivery time. Summaries are useful because they fit in a sentence and a dashboard. They are dangerous for the same reason: one number cannot carry the whole story, and each kind of summary hides something. This lesson is about choosing summaries that hide as little as possible.
Why one number is never enough
Two sales teams each report an average of ₹60,000 a month per person. In the first team everyone earns between ₹50,000 and ₹70,000. In the second team nine people earn ₹33,000 and one earns ₹306,000. The averages are identical, and the teams are not. Anyone planning a bonus pool, a loan eligibility rule or a hiring budget on the basis of "₹60,000 on average" will be badly wrong for the second team.
The cure is to describe a column with three questions, and each question has its own family of statistics.
Centre: what is a typical value? The mean, the median and the mode.
Spread: how much do values differ from one another? The range, the quartiles and the standard deviation.
Shape: where do values pile up and where do they thin out? Skew, tails and peaks.
Use all three together. A centre without a spread is half a fact.
Centre: mean, median and mode
The mean is the sum of the values divided by how many there are. It uses every value, which is its strength and its weakness. The median is the middle value once you sort the column. If there is an even number of values, it is the average of the two middle ones. The median ignores how large the top and bottom values are, and only cares about their order. The mode is the most frequent value. It is the only one of the three that works for categories, such as the most common payment method.
Here are ten monthly salaries from a small team, in thousands of rupees, already sorted: 24, 26, 28, 30, 32, 34, 36, 40, 44 and 306.
The sum is 24 + 26 + 28 + 30 + 32 + 34 + 36 + 40 + 44 + 306 = 600, so the mean is 600 ÷ 10 = 60. The median is the average of the fifth and sixth values, 32 and 34, which is 33. There is no mode, since no value repeats. The mean of ₹60,000 is higher than nine of the ten people's salaries. Nobody in the team earns anything close to it. The median of ₹33,000 describes the middle of the team far better.
Predict first: if the largest salary, 306, were replaced by 3,060, ten times larger, what would happen to the mean and to the median? Decide before you read on.
The mean would jump, because it adds the extra amount and shares it among ten people: (294 + 3,060) ÷ 10 = 335.4. The median would not move at all, because the fifth and sixth values are still 32 and 34. The mean chases extreme values. The median is robust, which is the statistician's word for resistant to extremes. Try the sliders and watch the two lines.
Slide the largest salary from 44 up to 1,000. The median stays at 33 the whole way, and the mean climbs from 33.8 to 129.4. That is the single most useful picture in descriptive statistics. When the mean and the median are far apart, you have learned something about the shape of the data before you have drawn a single chart: a few large values are pulling the mean away from the centre.
Which should you report?
Use the mean when values are fairly symmetric and no extreme values dominate, such as heights, exam marks in a large class, or measurement errors. Use the median when values are skewed, such as incomes, house prices, transaction sizes, delivery delays and claim amounts. When you are unsure, report both and the gap between them, because the gap is itself informative. A report that says "average order value ₹2,400, median ₹900" tells a decision-maker much more than either number alone.
The weighted mean
Sometimes rows are not equally important, and the plain mean treats them as if they were. A lender holds three loans: ₹10 lakh at 9 per cent, ₹40 lakh at 12 per cent and ₹50 lakh at 14 per cent. The simple average of the three rates is (9 + 12 + 14) ÷ 3 = 11.67 per cent. But half the money is in the 14 per cent loan, so that average understates what the portfolio actually earns. The weighted mean multiplies each value by its weight, adds, and divides by the total weight: (10 × 9 + 40 × 12 + 50 × 14) ÷ 100 = (90 + 480 + 700) ÷ 100 = 12.7 per cent.
The gap of about one percentage point is large enough to change a pricing decision. Whenever a column describes groups of different sizes, such as rates across loans, scores across schools or prices across stores with different sales, ask whether a weighted mean is the honest one.
Spread: how much values differ
Two columns can share a centre and still behave very differently. A courier whose deliveries take 28 to 32 minutes is predictable. A courier whose deliveries take anywhere from 10 to 50 minutes is not, even if both average 30. Spread statistics put a number on that difference.
Range: the largest value minus the smallest. It is easy to compute and easy to fool, because it depends on only two values.
Quartiles and the interquartile range: the first quartile Q1 has a quarter of the values below it, the median has half, and the third quartile Q3 has three quarters. The distance from Q1 to Q3 is the interquartile range, or IQR, and it covers the middle half of the data. You met it in Lesson 1.2 as the basis of the outlier rule.
Variance and standard deviation: the standard measure of typical distance from the mean.
Standard deviation, step by step
The standard deviation is the most used spread statistic, and it is worth computing by hand once. Take the eight values 2, 4, 4, 4, 5, 5, 7 and 9.
1. Find the mean: (2 + 4 + 4 + 4 + 5 + 5 + 7 + 9) ÷ 8 = 40 ÷ 8 = 5.
2. Find each value's deviation from the mean: −3, −1, −1, −1, 0, 0, 2, 4.
3. Square each deviation, so that negatives and positives do not cancel: 9, 1, 1, 1, 0, 0, 4, 16.
4. Add the squares: 9 + 1 + 1 + 1 + 0 + 0 + 4 + 16 = 32.
5. Divide by the number of values to get the variance: 32 ÷ 8 = 4.
6. Take the square root to get back to the original units: √4 = 2.
The standard deviation is 2: a typical value sits about 2 away from the mean. The squaring in step 3 is what makes the variance awkward, because its units are squared (rupees squared, minutes squared). The square root in step 6 repairs that.
One detail confuses nearly every beginner. When your data is a sample of a larger group and you want to say something about the whole group, you divide by n − 1 instead of n in step 5. Here that gives 32 ÷ 7 = 4.571, and a standard deviation of √4.571 = 2.138. The reason is that the sample mean is calculated from the same values, so the deviations from it come out a little too small. Dividing by n − 1 corrects the shrinkage. Software tools differ: pandas divides by n − 1 by default, while NumPy divides by n unless told otherwise. If two tools give you slightly different standard deviations, this is almost always why. For large tables the difference is tiny, and for tables of a dozen rows it is not.
Two couriers, one mean
Here are six delivery times, in minutes, for each of two couriers. Courier A took 28, 29, 30, 30, 31 and 32. Courier B took 10, 20, 30, 30, 40 and 50. Both means are 30. For A the deviations are −2, −1, 0, 0, 1 and 2, the squares add to 10, the sample variance is 10 ÷ 5 = 2, and the standard deviation is √2 = 1.41 minutes. For B the deviations are −20, −10, 0, 0, 10 and 20, the squares add to 1,000, the sample variance is 200, and the standard deviation is √200 = 14.14 minutes.
If a customer is promised delivery within 35 minutes, Courier A meets it every time. Courier B breaks it two times in six, at 40 and 50 minutes. The average hid exactly the fact the customer cares about. This is why service levels are written as percentiles, such as "95 per cent of deliveries within 40 minutes", and not as averages.
Percentiles and the five-number summary
A percentile tells you the value below which a given share of the data lies. The median is the 50th percentile, Q1 is the 25th and Q3 is the 75th. The five-number summary lists the minimum, Q1, the median, Q3 and the maximum, and it is the quickest honest description of a column. For Courier B the minimum is 10, Q1 is 20, the median is 30, Q3 is 40 and the maximum is 50. A box plot draws exactly these five numbers as a box from Q1 to Q3 with a line at the median and whiskers out toward the extremes. You will meet box plots in Lesson 1.4.
Percentiles also behave well for very skewed data. A bank might say that the median call wait is 2 minutes but the 95th percentile is 11 minutes. Both are true, and the second explains why some customers are angry.
Shape: where the values pile up
Shape is what a chart called a histogram shows. A histogram splits the range of values into bins and counts how many values fall in each. Three features of shape matter most.
Skew describes lopsidedness. A distribution is right-skewed when it has a long tail of large values, as with salaries, order values and waiting times. In right-skewed data the mean is pulled above the median, which is the pattern you saw in the salary example. A left-skewed distribution has a long tail of small values, for example the age at which people retire, where most are near 60 and a few retire very young. For a rough rule, mean above median suggests right skew and mean below median suggests left skew.
Tails describe how often extreme values occur. Data with heavy tails produces surprises more often than a bell-shaped curve would predict, as with stock returns and insurance claims. Never assume a bell curve for financial data without checking.
Peaks are the third feature, and the one most often missed. A column can have two humps. A summary like the mean then lands in the valley between them and describes nobody. Here is a table of 400 trips from a ride-hailing app, counted by length.
Using the middle of each bin, the mean works out to (120 × 2.5 + 60 × 7.5 + 20 × 12.5 + 40 × 17.5 + 110 × 22.5 + 50 × 27.5) ÷ 400 = 5,550 ÷ 400 = 13.9 km. Yet only 20 of the 400 trips, 5 per cent, fall in the 10 to 15 km bin that contains the mean. The average trip is a trip almost nobody takes. The right description is two kinds of trip, which might need two different pricing rules. Whenever a mean or median feels odd, draw the histogram before deciding anything.
Standardising: how unusual is a value?
To compare values from different columns, or to say how unusual one value is, convert it to a z-score: the number of standard deviations it lies from the mean. The formula is z = (value − mean) ÷ standard deviation. A student who scored 72 in a test where the mean was 60 and the standard deviation was 8 has z = (72 − 60) ÷ 8 = 1.5, which means 1.5 standard deviations above average.
For data that is roughly bell-shaped, the empirical rule gives a quick feel: about 68 per cent of values lie within one standard deviation of the mean, about 95 per cent within two, and about 99.7 per cent within three. A z-score beyond 2 in either direction is unusual, and beyond 3 is rare. The rule is only reliable for bell-shaped data. For skewed data, use percentiles instead.
The z-score is also the link back to Lesson 1.2. The interquartile range rule and the "more than three standard deviations" rule are two ways of saying the same thing: this value is far from the rest. The IQR rule suits skewed data, and the z-score rule suits data that is roughly bell-shaped.
Summaries by group, and the average of averages
Most business questions are about groups: average by branch, by month, by product. A group-by produces one summary per group. It is powerful and easy to misuse in one specific way, which is averaging the averages.
Suppose Branch A has 100 customers who spend an average of ₹50 a month, and Branch B has 900 customers who spend an average of ₹10. The simple average of the two branch averages is (50 + 10) ÷ 2 = ₹30. The true average across all customers is (100 × 50 + 900 × 10) ÷ 1,000 = (5,000 + 9,000) ÷ 1,000 = ₹14. The average of averages is more than double the truth, because it gives a branch of 100 the same weight as a branch of 900. It is the weighted mean again: the correct overall average weights each group by its size. The safest habit is to compute the overall figure from the raw rows, or from group totals and counts, and never from group averages.
Doing it in Python
All of this takes a few lines in pandas. The describe() method gives the count, mean, standard deviation, minimum, quartiles and maximum at once, and groupby computes any of them by group.
python
import pandas as pd
df = pd.readcsv("trips.csv")
df["km"].describe() # count, mean, std, min, Q1, median, Q3, max
df["km"].mean(), df["km"].median() # centre
df["km"].std(), df["km"].std(ddof=0) # sample (n-1) and population (n) versions
df["km"].quantile([0.05, 0.5, 0.95]) # any percentiles
df["km"].skew() # positive means a long right tail
df["km"].mode() # most frequent value(s)
df.groupby("city")["km"].agg(["count", "mean", "median", "std"])
weighted mean: total weighted value divided by total weight
(df["rate"] df["amount"]).sum() / df["amount"].sum()
Always put the count next to a group mean. An average of 4.9 stars from 3 reviews and one of 4.6 from 3,000 reviews are not comparable, and a table that hides the count invites the wrong conclusion.
Choosing summaries: a quick guide
Column · Centre · Spread · What to checkRoughly symmetric numbers (heights, marks) · Mean · Standard deviation · Histogram for bell shape
Skewed numbers (income, claims, waiting time) · Median · IQR and percentiles · Mean against median
Rates across groups of unequal size · Weighted mean · Weighted spread or per-group values · Group sizes
Categories (payment type, city) · Mode and shares · Number of distinct values · Rare categories
Anything with two humps · Describe each hump · Separate summaries · Histogram first
## Common mistakes
Reporting only the mean. Always pair a centre with a spread, and when the mean and median differ widely say so.
Averaging averages. Combine groups with a weighted mean, or compute from the raw rows.
Ignoring the count. A small group's average is noisy. Show the number of rows beside every group statistic.
Using the standard deviation for skewed data. A few extreme values inflate it. Quartiles and percentiles are safer.
Mixing the n and n − 1 versions. If two tools disagree slightly, check which divisor each used.
Trusting a summary without a picture. Two-peaked data and hidden outliers make summaries lie. Look at a histogram first.
Treating the mean as typical. In the salary example the mean was higher than nine of the ten people.
Key takeaways
Describe every numeric column with a centre, a spread and a shape. One number is never enough.
The mean uses every value and is pulled by extremes. The median is robust. When they differ widely, the data is skewed.
Use a weighted mean when rows stand for groups of different sizes, and never average group averages.
The standard deviation is the typical distance from the mean. Use n − 1 for a sample, and know which one your tool uses.
Quartiles, the interquartile range and percentiles describe spread without being thrown off by extremes, and service promises belong in percentiles.
Shape matters: right skew, heavy tails and two peaks each change which summary is honest. A z-score tells you how many standard deviations a value sits from the mean.
Check yourself
1. The salaries 24, 26, 28, 30, 32, 34, 36, 40, 44 and 306 have a mean of 60. What is the median, and what does the gap say?
The median is 33. The mean is far above it because one large salary pulls it up, so the data is right-skewed and the median is the fairer centre.
2. Why do we divide by n − 1 when computing a sample standard deviation?
The sample mean comes from the same values, so deviations from it are slightly too small. Dividing by n − 1 corrects that shrinkage.
3. A lender has ₹20 lakh at 10 per cent and ₹80 lakh at 15 per cent. What is the weighted average rate?
(20 × 10 + 80 × 15) ÷ 100 = 14 per cent. The simple average of 12.5 per cent understates it.
4. A student scores 72 where the mean is 60 and the standard deviation is 8. What is the z-score and is it unusual?
z = 1.5. It is above average but within two standard deviations, so it is not unusual.
5. Branch A has 50 customers averaging ₹80 and Branch B has 450 customers averaging ₹20. What is the overall average?
(50 × 80 + 450 × 20) ÷ 500 = 13,000 ÷ 500 = ₹26, not the ₹50 you get by averaging the two branch averages.
Try this
Use a real numeric column: your monthly expenses, delivery times, call durations, marks, or a column from an open data file. Set aside thirty-five to forty-five minutes.
1. Take at least thirty values. Sort them and write down the minimum, Q1, median, Q3 and maximum, which is the five-number summary.
2. Compute the mean and the median. If they differ by more than about 10 per cent of the median, note which way the skew goes.
3. Compute the standard deviation, using the sample version, and the interquartile range. Say in one sentence what each tells you.
4. Find the z-score of the largest and the smallest value. Are either unusual? Check them against the interquartile range rule from Lesson 1.2.
5. Split the column into groups, such as weekdays and weekends or two categories. Compute the count, mean and median for each group, and then compute the overall mean from the group counts and means to confirm it matches the overall mean from the raw rows.
6. Use the mean-and-median simulator above to see how much one value at ten times the maximum would move each.
7. Write a four-sentence description: the typical value, the spread, the shape, and one thing a reader should not conclude from the mean alone.
A good result is a short paragraph that a manager could act on, for example "the typical delivery takes 31 minutes, 90 per cent finish within 44, and the distribution has a long tail, so promises should be written as percentiles". Keep it: the next lesson turns the same summaries into charts.
Tags: Data Science Statistics Descriptive statistics Mean and median Beginner