Lesson 6.3: Regression metrics, calibration and choosing a threshold
Learn MAE, RMSE and MAPE for numeric predictions, how to check that predicted probabilities are honest, and how to pick a cutoff from the cost of each mistake.
By the end of this lesson you will be able to score a numeric prediction with MAE, RMSE, MAPE and R-squared and say which one fits a decision, test whether predicted probabilities can be believed with a calibration table and a Brier score, and choose a cutoff or a forecast level from the costs of the two kinds of mistake. You will compute every regression metric by hand from five predictions, watch one large miss pull the RMSE above the MAE, and work out that when a missed default costs twenty times a wrongly declined borrower, the best cutoff is a 4.8 per cent chance of default, not 50. You should have read Lessons 4.1, 4.2 and 6.2, since this lesson completes the set of scores for both kinds of model, and Lesson 2.1 for expected value. The mathematics is averages, squares and one ratio.
Lesson 6.2 judged a model by how well it ranks cases. But a ranking is only half the story. When the model predicts an amount, such as a loss, a price or a demand figure, you need a measure of how far off it is. When it predicts a probability that will be used for pricing, provisioning or a cutoff, you need to know that the probability is right and not merely well ordered. And when a decision must be made, the last step is a rule that turns a number into an action, which depends on what each mistake costs. This lesson covers all three.
Scoring a numeric prediction
The basic object is the error of a prediction: the predicted value minus the actual value. Metrics differ in how they combine the errors across many rows. Take five loans whose recovery amounts, in ₹ thousand, were 100, 120, 80, 150 and 200. A model predicted 110, 115, 90, 140 and 170. The errors, predicted minus actual, are +10, −5, +10, −10 and −30.
Loan · Actual · Predicted · Error · Absolute error · Squared error · Absolute % error1 · 100 · 110 · +10 · 10 · 100 · 10.0%
2 · 120 · 115 · −5 · 5 · 25 · 4.2%
3 · 80 · 90 · +10 · 10 · 100 · 12.5%
4 · 150 · 140 · −10 · 10 · 100 · 6.7%
5 · 200 · 170 · −30 · 30 · 900 · 15.0%
Total · · · −25 · 65 · 1,225 · 48.4%
From this table every standard metric follows.
Mean error, or bias: the average error, −25 ÷ 5 = −5. It shows whether the model is systematically too high or too low. Here it under-predicts by 5 on average. Errors of opposite sign cancel, so a model can have zero bias and still be poor.
Mean absolute error, MAE: the average size of the error, 65 ÷ 5 = 13. It is in the units of the outcome, easy to explain, and treats every unit of error equally.
Mean squared error, MSE: the average of the squared errors, 1,225 ÷ 5 = 245. Its units are squared, so it is hard to interpret.
Root mean squared error, RMSE: the square root of the MSE, √245 = 15.65. It is back in the original units, and it punishes large errors more than small ones.
Mean absolute percentage error, MAPE: the average of the absolute errors as a share of the actual value, 48.4 ÷ 5 = 9.7 per cent. It is unit-free, and breaks down when actual values are near zero.
R-squared on the test set: 1 − the sum of squared errors ÷ the total variation of the actual values around their mean. The mean actual is 130, the deviations are −30, −10, −50, 20 and 70, the sum of their squares is 8,800, so R² = 1 − 1,225 ÷ 8,800 = 0.86.
Predict first: the largest error is 30, on the last loan, and the other four errors are all 10 or less. Which of MAE and RMSE does that one error push up more, and by roughly how much? Decide before reading on.
It pushes up the RMSE far more. The MAE of 13 treats the error of 30 as three times the error of 10. The RMSE squares errors first, so the error of 30 contributes 900 against 100 for each of the others, 9 times as much, and the RMSE rises to 15.65. A gap between RMSE and MAE is a sign of large, uneven errors. If all five errors had been exactly 13, the MAE and RMSE would both be 13.
Choosing among them
Metric · Penalises large errors · Units · Use it when · WeaknessMAE · Equally · Same as outcome · You want a plain average miss, and large errors are not especially costly · Ignores that big misses may matter more
RMSE · Heavily · Same as outcome · Large errors are especially costly, such as a big loss · Dominated by a few outliers
MAPE · By relative size · Per cent · Comparing across series of different scale · Undefined at zero, and favours under-prediction
R-squared · Heavily · None · Share of variation explained, quick comparison with the average · Says nothing about the size of errors in real terms
Median absolute error · Hardly at all · Same as outcome · Outliers are common and you want a typical miss · Ignores the tail
The deciding principle is to ask what a mistake costs. If an error of 20 costs twice as much as an error of 10, MAE matches the loss. If it costs four times as much, RMSE does. In demand forecasting, an error of 10 per cent on a small item matters less than the same percentage on a large one, so an absolute measure such as the weighted version of MAPE or MAE may be better than plain MAPE. Whatever you choose, compare with a baseline. An RMSE of 15.65 means little until you know that predicting the average for everyone gives an RMSE of √(8,800 ÷ 5) = 42.
When over- and under-prediction cost different amounts
A symmetric error metric treats a prediction that is too high like one that is too low. Business rarely does. A retailer that under-forecasts demand loses sales, and one that over-forecasts holds unsold stock. Suppose a lost sale costs ₹50 of margin and an unsold item costs ₹10 to hold. The best stock level is not the median forecast. It is the level that balances these costs, called the critical ratio: the cost of being short divided by the cost of being short plus the cost of being long, 50 ÷ (50 + 10) = 0.83. The ideal stock is the 83rd percentile of the demand distribution, so you stock enough to meet demand 83 per cent of the time. If demand has a median of 100 and the 83rd percentile is 118, stock 118, not 100. Models that predict quantiles directly, called quantile regression, are trained to do exactly this.
Are the probabilities honest? Calibration
Lesson 6.2's AUC and KS judge only the ordering of scores. If the model says 30 per cent and means it, a lender can price and provision on it. If it says 30 per cent and the real rate is 41 per cent, the ordering may be perfect and the pricing wrong. Calibration is the agreement between predicted probabilities and observed frequencies.
The check is a reliability table: group the cases by predicted probability, and in each group compare the average predicted probability with the actual rate. The table from Lesson 4.2 is an example.
Two summaries compress this. The expected calibration error is the weighted average gap between predicted and actual, weighting each band by its number of borrowers: with gaps of 0.4, 0.4, 4 and 11 points on 500, 300, 150 and 50 borrowers, it is (0.4 × 500 + 0.4 × 300 + 4 × 150 + 11 × 50) ÷ 1,000 = 1,470 ÷ 1,000 = 1.47 points. The Brier score is the average squared gap between the predicted probability and the outcome, coded 1 for the event and 0 otherwise. For four loans with predicted default probabilities of 0.9, 0.2, 0.6 and 0.1, and outcomes of default, repay, default and repay, the squared errors are 0.01, 0.04, 0.16 and 0.01, and the Brier score is 0.22 ÷ 4 = 0.055. A model that says 0.5 for everyone scores 0.25, so lower is better. The Brier score rewards both good ranking and honest probabilities, and, like log loss from Lesson 4.2, it is a proper scoring rule: a model can only minimise it by reporting its true beliefs.
If a model is badly calibrated, the repair is a second small model that maps the raw scores to corrected probabilities. Platt scaling fits a logistic regression on the scores, and isotonic regression fits a stepped, increasing curve. The correction must be fitted on data the model has not trained on, and then checked on a further set. Calibration also drifts over time, since default rates change with the economy, so it must be monitored, a topic of Lesson 8.2.
From a probability to a decision: the cutoff
A calibrated probability and a cost table give the best cutoff without any search. Suppose a defaulting loan loses ₹2 lakh, while declining a good borrower loses ₹10,000 of profit. Flag an applicant if the expected loss from approving, p × 2,00,000, exceeds the expected loss from declining, (1 − p) × 10,000. Solving, p ÷ (1 − p) must exceed 10,000 ÷ 2,00,000 = 0.05, which gives a cutoff of 0.05 ÷ 1.05 = 0.048, or 4.8 per cent. In general the best cutoff is
cutoff = cost of a false positive ÷ (cost of a false positive + cost of a false negative)
With a missed default costing twenty times a wrongly declined good borrower, the cutoff is 1 ÷ 21 = 4.8 per cent, far below the habitual 50 per cent. A cutoff of 50 per cent is optimal only when the two mistakes cost the same.
The result can be tested directly on the 1,000 applicants of Lesson 6.2, where 60 default. At three cutoffs the counts of misses and false alarms were these. At 0.5, there were 46 missed defaulters and 6 false alarms. At 0.2, there were 19 misses and 109 false alarms. At 0.1, there were 6 misses and 276 false alarms. With a miss costing ₹200 thousand and a false alarm ₹10 thousand, the total costs are 46 × 200 + 6 × 10 = ₹9,260 thousand at 0.5, 19 × 200 + 109 × 10 = ₹4,890 thousand at 0.2 and 6 × 200 + 276 × 10 = ₹3,960 thousand at 0.1.
The cutoff of 0.5 is more than twice as costly as 0.1, and the theory says to go lower still, to about 0.05. Real decisions have wrinkles. Capacity limits mean you cannot review every flagged case, so the cutoff may be set by how many cases the team can handle. Costs are uncertain, so it helps to see how the best cutoff moves across a plausible range. Declined applicants may complain or be lost for good. And the probabilities must be calibrated for the formula to hold. But the principle survives all of these: the cutoff is a business decision, and it comes from costs.
When capacity sets the cutoff
Sometimes no cost table exists, because the real limit is people. A fraud team can review 200 alerts a day out of 100,000 transactions, of which about 100 are fraudulent. The cutoff is then whatever score places 200 transactions above it, and the question becomes how many frauds are in those 200. Suppose the model's precision in its top 200 is 20 per cent. Then 40 of the 100 frauds are caught each day, a recall of 40 per cent, and the other 60 pass through. Doubling the team to 400 reviews a day might raise precision in the extra band to only 8 per cent, so the second 200 reviews find 16 more frauds, a recall gain of 16 points for the same cost as the first 200, which found 40. This is the shape of nearly every capacity problem: the first reviews are the most productive, and returns fall as the team goes down the list.
The decision rule is to keep adding reviews while the value of the next frauds caught exceeds the cost of the reviews. If a caught fraud saves ₹8,000 and a review costs ₹150, then a band with 8 per cent precision returns 0.08 × 8,000 = ₹640 per review against a cost of ₹150, and is still worth doing, while a band with 1.5 per cent precision returns only ₹120 and is not. The break-even precision is the cost of a review divided by the value of a catch, 150 ÷ 8,000 = 1.9 per cent. This is the same cost-based cutoff as before, written in terms of precision, and it shows that the idea of choosing a cutoff from costs reaches well beyond lending.
In Python the regression metrics and calibration tools are one-liners.
python
import numpy as np
from sklearn.metrics import (meanabsoluteerror, meansquarederror, r2score,
meanabsolutepercentageerror, brierscoreloss)
from sklearn.calibration import calibrationcurve, CalibratedClassifierCV
pred = reg.predict(Xtest)
mae = meanabsoluteerror(ytest, pred)
rmse = np.sqrt(meansquarederror(ytest, pred))
mape = meanabsolutepercentageerror(ytest, pred)
baselinermse = np.sqrt(meansquarederror(ytest, np.fulllike(ytest, ytrain.mean(), dtype=float)))
probs = clf.predictproba(Xtest)[:, 1]
print(brierscoreloss(ytest, probs))
actual, predicted = calibrationcurve(ytest, probs, nbins=10, strategy="quantile")
calibrated = CalibratedClassifierCV(clf, method="isotonic", cv=5).fit(Xtrain, ytrain)
costfn, costfp = 200000, 10000
cutoff = costfp / (costfp + costfn) # 0.048, if the probabilities are calibrated
flag = calibrated.predictproba(Xtest)[:, 1] >= cutoff
Common mistakes
Using one error metric by habit. Ask what a mistake costs and pick MAE, RMSE or a cost-weighted measure to match.
Ignoring bias. A model can have a small average absolute error and still be too low every time.
Using MAPE with values near zero. It explodes, and it favours under-prediction.
No baseline. An RMSE of 15 means nothing until you know what the average predicts.
Trusting probabilities that were never checked. A good AUC does not guarantee good calibration.
Calibrating on the training data. Fit the correction on held-out rows and check on others.
A cutoff of 0.5 by default. It is right only when both mistakes cost the same.
Applying the cost formula to uncalibrated scores. The cutoff is a probability, so the scores must be probabilities.
Forgetting capacity. A cutoff that flags more cases than anyone can review is not a decision.
Key takeaways
The error is predicted minus actual. Bias is its average, MAE the average absolute size, RMSE the square root of the average square, which punishes large misses, and MAPE the average percentage miss.
A gap between RMSE and MAE signals a few large errors. Always compare with a baseline such as predicting the average.
When over- and under-prediction cost differently, the best forecast level is the percentile given by the critical ratio, short ÷ (short + long).
Calibration checks that predicted probabilities match observed rates. Use a reliability table, the expected calibration error and the Brier score, and repair with Platt scaling or isotonic regression fitted on held-out data.
The best cutoff for calibrated probabilities is cost of a false positive ÷ (cost of a false positive + cost of a false negative). At a cost ratio of 20 to 1 it is 4.8 per cent, not 50.
Capacity, uncertain costs and drift matter, but the cutoff is a business decision that starts from costs.
Check yourself
1. Actuals 10, 20 and 30, predictions 12, 18 and 36. What are the MAE and the RMSE?
Errors +2, −2, +6. MAE = 10 ÷ 3 = 3.33. RMSE = √(44 ÷ 3) = √14.67 = 3.83.
2. A model has an MAE of 5 and an RMSE of 12. What does the gap suggest?
A few large errors dominate, since squaring magnifies them. Investigate those rows.
3. Being short costs ₹30 per unit and being long costs ₹10. What chance of meeting demand should you plan for?
30 ÷ 40 = 75 per cent, so stock the 75th percentile of demand.
4. Predicted probabilities for four cases are 0.8, 0.3, 0.6 and 0.1, and outcomes are 1, 0, 0 and 0. What is the Brier score?
Squared errors are 0.04, 0.09, 0.36 and 0.01, so the score is 0.50 ÷ 4 = 0.125.
5. A missed fraud costs ₹5,000 and a false alarm costs ₹100. What is the best cutoff for a calibrated model?
100 ÷ 5,100 = 0.0196, about 2 per cent.
Try this
Use a regression model and a classification model you have built, or fit simple ones on open data. Set aside forty-five minutes.
1. For the regression model, compute bias, MAE, RMSE and MAPE on a test set, and the RMSE of predicting the training average. Describe in a sentence how the model compares with the baseline.
2. Find the five rows with the largest errors. Say whether they explain a gap between RMSE and MAE, and what they have in common.
3. Decide what a too-high and a too-low prediction cost in your setting, and say which metric or quantile fits. Use the service-level simulator to find the percentile you would plan for.
4. For the classifier, build a reliability table with five to ten bands, compute the expected calibration error and the Brier score, and compare the Brier score with the score for predicting the average rate.
5. If the calibration is poor, fit an isotonic or Platt correction on a held-out set and check the table again on fresh rows.
6. Estimate the cost of a missed case and a false alarm. Compute the cost-based cutoff, and compare the total cost at that cutoff with the cost at 0.5 on your test set.
7. Write a short note: the metric you chose and why, the baseline, the calibration result, the cutoff and its cost.
A good result is a note where every metric is set against a baseline and the cutoff follows from stated costs. The last lesson of the module deals with the most dangerous problem in validation: results that look excellent because the future leaked into the past.
Tags: Data Science Regression metrics Calibration Brier score Cost-based threshold