Lesson 2.3: Bivariate analysis and finding risk drivers
Learn to relate each credit variable to default with risk tables, lift and two-way tables, and to avoid confounding traps such as Simpson's paradox.
By the end of this lesson you will be able to take a cleaned credit variable and measure how strongly it is linked to default: build a risk table by band, read bad rate and lift, check that the relationship moves in a sensible direction, test whether a difference between two groups is bigger than chance, and look at two variables together in a two-way table. You will also see how a variable can appear to raise risk overall and lower it within every subgroup, which is the trap called Simpson's paradox. You need Lesson 2.1 (the dataset) and Lesson 2.2 (profiling a single variable). The arithmetic is percentages and ratios.
Bivariate analysis looks at two variables at once. In credit risk, one of them is always the outcome, the good or bad label from Lesson 1.4, and the other is a candidate driver: a bureau score, an income band, a city, an employment type. The aim is practical: find out which variables separate good borrowers from bad ones, in which direction, and by how much. This is the evidence that decides which variables go into a scorecard, which are ignored, and which need more care.
The basic tool: a risk table
The risk table is the workhorse of credit analytics. Split a variable into bands, and for each band count the loans and the bad loans. Everything else follows from those two columns.
Using the 8,807-loan sample from Lesson 2.1, where 391 loans are bad, an overall bad rate of 4.4 per cent, group the bureau score into four bands.
Bureau score band · Loans · Bad loans · Bad rate · Share of loans · Share of all bad loans · Lift300 to 549 · 1,000 · 125 · 12.5% · 11.4% · 32.0% · 2.82
550 to 649 · 2,200 · 143 · 6.5% · 25.0% · 36.6% · 1.46
650 to 749 · 3,800 · 99 · 2.6% · 43.1% · 25.3% · 0.59
750 and above · 1,807 · 24 · 1.3% · 20.5% · 6.1% · 0.30
Total · 8,807 · 391 · 4.4% · 100% · 100% · 1.00
These numbers are illustrative, but they follow how real tables behave. Four columns are worth knowing precisely.
Bad rate is bad loans divided by loans in the band: 125 ÷ 1,000 = 12.5 per cent.
Share of loans is the band's loans divided by all loans: 1,000 ÷ 8,807 = 11.4 per cent.
Share of bad loans is the band's bad loans divided by all bad loans: 125 ÷ 391 = 32.0 per cent.
Lift is the band's bad rate divided by the overall bad rate: 12.5 ÷ 4.4 = 2.82. A lift of 2.82 means loans in the band are 2.82 times as likely to be bad as an average loan. A lift below 1 means safer than average.
Predict first: look at the bottom two bands together, the 650 and above group. What share of the loans do they hold, and what share of the bad loans? Work it out before reading on.
The 650-and-above bands hold 43.1 + 20.5 = 63.6 per cent of loans but only 25.3 + 6.1 = 31.4 per cent of bad loans. At the other end, the two lowest bands hold 36.4 per cent of loans (3,200 of 8,807) and 68.5 per cent of bad loans (268 of 391). More than two thirds of the defaults come from a little over a third of the borrowers. That is exactly what a lender wants to discover, because it is a place to act: tighter limits, higher prices or decline.
The chart shows the second thing to look for after strength: monotonicity. A variable is monotonic when its bad rate moves in one direction as the variable rises. Here the bad rate falls at every step. Monotone relationships are easy to explain, easy to defend to a regulator and stable over time. A pattern that rises, falls and rises again deserves suspicion. It might be real, or it might be noise from small bands, a data error, or an interaction with another variable.
Try the calculation with your own band.
How to choose the bands
Bands for a risk table are chosen so each one has enough loans and enough bad loans to give a stable rate. For a continuous variable, a simple start is to cut it into ten bands with equal numbers of loans, called deciles, and look at the bad rate in each. If the bad rate is erratic from decile to decile, merge neighbouring bands. For a categorical variable, each category is a band, with small ones grouped. As a rough guide, many teams want at least 30 to 50 bad loans in a band before trusting its rate, though the figure depends on the sample. A band of 40 loans with 2 bads has a bad rate of 5 per cent but is far too thin to rely on. Module 5 turns this into the formal binning and weight of evidence method.
python
import pandas as pd
data["band"] = pd.qcut(data["bureauscore"], q=10, duplicates="drop") # deciles
tbl = (data.groupby("band", observed=True)["target"]
.agg(loans="count", bads="sum"))
tbl["badrate"] = tbl["bads"] / tbl["loans"]
tbl["shareloans"] = tbl["loans"] / tbl["loans"].sum()
tbl["sharebads"] = tbl["bads"] / tbl["bads"].sum()
tbl["lift"] = tbl["badrate"] / data["target"].mean()
tbl
The duplicates="drop" option handles variables with many repeated values, where two cut points would coincide. After building the table, check the bad rate column from top to bottom for the direction and for any reversal.
Is the difference real?
A bad rate of 5.6 per cent in one group and 3.5 per cent in another looks like a difference, but how do you know it is not chance? Defaults are random events, so rates from finite samples wobble. A simple way to judge is to compute a margin of error for the difference of two proportions.
For two groups with bad rates p1 and p2 and sizes n1 and n2, the standard error of the difference is √(p1 × (1 − p1) ÷ n1 + p2 × (1 − p2) ÷ n2). A rough 95 per cent range is the difference plus or minus 1.96 times that. Suppose group 1 has 4,000 loans with 225 bad (5.6 per cent) and group 2 has 4,000 loans with 140 bad (3.5 per cent). The difference is 2.1 percentage points. The standard error is √(0.05625 × 0.94375 ÷ 4,000 + 0.035 × 0.965 ÷ 4,000) = √(1.327 × 10⁻⁵ + 0.844 × 10⁻⁵) = √(2.17 × 10⁻⁵) = 0.00466, so the margin is 1.96 × 0.00466 = 0.0091, about 0.9 percentage points. A difference of 2.1 with a margin of 0.9 is larger than chance would plausibly produce.
Reduce each group to 400 loans, keeping the same rates, and the margin widens to about 2.9 points, so the same gap becomes too small to trust. Sample size matters as much as the gap. For a formal version of this question, the chi-square test and the idea of a p-value appear in the Data Science track's statistics module; here, the margin of error is enough for a first screen.
Other ways to measure the link
A risk table suits categories and banded variables. A few other tools cover the rest.
Correlation with the target. For a numeric variable, a rank correlation such as Spearman's measures whether higher values go with more defaults, without assuming a straight line. It is a quick way to rank many variables, and it captures strength and direction in one number. It will miss patterns that are not monotone.
Comparison of distributions. Compare the distribution of a variable among bad loans and among good loans. If the two histograms overlap almost entirely, the variable does not separate them. If they are far apart, it does.
Information value. A summary of a risk table into one number, covered in Module 5, which ranks variables by their power to separate good from bad.
python
from scipy.stats import spearmanr
corr, p = spearmanr(data["bureauscore"], data["target"])
print(round(corr, 3)) # negative: higher scores go with fewer defaults
compare two groups, such as thin-file vs full-file customers
data.groupby(data["bureauscore"].isna())["target"].agg(["count", "mean"])
The second snippet is easy to forget and valuable: compare the missing group with the rest. As Lesson 2.2 noted, a missing bureau score often marks a thin-file customer, and the bad rate of that group may be different from every scored band.
Two variables at once
A single variable rarely tells the whole story. A two-way table shows the bad rate for combinations of two variables, and it reveals interactions: effects that depend on both. Take the fixed obligations to income ratio from Lesson 2.1 and the bureau score. The bad rates below are illustrative.
FOIR band · Score below 650 · Score 650 and aboveUnder 40% · 5.2% · 1.6%
40% to 60% · 8.1% · 2.7%
Over 60% · 13.4% · 4.9%
Both variables matter. Within each score group, the bad rate rises with FOIR. Within each FOIR band, a high score cuts the bad rate to roughly a third. The worst corner, high FOIR and low score, has a bad rate of 13.4 per cent, about 8 times that of the best corner at 1.6 per cent. This is the kind of table that guides policy: a lender might approve the top-right corner freely, price the middle carefully and decline the bottom-left. It is also a check on the model that follows, which should reproduce this ordering.
The trap: Simpson's paradox
Looking at two variables can mislead when a third variable is hiding. The classic case is called Simpson's paradox: a comparison that points one way for the whole group reverses within every subgroup.
Suppose a lender compares two channels. It has 4,000 loans through branches and 4,000 through a digital app. Borrowers differ by income. In branches, 1,000 low-income loans have a bad rate of 8 per cent (80 bad) and 3,000 high-income loans have 2 per cent (60 bad). Through digital, 3,000 low-income loans have 7 per cent (210 bad) and 1,000 high-income loans have 1.5 per cent (15 bad).
Overall, branches have 140 bad out of 4,000, a bad rate of 3.5 per cent. Digital has 225 out of 4,000, a rate of 5.6 per cent. Digital looks much riskier. Yet within each income group, digital is better: 7 per cent against 8 per cent for low-income, and 1.5 per cent against 2 per cent for high-income.
The paradox is resolved by looking at the mix. Digital loans are three quarters low-income, and low-income borrowers are risky in both channels. The channel is not the cause of the overall gap. The income mix is. Had the lender judged the app by the raw comparison, it might have cut a channel that is actually performing better. The lesson is general: before concluding that a variable drives risk, ask what else differs between the groups. In practice you check by splitting the comparison by the most likely confounders, such as income, product and region, or by fitting a model that includes them together. Logistic regression in Module 6 does the latter.
A worked example: ranking three variables
Suppose you have profiled three candidate variables and built a risk table for each from the same 8,807 loans, with 391 bad and an overall bad rate of 4.4 per cent. How would you compare them?
Variable · Lowest-risk band · Highest-risk band · Lift at the extremes · Shape · ConcernBureau score · 1.3% · 12.5% · 0.30 to 2.82 · Falls steadily · None obvious
FOIR · 2.1% · 11.0% · 0.47 to 2.48 · Rises steadily · Self-declared income
City tier · 4.0% · 4.9% · 0.90 to 1.10 · Flat · Little separation
The bureau score separates best: its extremes are nearly ten times apart in bad rate, 12.5 against 1.3 per cent, and the pattern is smooth. FOIR is also strong and monotone, with a gap of about five times. City tier barely moves the bad rate, with a lift range of 0.90 to 1.10, so alone it tells you little, although it might add something in combination with another variable. This ranking is the basis for shortlisting variables for the scorecard in Module 5. It does not replace the checks: confirm the bands have enough bad loans, repeat the tables by quarter, and look for a hidden third variable.
Notice also what the table does not say. A strong variable can still be unusable. If the bureau score is available only for customers with a credit history, it cannot be used for thin-file applicants, and a weak variable available for everyone may matter more in practice. Bivariate analysis tells you about strength. Availability, stability, fairness and cost of obtaining the variable are separate questions that a lender must also answer before putting a variable in a decision.
Reading a bivariate result well
A short checklist helps you move from a table to a decision.
1. Is the relationship strong? Look at lift at the extremes and the share of bad loans in the worst bands.
2. Is it monotone and sensible? Does it match business logic? A higher FOIR should mean higher risk. A reverse would need an explanation.
3. Is it based on enough loans? Check the number of bad loans in each band and the margin of error.
4. Is it stable? Repeat the table by origination quarter. A relationship that flips sign across quarters is unreliable.
5. Could a third variable explain it? Split by likely confounders.
6. Could it be a leak? A variable that separates good from bad almost perfectly deserves suspicion, because it may be known only after the outcome.
Common mistakes
Using bands with too few bad loans. A band of 40 loans with 2 bad loans has a rate that moves wildly. Merge until bands are stable.
Forgetting the direction. A strong but backwards relationship is a sign of a data problem or a hidden factor, not a discovery.
Treating a gap as proof without a margin. Differences between small groups can be chance.
Reading cause from a comparison. A raw gap can come from the mix of customers, as Simpson's paradox shows.
Ignoring the missing group. Compare blank values with the rest, since they are often a distinct segment.
Looking at only one variable at a time. Interactions matter, and two-way tables show them.
Selecting variables by their relationship to the target on the whole data. Do this on the development sample only, or the validation results will be flattered.
Key takeaways
A risk table gives, for each band, the number of loans, the bad rate, the share of loans, the share of bad loans and the lift.
A useful variable has a strong, monotone, sensible pattern based on enough loans, and one that is stable over time.
Check whether a difference is bigger than chance with a margin of error, and remember that sample size matters as much as the gap.
Rank correlation, comparison of distributions and information value are other ways to measure the link.
Two-way tables reveal interactions and guide policy. Simpson's paradox shows that a raw comparison can reverse within every subgroup when the mix differs.
Always ask what else differs between groups before calling a variable a cause of risk.
Check yourself
1. A band has 500 loans and 30 bad loans, in a sample with a 4 per cent overall bad rate. What are its bad rate and lift?
Bad rate 30 ÷ 500 = 6 per cent, and lift 6 ÷ 4 = 1.5.
2. Two bands hold 40 per cent of loans and 70 per cent of bad loans. What does this say?
They are much riskier than average, with roughly 1.75 times the average bad rate, and they are where most of the losses come from.
3. Why is monotonicity valued in a credit variable?
It is easy to explain, easy to defend to regulators and more stable over time than a pattern that rises and falls.
4. In Simpson's paradox, why can a channel look riskier overall but safer in every income group?
Its mix includes many more high-risk, low-income borrowers, which raises the overall rate even though it does better within each group.
5. Bad rates are 6 per cent in group A (500 loans) and 4 per cent in group B (500 loans). Is the gap clearly bigger than chance?
The margin is about 2.7 points, bigger than the 2-point gap, so it could easily be chance with groups this small.
Try this
Build a bivariate analysis on a small table. Use the dataset from Lesson 2.1, an open credit dataset or an invented one of at least 2,000 loans with a bad flag. Set aside forty-five minutes.
1. Pick three variables: one numeric such as a score, one categorical such as employment type and one ratio such as FOIR.
2. Build a risk table for each, with at least four bands. Compute bad rate, share of loans, share of bad loans and lift, and check that the columns add up to the totals.
3. Draw the bad rate by band as a bar chart. Mark each variable as monotone or not, and say what you would do about any reversal.
4. Pick two groups, such as two employment types, and use the difference simulator to say whether the gap is bigger than chance.
5. Make a two-way table of two of your variables and describe the best and worst corners.
6. Choose one comparison where a third variable might be hiding, such as product or channel against income, and test it by splitting.
7. Write a half-page summary: which variables look strong, which look weak, one result you are unsure of and what you would check next.
A good result is a short note in which every claim is backed by a table, a count of bad loans and a margin of error. The next lesson turns from variables to the portfolio as a whole, with the buckets and ratios that lenders report every month.
Tags: Credit Risk Analytics Bivariate analysis Risk drivers Lift Simpson's paradox