Lesson 3.1: The analytics workflow: CRISP-DM and exploratory analysis
Learn the six phases of CRISP-DM, how to turn a vague request into a testable question, and a step-by-step routine for exploratory data analysis.
By the end of this lesson you will be able to run a data project in the right order, turn a vague request into a question that data can answer, and explore a new table systematically before building anything on it. You will learn the six phases of CRISP-DM, the standard methodology for data projects, the questions each phase must answer, and a repeatable exploratory data analysis routine with a worked example on a loan table. You should have read Modules 1 and 2, since exploration uses cleaning, summaries, charts and the statistics of sampling error. The Python is a few lines of pandas, and the rest is a way of thinking.
Beginners often treat a data project as a modelling problem: get the data, fit an algorithm, report the accuracy. Experienced teams know that most failures happen before and after the algorithm. The model answers the wrong question, the data does not mean what everyone assumed, or the result is never used. A methodology is a checklist for these failure points, and the most widely used one is nearly three decades old because it still works.
CRISP-DM: six phases, in a loop
CRISP-DM stands for Cross-Industry Standard Process for Data Mining. It describes a data project as six phases. They are not a straight line. Teams move back and forth, and the whole process repeats as each result raises new questions.
Phase · The question it answers · Typical outputs1. Business understanding · What decision are we trying to improve, and how will we know? · A problem statement, a success metric, constraints
2. Data understanding · What data do we have, and what is it really like? · A data inventory, profile, first findings
3. Data preparation · How do we turn raw data into an analysis-ready table? · A clean table, features, a data dictionary
4. Modelling · Which methods can find the pattern? · Candidate models or analyses
5. Evaluation · Does it work, and does it meet the business goal? · Metrics, a review against the success metric, a go or no-go
6. Deployment · How is it used and kept working? · A report, dashboard, scoring service, monitoring plan
Predict first: in a typical data project, which phase do you think takes the largest share of the time? Decide before reading on.
Most practitioners report that data understanding and preparation together take more time than modelling, often well over half the project. This is why Module 1 of this course was about reading and cleaning tables, and why a team that rushes to modelling usually returns to cleaning later at greater cost. The loop matters too. Evaluation may show that the model is accurate but not useful, sending you back to business understanding. Deployment will reveal data problems that never appeared in the training extract.
CRISP-DM is not the only framework. It is simply the one most teams recognise.
Framework · Origin · EmphasisCRISP-DM · Industry consortium, late 1990s · Business-first, six phases, loops
KDD · Academic, early 1990s · Knowledge discovery: selection, preprocessing, transformation, mining, interpretation
SEMMA · SAS Institute · Sample, explore, modify, model, assess: tool-oriented, narrower on business
They overlap heavily. If you can explain CRISP-DM, you can follow the others.
Phase 1: business understanding, the phase people skip
The most valuable skill in this phase is turning a request into a question. A collections head asks: "Can you build something to reduce delinquency?" That is a goal, not a question. A good analyst asks what the team does today and what decision could be improved. After a short conversation the real situation emerges: the team has capacity for 200 calls a day against 3,000 overdue accounts, and calls are made in no particular order. The decision is whom to call first. The question becomes: "Can we rank overdue accounts by the chance they pay after a call, so that the 200 daily calls recover more money?"
Three things make a problem statement usable.
A decision: someone will do something differently because of the answer.
A success metric in business terms: rupees recovered per call-hour, not "model accuracy".
A baseline: what happens today, so that improvement can be measured.
Suppose the team calls 200 accounts a day with an average overdue amount of ₹8,000, and about 12 per cent of calls today lead to payment. A ranking that puts the likelier payers first might raise that to 20 per cent. The recovered amount per day rises from 200 × 0.12 × 8,000 = ₹1,92,000 to 200 × 0.20 × 8,000 = ₹3,20,000, which is ₹1,28,000 more each day, or about ₹3.2 crore over 250 working days. Writing this arithmetic down at the start does two things: it shows whether the project is worth doing, and it fixes the number against which success will be judged.
Three requests, reframed
The same move works on almost any request. Here are three that analysts hear every week, with the questions that turn each into a project.
The request · The decision behind it · A usable success metric · A baseline to find out"Reduce churn." · Which customers to contact, and with what offer · Customers retained per 1,000 contacted, net of offer cost · Retention among customers contacted without a ranking
"Build a fraud model." · Which transactions to hold for review · Fraud rupees stopped per review-hour, with a cap on genuine customers blocked · Fraud caught by today's rules, and the review queue size
"Understand why sales dropped." · Where to spend effort this quarter · The share of the drop explained by identifiable segments · Sales by region, product and week for a year earlier
Notice that only the third request is a description, and even there the useful version names a decision. In each case the baseline is something you must go and find out, and finding it is often the first valuable result: teams are sometimes surprised to learn how little today's process achieves, or how much.
A closing check for this phase is to write the sentence "If the answer is X we will do A, and if it is Y we will do B." If you cannot complete it, the analysis has no decision attached, and the best next step is another conversation, not more data.
A final element of this phase is constraints: what the data allows, what rules apply, how the answer will reach the user. A model that needs data the collections team will not have at call time is useless however accurate it is.
Phase 2: data understanding
Here you meet the data. The questions are practical. Where does each table come from? What does one row stand for, which is the grain of Lesson 1.1? Which time period does it cover? How were the fields captured, and by whom? What is missing and why? Most important for any prediction project: what was known at the time of the decision? A column filled in after an account defaults must not be used to predict default, a trap Lesson 6.4 treats in detail.
The tool of this phase is exploratory analysis, which we turn to shortly. Its product is a short written understanding of the data: what it can and cannot support.
Phase 3: data preparation
Preparation covers the cleaning of Lesson 1.2 and the construction of the analysis table: joining sources, choosing the grain, deriving fields such as "days overdue" or "ratio of loan to income", and handling missing values. Record each step as code. A prepared table that cannot be rebuilt is a liability, because the project will be repeated on new data.
Phase 4: modelling
Modelling means applying a method to find the pattern. It may be a simple group comparison, a regression or a machine-learning model. The choice follows from the question: description, explanation, prediction or segmentation call for different tools, and later modules cover them. A habit worth building now is to start with the simplest baseline, such as the overall average or a one-rule split, so that every later model is judged against something concrete.
Phase 5: evaluation
Evaluation asks two questions, and teams often answer only the first. Is the model technically sound? Does it meet the business success metric? A model that ranks accounts well in a test table may still not raise recovery if the calls cannot be redirected. Evaluate on data the model has not seen, against the baseline, and in the units of the business metric. Also review what was learned: did the process reveal anything that changes the original question?
Phase 6: deployment
Deployment is not the end. It means putting the result into a form people use, such as a weekly ranked list, a dashboard or a service called by another system, and then monitoring that it keeps working. Data drifts, behaviour changes, and a model that was right in March can be wrong by September. Lesson 8.2 returns to this. Plan for deployment from phase 1, because a result nobody can use has no value.
Exploratory data analysis: a routine
Exploratory data analysis, or EDA, is the disciplined looking that happens in phases 2 and 3. Its purpose is to understand structure, find problems and form questions, not to prove anything. A practical order keeps it from turning into random plotting.
1. Shape and grain. How many rows and columns? What does one row stand for? What is the time range?
2. Quality. Types, blanks, duplicates, impossible values, as in Lesson 1.2.
3. One variable at a time. For each numeric column, the summaries and a histogram. For each category, the counts and shares. Look for skew, odd peaks and rare categories.
4. The outcome. If you are predicting something, such as default, study it on its own: how common is it, and how does it change over time?
5. Variables against the outcome. For each candidate input, compare the outcome across its groups or ranges.
6. Variables against each other. Correlations and cross-tabulations, to find redundant or confounded columns.
7. Time and segments. Does the pattern hold in each month, region or product? A pattern that appears only in one segment is a different finding.
8. Write down findings and questions. Each plot should end in a sentence.
A worked example
A lender has a table of 5,000 past personal loans, with a column default that is 1 if the loan went bad. The first step finds 301 defaults, so the overall default rate is 301 ÷ 5,000 = 6.0 per cent. That figure is the baseline for everything that follows: a segment with a rate of 6 per cent is unremarkable.
Now compare the outcome across employment types. Among 3,000 salaried borrowers there were 135 defaults, a rate of 4.5 per cent. Among 1,200 self-employed borrowers there were 102 defaults, 8.5 per cent. Among 800 business owners there were 64, or 8.0 per cent. The three numbers add to 301, which matches the total, a check you should always make.
Next look at income. Among 800 borrowers earning under ₹20,000 a month there were 76 defaults, 9.5 per cent. Among 2,200 earning ₹20,000 to ₹40,000 there were 143, 6.5 per cent. Among 1,400 earning ₹40,000 to ₹80,000 there were 63, 4.5 per cent. Among 600 earning over ₹80,000 there were 19, 3.2 per cent. Again 76 + 143 + 63 + 19 = 301.
The chart shows a clear, steady pattern, the kind you hope to find. Three cautions apply before you believe it. First, how reliable is each bar? The top income band has only 600 loans, so its rate of 3.2 per cent is less certain than the 6.5 per cent from 2,200 loans. Using the standard error of a proportion from Lesson 2.3, the margin of error for 3.2 per cent from 600 loans is 1.96 × √(0.032 × 0.968 ÷ 600) = 1.4 points, so the true rate could plausibly be anywhere from about 1.8 to 4.6 per cent. Second, is income standing in for something else, such as employment type, where self-employed borrowers have lower and more irregular incomes? Third, does the pattern hold in each year of the data?
The employment comparison needs the same care. A gap between 4.5 per cent for salaried and 8.5 per cent for self-employed borrowers rests on 3,000 and 1,200 loans, so it is large relative to the noise. The gap between self-employed at 8.5 per cent and business owners at 8.0 per cent, on 1,200 and 800 loans, is well within the margin and should not be treated as a finding. A good EDA habit is to put the count and an interval beside every group rate.
In pandas, most of this routine is a few lines.
python
import pandas as pd
df = pd.readcsv("loans.csv", parsedates=["issuedate"]) # so the .dt date tools work later
df.shape, df.dtypes
df["default"].mean() # overall rate: the baseline
df.groupby("employment")["default"].agg(["count", "sum", "mean"])
df["incomeband"] = pd.cut(df["income"], [0, 20000, 40000, 80000, 1e9],
labels=["<20k", "20-40k", "40-80k", "80k+"])
df.groupby("incomeband", observed=True)["default"].agg(["count", "mean"])
pd.crosstab(df["employment"], df["incomeband"], normalize="index") # are they linked?
df.groupby(df["issuedate"].dt.toperiod("Q"))["default"].mean() # stable over time?
Notice what each line is: a question. The last two questions are the important ones, because they test whether the pattern is confounded, and whether it is stable across time.
What EDA should leave you with
At the end of EDA you should be able to write a half-page summary with five parts: what the table is, what you cleaned and why, the three or four strongest patterns with their sizes and uncertainty, the traps you found (leakage candidates, small segments, shifts over time), and the questions you would now ask. This summary is the deliverable of phases 2 and 3, and it is what prevents the modelling phase from starting on false assumptions.
· Exploratory analysis · Confirmatory analysisPurpose · Find patterns and questions · Test a stated hypothesis
When · Before modelling · After the hypothesis is set
Risk · Seeing patterns that are chance · Testing too narrow a question
Evidence standard · Suggestive · Held to the tests of Lesson 2.3
The risk in the first column is real. If you look at fifty segments and report the largest gap, you have run fifty tests, as Lesson 2.3 warned. Treat exploratory findings as leads, and confirm important ones on fresh data or with a planned test.
Common mistakes
Starting with the algorithm. Begin with the decision and the success metric. A model without a decision is a demonstration.
A success metric nobody cares about. "Higher accuracy" is not a business outcome. Use money, time or risk.
No baseline. Without today's performance, you cannot say whether anything improved.
Skipping the grain and the timing. Know what a row is, and what was known when the decision was made.
Plotting without a plan. Follow the routine, and write a sentence for each chart.
Ignoring group sizes. A rate from 60 loans is not comparable to one from 6,000.
Mistaking exploration for proof. Patterns found by looking need confirming.
Forgetting deployment. Ask in phase 1 who will use the result and how.
Key takeaways
CRISP-DM has six phases: business understanding, data understanding, data preparation, modelling, evaluation and deployment, and it loops. Data understanding and preparation usually take the most time.
A usable problem statement names a decision, a business success metric and a baseline. Work out what improvement is worth before you start.
Always ask what was known at the time of the decision. A column filled in later cannot be used to predict.
EDA follows a routine: shape and grain, quality, single variables, the outcome, variables against the outcome and each other, time and segments, then written findings.
Put the count and an interval beside every group rate. Small segments are noisy, and a gap inside the margin of error is not a finding.
Exploration finds leads, and confirmation needs fresh data or a planned test. Plan deployment and monitoring from the start.
Check yourself
1. A manager asks you to "reduce churn". What would you ask to turn this into a data question?
What decision will change: who to contact, what to offer, when? How is churn defined, and what would success be worth in rupees? What happens today as a baseline?
2. A team is called 200 times a day today with a 12 per cent recovery rate on ₹8,000 accounts. A ranked list lifts the rate to 18 per cent. What is the extra daily recovery?
200 × 8,000 × 0.06 = ₹96,000 a day.
3. A segment of 400 loans has a default rate of 5 per cent. What is its approximate 95 per cent margin of error?
1.96 × √(0.05 × 0.95 ÷ 400) = 1.96 × 0.0109 = 2.1 points, so roughly 2.9 to 7.1 per cent.
4. Why must you check whether a pattern holds in each time period?
A pattern that appears only in some periods may be a one-off or may be shifting, and a model built on it would fail.
5. Which phase of CRISP-DM do you return to if a model is accurate but cannot be used by the team that needs it?
Business understanding, because the problem or its constraints were not framed correctly.
Try this
Choose a project you could do at work, or use any open dataset with an outcome column, such as default, churn or a returned order. Set aside forty-five minutes.
1. Write the problem statement: the decision, the metric in business terms, the baseline today and what an improvement would be worth in rupees. Use the call-list simulator if it helps.
2. List the tables you would need and the grain of each. For each column you might use, note whether it would be known at the time of the decision.
3. Run the EDA routine on the data: shape, quality, one variable at a time, the outcome, the variables against the outcome.
4. For three groups, compute the rate, the count and a margin of error. Mark any difference that falls inside the margin.
5. Check whether your strongest pattern holds in each half of the time range, or in two regions.
6. Write the half-page summary: what the data is, what you cleaned, the strongest patterns with sizes and uncertainty, the traps, and your next questions.
7. Decide what the deployment would look like and who would use it.
A good result is a half-page that a colleague can act on, with every claim carrying a count and a range. The next lesson teaches SQL, the language in which most analysts pull and reshape the tables that this workflow depends on.
Tags: Data Science CRISP-DM Exploratory data analysis Analytics workflow Methodology