Lesson 2: Finding real problems worth solving
Many failed products solve a problem nobody had. Learn to split problems from solutions, find evidence, size a problem in rupees and know when you have enough.
The most expensive mistake in product work is building something well that nobody needed. It happens in large companies and small ones, and it usually starts the same way: someone had an idea, it sounded sensible in a meeting, and the team moved straight to building. By the end of this lesson you will be able to tell a problem from a solution in a request, trace a symptom back to its cause, gather evidence from at least four places, size a problem in rupees and in people, judge how much evidence is enough before you commit a team, and write down what you found in a form others can challenge. You should have read the first lesson on what a product manager does. We will keep using the bill-payment app from that lesson, a consumer app where people pay electricity, water, broadband and mobile bills, and we will add a second example from a small lender. All the numbers are illustrative, and you can change them with the sliders.
Problems and solutions are not the same thing
Listen to how requests reach a product manager. A senior leader says, "We need a dark mode." A sales head says, "Customers want a bulk upload button." An engineer says, "We should rewrite the payment screen." A user writes, "Please add a reminder feature." Every one of these is a solution: a specific thing to build. None of them says what is going wrong for a person.
A problem is a gap between what a person is trying to do and what happens when they try. "I pay my electricity bill on the last day and sometimes forget, so I pay a late fee" is a problem. A reminder feature is one possible solution to it. So is automatic payment, so is a bill that arrives with a due date in large print, and so is a text message the day before. Once you see the problem, you find that there are several solutions with different costs, and that the one first proposed may not be the best.
This matters for a practical reason. If you accept the solution as stated, you stop thinking. You build the reminder feature, you ship it, and you may discover that people who forget are not helped by yet another notification from yet another app. If you hold on to the problem, you can test whether it is real, how big it is and which solution fits.
Predict first: a customer of the bill-payment app writes, "Add a calendar view of my bills." Is this a problem or a solution? And what question would you ask them next? Decide before reading on.
It is a solution. A good next question is not "what should the calendar look like" but "tell me about the last time you missed or nearly missed a bill. What happened?" The customer's story will show whether their issue is forgetting, not knowing the amount, not having money on the day or something else. A calendar might help with the first, do nothing for the second and be irrelevant to the third.
A simple habit: whenever you hear a request, write it in this form: "[Person] is trying to [do something] but [obstacle], which results in [consequence]." If you cannot fill in all three parts, you have a solution, not a problem, and you have more questions to ask. For the calendar request, the sentence might become: "A salaried tenant is trying to pay five different bills before their due dates but cannot see them in one place, which results in a late fee of around Rs 50 to Rs 150 about once a quarter." Now there is something to measure.
Symptoms and causes
Even a well-stated problem can sit on top of another one. When a team sees that many users abandon a payment halfway, the abandonment is a symptom. The cause is somewhere underneath, and fixing the symptom without the cause tends to move the trouble elsewhere.
A useful tool is the five whys: you ask why something happened, then why that happened, and keep going until you reach something you can act on. You rarely need exactly five, but the habit of not stopping at the first answer is the point. Here is an example from the bill-payment app.
Why do users abandon the payment screen? Many of them see an error and leave.
Why do they see an error? The payment fails when they use a particular bank's net banking.
Why does it fail for that bank? The confirmation from the bank arrives after our screen has timed out.
Why does the screen time out so early? The timeout was set when most payments were completed through another method that is much faster.
Why was it never revisited? Nobody owns the timeout, and the failure only shows up in an overall abandonment rate that nobody breaks down by bank.
Notice where this ends up. The first problem sounded like a design problem on the payment screen. The real causes are a timeout setting and a gap in who looks at the data. A redesign of the screen would have cost weeks and changed nothing. Changing the timeout and adding a breakdown by bank to the weekly report might take days.
Three cautions about the five whys. First, each answer must be something you have checked, not something you believe. Second, there can be more than one chain, since several things can cause the same symptom. Third, stop when you reach something the team can change. Asking "why" until you get to the state of the economy is not useful.
Where evidence comes from
You will rarely find a problem by sitting and thinking. You find it by looking at what people actually do and say. No single source is enough, because each has its own blind spot. Good product managers draw on at least four, and look for places where they agree or disagree.
Support tickets and call logs. These are the cheapest source, because people have already told you in their own words what went wrong. Their blind spot is that only some people complain. Many just leave.
Product data. Funnels, drop-offs, time taken, failures by device or bank. This shows what people do, at scale. Its blind spot is that it shows what and not why.
Interviews and observation. Talking to people about their last real experience, or watching them use the product. This gives you the why. Its blind spot is that you can only talk to a handful of people.
Front-line staff. Customer care, sales, collections agents and branch staff hear problems daily and have patterns in their heads. Their blind spot is that they remember the dramatic cases and hear mostly from the people who reach them.
Competitors and the wider world. Reviews of other apps, regulator notices, news. This shows problems your users may have that you have not yet seen.
The most useful distinction among these is between what people say and what people do. Asked in a survey, many people will say they would use a budgeting feature. Product data later shows that few open it. Stated interest is cheap, and behaviour costs something, such as time, effort or money. When the two differ, believe behaviour, and then use interviews to find out why.
A worked example: counting complaints
Suppose the bill-payment app's team reads the last 200 support tickets that mention a payment problem and sorts each into one main category. They find that 62 say the biller could not be found, 48 say a payment failed or was debited without confirmation, 31 are about a fee that was unclear, 24 are about slow confirmation, and 35 are about other things. Here is the picture.
The chart tells the team where to look first. "Biller not found" accounts for 31 per cent of tickets, and failures for 24 per cent, so together they cover 55 per cent. But the chart also raises questions that it cannot answer. Are 62 people a lot, or a tiny share of all users? Is "not found" annoying or does it make people give up? Do failed payments cost more, since money is involved and trust is at stake? Counting tickets is where analysis starts, not where it ends. To go further, you need to size the problem.
Sizing a problem
Before you commit a team, you want a rough idea of how big the problem is. Not a precise number, but enough to compare it with other problems and with the cost of fixing it. Three questions give you most of what you need.
1. Reach. How many people hit this problem? Count users, not incidents, and say over what period.
2. Frequency. How often does each affected person hit it? Something that happens once in a lifetime matters less than something that happens weekly.
3. Severity. What does it cost when it happens? In rupees to the user or to the company, in minutes lost, in trust damaged or in people who give up and never return.
Multiply the three and you have a rough size. Add the cost of an incident, and you have a rupee figure. Here is the arithmetic for the failed-payment problem. Suppose the app has 100,000 users who pay at least one bill a month. Suppose 8 per cent of them hit a payment failure in a month, which is 8,000 people. Suppose each of those has 1.5 failures a month on average, which is 12,000 failures. Each failure leads to a support contact costing the company about Rs 45 to handle, plus an assumed Rs 20 of lost fee income when a user switches to another app and does not come back that time. So each failure costs about Rs 65, and the monthly cost is 12,000 × Rs 65, which is Rs 7.8 lakh, or about Rs 93.6 lakh a year.
Try your own numbers below. The first output shows the yearly cost of the problem as it stands, and the second shows the part of that you could save if a fix removed some share of the failures.
At the defaults the yearly cost is about Rs 93.6 lakh, and a fix that removes 70 per cent of failures would save about Rs 65.5 lakh a year. Remember what this number leaves out. It does not count people who tell friends the app is unreliable, or users who stop paying through the app altogether. When severity includes trust, the true cost is usually higher than a simple sum shows. On the other side, the sum assumes the problem will keep affecting the same number of people, and it may shrink by itself, for example if a bank upgrades its system.
The value of a sizing exercise is not that the answer is right. It is that it forces you to write down your assumptions, so that others can say "I think the 8 per cent is too high" and you can go and check. A number with visible assumptions is more useful than a confident feeling.
Comparing problems on one page
A table helps you set problems beside each other. These rows use invented figures for the same app.
Problem · Users hit per month · Times each · Cost per incident · Yearly cost (illustrative)Payment failed or debited without confirmation · 8,000 · 1.5 · Rs 65 · Rs 93.6 lakh
Biller not found · 12,000 · 1.0 · Rs 20 · Rs 28.8 lakh
Unclear fee · 5,000 · 1.0 · Rs 25 · Rs 15.0 lakh
On the money alone, failed payments come first, even though "biller not found" was the more common ticket. This is why you size problems instead of ranking them by how often people mention them. A frequent small irritation can cost less than a rare serious failure. Also notice what the table does not say. If "biller not found" makes new users give up before their first payment, it may cost far more than the table shows, because those users never become regular. When you size a problem, ask who is affected as well as how many. A problem that hits your most valuable or newest users deserves extra weight.
How much evidence is enough?
There is no number of interviews that makes you certain. But there is a useful way of thinking about how many you need, and it comes from simple probability.
Suppose a problem affects 20 per cent of your users, which is one in five. If you talk to one user, the chance they have the problem is 20 per cent. If you talk to five, the chance that at least one has it is not 100 per cent. It is 1 minus the chance that none of them do, which is 1 − 0.8 × 0.8 × 0.8 × 0.8 × 0.8, about 67 per cent. Talk to ten and the chance rises to about 89 per cent. For a rarer problem, affecting 5 per cent, ten interviews give only a 40 per cent chance of meeting even one person who has it.
This has two lessons. A common rule of thumb says that a handful of interviews show you most of the big problems, and the arithmetic backs this up for problems that affect many people. But a few interviews cannot reliably find a rare problem, and cannot tell you how common something is. For that, you need data from many users, such as tickets or product logs. Interviews tell you what the problems are and why. Data tells you how many.
With the defaults, five interviews give a 67 per cent chance of meeting someone with a problem that affects one in five users, and about 11 are needed for a 90 per cent chance. Drop the share to 5 per cent and the number needed for a 90 per cent chance rises to about 45. Use this to decide how to look. If you suspect a common problem, interviews are efficient. If you suspect a rare but serious one, go to the data.
Signs you have enough
You never have complete evidence. A reasonable stopping point is when most of these are true.
You can describe the problem in the "[Person] is trying to [do something] but [obstacle], which results in [consequence]" form, with real examples for each part.
At least two different sources point the same way, such as tickets and product data.
You have a rough size, with your assumptions written down.
New interviews are repeating what earlier ones said and are no longer surprising you.
You know what you do not know, and it is a short list.
Talking to people without leading them
How you ask matters as much as whom you ask. Two simple rules help.
Ask about the past, not the future. "Would you use a bill reminder?" invites a polite yes. "Tell me about the last time you paid a bill late" gets a real story. People are poor at predicting their own behaviour and good at remembering what happened. The best questions begin with "tell me about the last time".
Do not pitch. If you describe your idea and ask what people think, they will often be kind. Instead, find out what they do today, what they tried, what annoyed them and what they did about it. If someone has already invented a workaround, such as a note in their phone, a spreadsheet or a message to themselves, that is a strong sign the problem is real, because they cared enough to act.
A third habit is to listen for the money and the time. When a person says "it's annoying", ask what it cost: how long it took, what they paid, what they did instead. Concrete answers are what you can size.
A second example: a small lender's collections app
A small lender's product manager hears from the collections head: "We need a dashboard that shows every borrower's call history." That is a solution. She asks the head to describe the last week. Agents, it turns out, spend a lot of time before each call working out whether the borrower has already promised a payment date. The call history exists, but it is spread across three screens.
She checks this against data and finds that agents spend an average of 4 minutes per call preparing, and make about 40 calls a day. She asks the agents too, and several mention a related problem: they sometimes call borrowers who have already paid that morning, which annoys the borrower and wastes a call. Now she has two linked problems: slow preparation and stale payment status. A dashboard of call history would help with the first, and not at all with the second. The solution she designs is a single screen that shows the latest promise and the latest payment. Had she built the original request she would have delivered what was asked and missed half of what was wrong.
Writing the problem down
The final step is to put what you found on one page where others can disagree with it. This is the job of a problem statement, which later lessons cover in detail. For now, a short version has five parts.
1. Who has the problem, described precisely.
2. What they are trying to do and what gets in the way.
3. Evidence, with the sources and dates.
4. Size, with your assumptions visible.
5. What you do not know yet, and how you would find out.
Writing the last part is a sign of maturity. A document that claims certainty invites argument about the wrong things. A document that says "we think the failure rate is 8 per cent, based on one month of data; we have not checked whether it is seasonal" invites people to help.
Common mistakes
Accepting a solution as a problem. Always go back to the person and the obstacle.
Stopping at the first cause. Ask why until you reach something the team can change.
Counting complaints as if they were the whole population. Silent users who leave are missing from your tickets.
Believing what people say they will do. Weigh behaviour above stated interest.
Leading the witness. Pitching an idea in an interview produces polite agreement, not information.
Using only one source. Tickets, data, interviews and front-line staff each have blind spots.
Ranking by frequency alone. A rarer, more serious problem can cost more than a common small one.
Hiding assumptions in a sizing. If others cannot see the inputs, they cannot improve them.
Falling in love with the first problem you find. Look at two or three before you commit.
Key takeaways
A request is usually a solution; the problem is the gap between what a person is trying to do and what happens.
Write each problem as "[Person] is trying to [do something] but [obstacle], which results in [consequence]".
Use the five whys to move from a symptom to a cause the team can change.
Draw on at least four sources of evidence, and prefer what people do over what they say.
Size a problem with reach, frequency and severity, and keep your assumptions visible.
A few interviews can find common problems but not rare ones; use data to learn how many people are affected.
Check yourself
1. A sales head asks for a bulk upload button. Is that a problem or a solution, and what would you ask next?
A solution; ask about the last time a customer needed to add many items and what happened, to find the real obstacle.
2. Why can a rare, serious failure outrank a common, minor irritation?
Sizing multiplies reach, frequency and severity, so a high cost per incident can outweigh a larger number of cheap ones.
3. If a problem affects 5 per cent of users, why might ten interviews miss it?
Each interview has only a 5 per cent chance of meeting an affected person, so ten give roughly a 40 per cent chance of meeting any.
4. What is the weakness of relying only on support tickets?
Only some people complain; many users who have the problem just leave without writing.
5. Why is "tell me about the last time" a better question than "would you use this?"
People predict their own behaviour badly but remember real events well, so a past story gives facts instead of polite agreement.
6. Name two things you should write down alongside a sizing figure.
The assumptions behind each input and the sources of the numbers, so others can challenge and improve them.
Try this
Set aside about forty minutes and pick a real product you use, such as a food delivery app, a banking app or the app of your own employer.
1. Write down three complaints or wishes you have heard about the product, from friends, reviews or your own experience. Rewrite each in the "[Person] is trying to [do something] but [obstacle], which results in [consequence]" form. If you cannot, mark it as a solution and write the question you would ask.
2. Choose one problem and run a five-whys chain on paper. For each answer, note whether it is something you know or something you are guessing.
3. List the four sources of evidence you would use, and for each write one concrete way you would get at it within a week.
4. Using the sizing simulator, enter your best guesses for reach, frequency and cost per incident. Write down the yearly figure, and mark the one input you are least sure about.
5. Using the interview simulator, decide how many people you would need to talk to for a 90 per cent chance of meeting someone with the problem, given your guess of how common it is.
6. Write five "tell me about the last time" questions you would ask, and cross out any that suggest an answer.
7. Write a five-part problem statement: who, what, evidence, size and what you do not know yet.
A good result is a one-page problem statement that a colleague could read in two minutes and challenge, with a clear sentence about the problem, a rupee estimate with visible assumptions, and an honest list of what you still need to find out.
Tags: Product Management Discovery User research Problem solving Beginner