Lesson 1.3: Build, buy or fine-tune a model
Before training anything, try the cheapest options first. A practical ladder from prompting to retrieval to fine-tuning, and when each one is worth the cost.
Teams new to AI often jump to the most dramatic solution: train our own model. That is usually the slowest and most expensive way to find out whether an idea works. This lesson gives you a simple ladder of options, ordered from cheap and quick to costly and slow, so you climb only as high as you need. By the end you will be able to name the four rungs, say what problem each one fixes, choose the right rung for a given situation, and use a calculation to see whether a step up pays for itself. You need no machine learning background. If you have read the first two lessons of this track, you already have the vocabulary.
The decision matters because the rungs differ enormously in cost. A prompt can be written in an afternoon. A retrieval system takes days or weeks. A fine-tuned model takes weeks of preparing examples and keeps costing effort after launch. Training a model from nothing takes a research budget. Choosing the wrong rung early wastes months. Choosing the right one often means you are done by the end of the week.
Why the ladder exists
When a model gives a poor answer, the instinct is to reach for the biggest tool. It feels as though a custom model would "know our business". But a poor answer has a cause, and different causes need different fixes. If the model is given no information about your policy, no amount of training on your tone will help. If the model has the information but writes in the wrong format, you do not need new facts, you need clearer instructions or examples.
The ladder turns that insight into a habit: find out what is actually wrong, fix it with the cheapest tool that can fix it, measure the result and climb only if it is still not good enough. Teams that follow the habit usually get working products faster and spend far less. Teams that skip it often end up with an expensive model that still has the original problem, because the original problem was never the model's knowledge.
The four rungs
Start at the bottom and move up only when the step below is not good enough.
1. Use a hosted model with a good prompt. Write clear instructions and a few examples. This takes hours, costs little, and solves many problems.
2. Add your own information at request time. If the model lacks facts about your business, find the relevant documents and place them in the request. This is called retrieval and later lessons cover it fully.
3. Fine-tune a model. Continue training an existing model on your own examples so it adopts a style, format or narrow skill. This takes data, time and care.
4. Train a model from scratch. This needs huge amounts of data, compute and expertise. For almost every company it is the wrong choice.
Here is how much effort each rung might take for a small team. The numbers are illustrations of the order of magnitude, not measurements, and they vary widely with the problem and the team.
The steepness is the point. Each rung costs several times more effort than the one below, which is why the order matters. A prompt that gets you most of the way in two days is worth far more than a fine-tune that gets you a little further in two months.
Rung 1: a good prompt
The bottom rung is more powerful than it looks. A prompt is the instruction and the examples you send with each request, and a good one can change the quality dramatically. A strong prompt usually does several things at once.
It says who the model is acting as and for whom, for example "You are a support assistant for a bank's retail customers."
It states the task precisely: what to do, what to leave out and what to do when unsure, such as "If the answer is not in the text, say you do not know."
It specifies the format of the answer, such as three short bullet points or a fixed set of fields.
It includes two or three examples of a good answer, which often fix formatting and tone more reliably than a long description.
Many teams that think they need fine-tuning discover that they were using a vague one-line prompt. Before moving up, write the best prompt you can, test it on a set of real examples and measure. Only then do you know whether the prompt rung is truly exhausted.
Rung 2: retrieval
If the model gives vague answers because it lacks facts about your business, a better prompt will not help. The model has never seen your policies, product catalogue, customer history or yesterday's news. The fix is to supply them: find the relevant pieces of your own documents and place them in the request.
This is retrieval, and it is how most business assistants get their facts. It has a decisive advantage: the facts live in your documents, not in the model, so when a policy changes you update the document and the very next answer is correct. Later modules show how to build it. For now, remember the rule of thumb: facts belong in retrieval.
Rung 3: fine-tuning
Fine-tuning means taking an existing model and training it a little more on your own examples, usually pairs of "input" and "the answer we want". After that, the model has been adjusted to behave the way the examples do.
It is good at changing behaviour: a consistent tone of voice, a strict output format, a narrow task done the same way every time, or a skill that is hard to describe in words but easy to show in examples. It is a poor way to add facts. A fine-tuned model that learned last quarter's interest rates will keep quoting them until someone retrains it, and you cannot easily point to where it learned a fact or correct one fact without disturbing others.
Fine-tuning also has ongoing costs that newcomers underestimate. You need a few hundred to a few thousand good examples, and each one has to be correct, which means a person checking them. You need a way to test that the tuned model is better and has not become worse at other things. And when the base model is replaced by a newer one, you may need to tune again.
Rung 4: training from scratch
Training a general model from nothing needs vast amounts of data, large amounts of computing power and a team of specialists, and it is only sensible for a handful of organisations whose business is building models. For an ordinary company, it is the wrong choice almost every time. If you find yourself arguing for it, check that the problem is not one of the lower rungs.
A side branch: running an open model yourself
Separate from the ladder is a question of where the model runs. Sometimes data cannot leave your premises, or a rule or a customer forbids sending it to an outside service. In that case you may run an openly available model on your own servers or in your own cloud account. It does not replace the ladder: you still start with prompts, then retrieval, then perhaps fine-tuning, but on a model you host. It adds work, because you now operate the model, and it does not make the model better, only more private. If you work in a regulated industry, check what your regulator and your customers require before deciding.
What each step fixes
It helps to ask what is actually wrong. A short table connects symptoms to rungs.
What you see · What is probably wrong · Rung to tryVague, rambling or badly formatted answers · The instructions are unclear · Better prompt, with examples
Wrong or invented facts about your business · The model has never seen your information · Retrieval
Facts that change often, such as rates · The facts go stale inside a model · Retrieval, keeping the documents current
Correct facts but the wrong tone or format, even with examples · A behaviour is hard to describe but easy to show · Fine-tuning
One narrow task done millions of times, where cost matters · A smaller tuned model may match a big one more cheaply · Fine-tuning
Data cannot leave your control · A privacy limit, not a quality problem · An open model you host, plus the lower rungs
Pick a problem below to see which rung it points to.
Predict first: a bank wants its assistant to quote current interest rates, which change every month. Which rung fits best? Decide before reading on.
Retrieval. The rates change monthly, so you want the current rate sheet placed into each request. A fine-tuned model would keep quoting whatever rates it saw during training until someone retrained it.
Worked examples
A bank's rate assistant
This is the example above. The bank's rates change every month and must be quoted exactly. The right design puts the current rate sheet into the request. A prompt rule says "quote only rates that appear in the text; if a rate is missing, say you will check". An output check compares any rate in the answer with the sheet. No fine-tuning is needed, and the monthly update is a file replacement.
A support team that wants a consistent voice
A company's support replies must sound the same whoever the customer is: warm but brief, never promising what cannot be delivered, always ending with a clear next step. The first attempt, a long prompt describing the tone, works about three times in four. The team collects 500 past replies that a senior manager rated as ideal, and fine-tunes a model on them. The tuned model gets the tone right far more often and needs a shorter prompt. Here fine-tuning is the right rung, because the problem is a behaviour that is easier to show than to describe, not a lack of facts. The facts, such as order status, still come from retrieval.
A contract reviewer
A legal team wants to flag clauses that shift liability in a contract, and it has thousands of past contracts with the clauses already marked by lawyers. The task is narrow, repeated and well-defined, and the examples are plentiful and trusted. A first version uses prompting with three examples and gets decent results. The team then tests a fine-tuned smaller model on the marked contracts, which is cheaper per document and as accurate on their test set. The decision to move up the ladder came from a measurement, not a feeling.
An HR assistant
An HR assistant should answer questions about leave and benefits. The facts are in the policy pack, and they change when the policy changes. This is retrieval from the start. Fine-tuning on the policies would freeze them in the model, and every update would mean retraining. The team spends its effort on getting the right paragraph into the request and on testing the answers.
Does the step up pay for itself?
The last piece of the decision is arithmetic. Every rung may improve quality, and every rung costs money in model fees and in one-off effort. The right question is whether the improvement is worth it. Suppose a prompt-only solution works at 80 per cent quality and costs ₹1 per 100 requests in model fees. Adding retrieval brings quality to 92 per cent for ₹1.40 per 100 requests, and takes some effort to build. Fine-tuning might lift quality to 94 per cent, but needs weeks of preparing examples and ongoing upkeep. The last two points may not justify that effort.
The simulator below lets you test this with your own numbers. Set how many requests you handle, what a correct answer is worth, the quality and running cost of each rung and the one-off cost of building the second and third. It shows the daily profit from each rung, how many days it takes for a step up to repay its set-up cost, and a chart of the cumulative result over a year.
At the starting numbers, retrieval repays its set-up in about 16 days, which is an easy decision. Fine-tuning adds only about ₹3,000 a day over retrieval, so its ₹15 lakh set-up needs more than a year to pay back, and that is before counting the upkeep. Now try two experiments. First, raise the value of a correct answer to ₹50 and watch fine-tuning start to look better, because each extra point of quality is worth more. Second, set the quality with fine-tuning to 99 per cent, the case where the tuned model is far better, and see how quickly the decision changes. The method, comparing the daily gain with the one-off cost, is the same in every case, and the answer depends heavily on how much a correct answer is worth.
How to decide in practice
Putting the ideas together, here is a routine you can use on any new project.
1. Write down what good looks like. Collect 30 to 50 real examples with the answers you want. Without them you cannot tell whether anything helped.
2. Try the best prompt you can write. Measure it on the examples. Note which kinds of mistakes remain.
3. Classify the remaining mistakes. Are they missing facts, wrong format, wrong tone or reasoning errors?
4. Climb one rung for the kind of mistake you have. Facts mean retrieval. Behaviour means fine-tuning. Do not mix up the two.
5. Measure again and compare cost to gain. Use the calculation above. Stop climbing when the next step no longer pays.
6. Keep the test examples. You will use them again whenever the model, the prompt or the documents change.
Notice that every step in this routine depends on a set of examples and a measurement. This is the thread running through the whole track: you cannot choose between rungs, or know whether a change helped, without evaluating. A later module is devoted to it.
Common mistakes
These are the mistakes teams most often make when choosing how to build.
Starting at the top. Wanting a custom model before trying a good prompt wastes months.
Fine-tuning to teach facts. Facts go stale inside a model and are hard to correct. Use retrieval.
Writing a weak prompt and calling it a failure. A one-line prompt is not a fair test of rung one. Write a careful one with examples first.
No examples to test against. Without a test set, a better-looking answer on one example is an anecdote.
Ignoring the ongoing cost. A tuned model needs fresh examples, retesting and sometimes retuning. Count the upkeep.
Choosing on model fees alone. The one-off effort and the people checking the data are real costs too.
Confusing privacy with quality. Hosting your own model keeps data in your control. It does not make answers better.
Key takeaways
Climb the ladder in order: prompt, retrieval, fine-tune, and only rarely train from scratch.
Use retrieval for facts and fine-tuning for style, format or narrow skills.
Know what is wrong before choosing a fix, and classify the mistakes you see.
Compare each step's quality gain with its cost and effort, and use a test set to measure it.
Running your own model is a question of where data lives, not a way of getting better answers.
The same number can lead to different decisions depending on what a correct answer is worth.
Check yourself
1. A model keeps quoting last year's fee schedule. Which rung fixes it, and why not fine-tuning?
Retrieval: put the current schedule in each request. A tuned model would keep the old figures until retrained.
2. Your answers have the right facts but the tone varies from reply to reply. Which rung might help?
Fine-tuning, because tone is a behaviour that is easier to show with examples than to describe.
3. Why should you write a careful prompt with examples before judging that prompting "does not work"?
A vague one-line prompt is not a fair test, and a good prompt often solves the problem.
4. A step up gains ₹4,000 a day and costs ₹2 lakh to set up. About how many days until it pays for itself?
50 days, because 200,000 divided by 4,000 is 50.
5. Does hosting your own open model improve answer quality?
Not by itself. It keeps data in your control, but quality still comes from prompts, retrieval and tuning.
Try this
Set aside about forty minutes and think of an AI feature you would like to build.
1. Write what the model must know, where that information lives and how often it changes.
2. Write the format and tone you need back, with one example of a good answer.
3. Collect or invent ten realistic inputs and write the answer you would want for each. These are the start of your test set.
4. Decide which rung you would try first, and write what result would make you move up one rung.
5. Put your numbers into the simulator above: your daily volume, what a correct answer is worth and your guesses for the quality of each rung. See how many days a step up would take to repay itself.
6. Write down what would change your decision, such as a rule that stops data leaving your systems or a much higher value per answer.
A good result is a one-page note that names the rung you would start on, the evidence you would use to move up, and the payback you expect. It is the kind of decision note a team writes before spending any money on tuning.
Tags: AI Engineering Fine-tuning Prompting Decision making Beginner