Lesson 3.2: Structured outputs, so your code can trust the model
A model writes free text, but your code needs fields. Learn schemas, validation and retries so model answers become data your program can use safely.
A language model writes sentences. A program needs fields: a number it can add up, a category it can route on, a date it can store. Between the two sits a gap that causes more broken AI features than almost anything else. By the end of this lesson you will be able to explain why free text breaks software, describe the three ways of getting structured answers from a model and how much each one can be trusted, write a simple schema for a real task, validate a model's answer before your program uses it, design a retry that does not waste money, and estimate what a small failure rate costs at volume. You should already know what a prompt, a system message and a request are from Lesson 3.1. The code samples here are plain and generic and use no particular provider's library. The failure rates and rupee costs in the charts and sliders are illustrative assumptions for teaching, so replace them with numbers measured on your own task.
The problem: sentences are not data
Imagine you run a lending app and you ask a model to read a customer's message and decide what it is about. You write a prompt: "Classify this message." The model answers, and for the first message it writes "This looks like a complaint about a delayed refund." The next time it writes "Category: Refund delay". The third time it writes "I would say the customer is asking about refunds, probably a delay." A person reads all three and understands that they mean the same thing. Your program does not. It was waiting for one of five exact words so that it could send the message to the right team, and none of those answers matches.
This is the core problem. A model is trained to produce fluent, helpful text, and fluent text varies from one answer to the next. Software is the opposite: it needs a fixed shape every single time. A function that expects a number and receives "about fifty thousand rupees" does not guess, it fails. When the output of a model feeds another piece of code, the shape of that output becomes part of your system's contract, and a contract that holds only most of the time breaks in production.
There are three kinds of break to watch for.
Wrong shape. The answer is not in the format you asked for: extra words around the data, a missing field, a field with a different name, a list where you expected one value.
Wrong type. The field is there but holds the wrong sort of thing: the text "50,000" or "fifty thousand" where a number was needed, or a date written as "next Friday".
Wrong content. The answer is well formed and still wrong: a category that is not on your list, an amount that is negative, or a value the model invented because the document did not contain it.
The first two are problems of format, and good tools can nearly eliminate them. The third is a problem of meaning, and no format tool can fully remove it. Keep this distinction in mind through the lesson, because it tells you what to fix with a schema and what to fix with checks and evaluation.
The core idea: describe the shape, then check the shape
The solution has two halves. First you describe the shape you want, in a precise form that both the model and your code can use. Second you check the shape of what comes back before anything depends on it.
The common language for describing a shape is JSON, a plain text format built from a few parts: objects (a set of named fields in curly braces), lists (in square brackets), strings (text in quotes), numbers, true or false, and null for "nothing". It is easy for people to read, easy for almost every programming language to load, and models have seen a great deal of it. A structured answer for the lending-app message might look like this:
json
{
"category": "refunddelay",
"urgency": "high",
"amountrupees": 4500,
"needshuman": false
}
A description of that shape is called a schema. A schema lists the fields, the type of each, whether each is required and, where a field may take only a few values, the allowed list. In plain words, the schema for the example says: "an object with a category that must be one of these five labels, an urgency that must be low, medium or high, an amount that is a number or null, and a flag that is true or false". Schemas are the contract. You write one once, give it to the model, and use the same one to check the answers.
Notice that a schema is a design decision about your product, not a technical detail. Every field is a question you have decided to ask the model, and every allowed value is a branch your program has to handle. A schema with twelve fields and thirty categories asks for a lot of judgement from the model and a lot of code from you. A good first schema is small.
Three ways to get structure, and how far to trust each
There are three common ways to get structured output from a model API. They differ in how strongly they force the shape, and the right choice depends on what your provider offers and how much a failure would cost you. Names and options vary between providers, so check your provider's documentation for the exact features.
Way one: ask in the prompt
The simplest method is to tell the model what you want: "Reply with only a JSON object with these fields." This needs no special support. It works surprisingly often, and it fails in familiar ways: the model adds a friendly sentence before the JSON, wraps it in a code fence, forgets a field, or writes a long explanation after it. Some of these can be fixed by cleaning the text, but cleaning is guesswork, and it fails on the day the model does something you did not expect. Use this for a prototype, a one-off script or a case where a failure is cheap.
Way two: a JSON mode
Many APIs offer a setting that tells the model to produce valid JSON. This removes the problem of extra words around the answer and of broken syntax such as a missing bracket. It does not guarantee that the JSON has the fields you wanted, because the mode only promises some valid JSON, not your schema. You still need a prompt that describes the fields and a check afterwards.
Way three: schema-constrained output
The strongest option, offered by a growing number of providers, lets you hand the API a schema and have the system constrain the model so that the answer follows it. The rough idea is the one you met in Lesson 2.2: at each step the model chooses the next token from a list of probabilities. With constraint, the system removes the tokens that would break the schema before the choice is made. If the schema says the next thing must be one of five labels, only tokens that can begin those labels are allowed. The result is that the output is valid for your schema by construction, not by luck.
Related to this is tool calling or function calling, a feature where you describe a function and its arguments with a schema, and the model answers by producing the arguments for it. Many people use it simply as a way to get structured data out, even when no real function exists. You will meet it properly in the module on agents.
Constrained output has limits that are worth stating clearly. It guarantees the shape, which means the right fields, the right types and values from the allowed list. It does not guarantee the truth. The model can still put the wrong allowed label in the field, or fill a number field with a plausible number that the document never stated. Some providers also support only part of the schema language, so very complex schemas may be rejected. Constraint can occasionally make answers a little worse when it forces the model into a shape that does not fit what it wanted to say, which is a reason to include an "other" or "unknown" option, as you will see below.
The next chart compares the three ways on one illustrative task. The numbers are invented to show the typical pattern, in which each step up removes a class of failure, and they are not measurements of any provider.
Notice what the chart says and what it does not. Each method improves the share of usable answers, but none reaches certainty on its own. Even the strongest method leaves a small remainder, usually in the meaning of the content. That small remainder is why you also need the checks in the next sections.
Designing a schema that works
Good schemas follow a few habits that save a great deal of pain later.
Make every field earn its place. Ask what the program will do with each one. If nothing reads a field, remove it, since it only gives the model another chance to be wrong.
Use fixed lists wherever your code branches. If your program routes a message to one of five teams, the schema should allow exactly five values, not free text. Name them in plain, distinct words that cannot be confused with each other. Two labels that sound alike, such as "paymentissue" and "paymentproblem", invite inconsistent answers.
Always give the model a way out. If a field must be filled and the information is not there, the model may invent something to satisfy the schema. Add an explicit "unknown" value for categories, and allow null for numbers and dates that may be absent. Then say in the prompt: "If the document does not state the amount, use null. Do not guess." This single habit prevents a large share of wrong content.
Put a short description on each field. A schema can carry a sentence about every field, and the model reads it. "amountrupees: the refund amount the customer mentions, as a whole number in rupees, or null if no amount is stated" is far clearer than the bare name. Think of the descriptions as the instructions that travel with the field.
Ask for the evidence. When the content matters, add a field such as "quote" that asks the model to copy the exact words it relied on. Your code can then check that the quote really appears in the source text, which catches a common kind of invention.
Keep names stable. Once your program depends on a field name, changing it is a code change. Pick plain names and stick to them.
Order the fields so reasoning comes first when it helps. A model writes one token at a time, so a field that holds its explanation, placed before the field that holds its decision, can sometimes let it think before it commits. This is a technique rather than a rule, so test it on your task.
Here is a fuller schema for the lending-app message task, written as a plain description in the style most providers accept:
json
{
"type": "object",
"properties": {
"category": {
"type": "string",
"enum": ["refunddelay", "loanstatus", "emiquery", "fraudreport", "other"],
"description": "The main topic of the message."
},
"urgency": {
"type": "string",
"enum": ["low", "medium", "high"],
"description": "high if money is missing or fraud is suspected."
},
"amountrupees": {
"type": ["integer", "null"],
"description": "Amount the customer mentions, in whole rupees, or null if none is stated."
},
"evidence": {
"type": "string",
"description": "The exact words from the message that support the category."
}
},
"required": ["category", "urgency", "amountrupees", "evidence"]
}
Read it slowly. Every field has a type, the two label fields have a closed list, the amount may be null, and the evidence field gives you something to check. Schema languages differ slightly between providers, so treat this as the idea, not as text to paste.
Validate before you trust
Even with the strongest constraint, treat the model's answer as input from an outsider, the way you would treat data typed into a web form. Validate it before any other code uses it. Validation has layers.
1. Parse. Can the text be loaded as JSON at all? If not, it is a format failure.
2. Shape. Are all required fields there, with the right types? Libraries in most languages check this against the same schema you gave the model, so you describe the shape once and use it twice.
3. Range and rules. Is the amount above zero and below a sensible limit? Is the date in the past? Does the category belong to the list? Does the quote appear in the source? These are rules about meaning that only you know.
4. Cross-field logic. Does the answer agree with itself? A message marked "fraudreport" with urgency "low" is suspicious, and your code can say so.
Here is the same idea as a short piece of plain pseudocode, which any language can express:
answer = callmodel(prompt, schema)
data = tryparsejson(answer)
if data is not valid json: handlefailure()
if not matchesschema(data): handlefailure()
if data.amount is not null and data.amount < 0: handlefailure()
if data.evidence not in originalmessage: handlefailure()
use(data)
The point of the pseudocode is the order: nothing reaches use until it has passed every check. Failures go to one place, handlefailure, and what that place does is the subject of the next section.
When validation fails: retry, repair or hand over
A failed check is not an emergency. It is an event to plan for. You have three options, and good systems use them in order.
Retry. Call the model again. Because the model's answers vary, the second attempt often succeeds. Retries are cheap to write, but they cost money and time on every attempt, so you should limit them to a small number, usually one to three.
Retry with feedback. Instead of repeating the same request, add the error to the next one: "Your last answer failed this check: the amount must not be negative. Reply again with a corrected JSON object." This often works better than blind repetition because the model is told what to fix. It costs a little more per attempt because the input is longer.
Repair. For small, well-understood format slips, such as a trailing comma or a code fence around the JSON, a few lines of code can fix the text without another call. Keep repairs narrow and boring, because a clever repair routine becomes its own source of bugs.
Fall back. When retries run out, do not let the failure vanish. Send the item to a person, put it in a queue for review, or return a safe default and record that you did. The feature should degrade gracefully, and the log should say how often it happens, since that rate tells you whether your prompt and schema need work.
The simulator below shows why retries help a great deal at first and then very little. Suppose each attempt produces a usable answer with some probability, and failures are independent. Move the success rate and the number of retries and watch both the overall success chance and the average number of calls you pay for. Try a first-attempt success rate of 90 per cent and two retries.
Predict first: if each attempt works 90 times in 100, and you allow up to three attempts in total, what share of items will still fail? Is it about 10 in 100, about 1 in 100, or about 1 in 1,000? The answer is about 1 in 1,000, because all three attempts must fail, which is 0.1 × 0.1 × 0.1 = 0.001.
Look at the plot. Going from zero retries to one gives a large jump, from one to two a smaller one, and by the third the line is nearly flat. This shape, with sharply falling returns, is why a limit of one to three retries is usual. If the success rate is low to begin with, retries only cover for a weak prompt or schema, and the better fix is upstream.
A warning about the independence assumption in that simulator: failures are often not independent. If a document is genuinely confusing, the model is likely to fail on it every time, and retrying only repeats the cost. This is why retry-with-feedback and a final fallback matter, and why you should look at which inputs fail, not just how many.
Worked example one: extracting fields from an invoice
A small business wants to read supplier invoices and enter them into its accounts automatically. Each invoice is a block of text from an OCR step. The schema asks for the supplier name (string), the invoice number (string), the invoice date (a date in the format year-month-day, or null), the total in rupees (a number or null), and the GST amount (a number or null).
The prompt says: "Extract the fields. Use null for anything the invoice does not state. Do not calculate or guess." Then the validation layer runs four checks: the JSON parses, the shape matches, the date is not in the future, and the total is greater than or equal to the GST amount when both are present.
Now follow one failure. For an invoice with a smudged date, the model returns "12/03/26", a string in the wrong format. The shape check fails, since the date does not match the required pattern. The system retries with feedback: "The date must be in year-month-day format, for example 2026-03-12, or null if unclear." Notice that the real ambiguity here is whether 12/03/26 means 12 March or 3 December, which a format rule cannot settle. A careful designer adds a rule: if the date is ambiguous, return null and flag the invoice for a person. That is better than a confident wrong date entered into the accounts.
The lesson of the example is that the schema fixes the shape, while a thoughtful instruction about unknowns protects the meaning. The null option and the flag are what keep the system honest.
Worked example two: routing support messages
Return to the lending app. The schema has a category field with a closed list of five values, an urgency field with three values and an evidence quote. The routing code uses a simple table: fraud reports go to the security team, refund delays to payments, loan status to servicing, EMI queries to servicing, and "other" to a general queue.
Imagine 10,000 messages a month. A prompt-only approach produces unusable answers on some of them, say 12 per cent, in the illustrative figures above. That is 1,200 messages a month that need cleaning or a person. With schema-constrained output and an "other" value, the format failures nearly vanish, and what remains is wrong labels: a message about a failed UPI refund that the model labels "other". To catch these you cannot rely on the schema. You need a test set, a few hundred real messages that people have labelled, and a routine that compares the model's labels with theirs. That is the subject of the evaluation module, but note the connection now: structured output makes evaluation easy because the answers are comparable. Counting how often "category" matches the human label is trivial when it is one of five fixed values, and nearly impossible when it is a free sentence.
The next simulator puts a price on the leftover failures. Try the defaults, then change the failure rate to see what a small rate costs at your volume.
With the default figures, 1,200 bad answers at ₹40 each is ₹48,000 a month of manual work, whereas a retry at about ₹0.50 each costs ₹600. Even if retries fix only part of them, the comparison shows why you design for failure in the code and not in the people.
Worked example three: a small extraction with an "unknown" option
A consultancy wants a model to read short company descriptions and pull out the industry, with five allowed values: banking, insurance, retail, manufacturing and technology. Take a description that says "We run a chain of clinics across three states." None of the five values fits. A schema with a closed list of exactly those five and no escape forces the model to choose one, and it will, say "retail", because clinics serve customers. The answer is valid in shape and wrong in meaning, and nothing in your code can tell.
Add one value, "other", and an instruction: "Use other if no value clearly fits." Now the model says "other" and a person can read the exceptions. In an illustrative run of 200 descriptions, you might find that 14 are labelled "other", and when you look at them, 9 are healthcare. That tells you something valuable about your categories: healthcare is common enough to deserve its own value. This is the loop of good schema design. The "other" bucket is not a failure, it is a signal about what your schema is missing.
The same applies to numbers and dates: allow null, say what null means, and count how often it appears. A sudden rise in nulls is a sign that the input has changed, for example a new invoice format.
Edge cases and things that surprise people
Long outputs get cut off. If the answer runs into the maximum output length, the JSON may stop in the middle and fail to parse. Set limits with room to spare and treat truncation as a distinct failure you can detect.
Lists of unknown length. A field that holds a list of items, such as the line items on an invoice, can grow beyond what the model finishes writing. Consider extracting in chunks for very long documents.
Nested schemas get harder. Deeply nested objects are easier for a model to get wrong than flat ones. Prefer flat shapes when you can.
Free-text fields are still free. A "summary" field inside a perfect schema is still a free sentence. That is fine as long as no code branches on its content.
Numbers in text are slippery. "5 lakh" and "500,000" mean the same, "1.5k" is ambiguous, and different currencies cause traps. Say in the description exactly what unit and format you want, and validate the range.
The schema is also the prompt. Field names and descriptions are read by the model. Renaming a field from "amt" to "amountrupees" can improve accuracy with no other change.
Providers differ. Which schema features are supported, and how the constraint is applied, differs between providers and changes over time. Read your provider's current documentation and test with your real schema.
Safety. Treat the content of the fields as untrusted text. If a field is placed into a database query or a web page, a malicious input could be reflected in the model's output, so escape and validate it as you would any user input.
Common mistakes
Trusting the format and forgetting the meaning. A valid shape is not a correct answer. Add range checks, evidence checks and an evaluation set.
No way out. A closed list with no "other" or "unknown", and a required field with no null, push the model to guess. Always give it a truthful option.
A schema that is too large. Twenty fields in one call give the model more chances to err and cost more to check. Start with the few fields your code truly uses.
Cleaning text with clever string tricks. Hunting for the first bracket in a reply and hoping for the best works until the day it does not. Prefer a mode or constraint that guarantees valid JSON, and keep repair code small.
Unlimited retries. Without a cap, a hard input can loop and burn money. Set a limit, log the failure and fall back to a person.
Retrying without looking at why. If one input fails every time, repeating the call is waste. Send the error back, or route it away.
Never measuring the failure rate. The share of items failing validation is the best health signal you have. Log it and watch it after every prompt or model change.
Changing the schema without changing the code. A new allowed value that your routing code does not handle causes silent errors. Treat a schema change like any other contract change.
Key takeaways
- Free text breaks software because its shape varies, so any model output that feeds code needs a fixed structure.
- A schema describes the fields, types and allowed values, and the same schema can be used to guide the model and to check its answer.
- There are three ways to get structure, asking in the prompt, a JSON mode and schema-constrained output, and each step up removes more format failures but none removes wrong content.
- Always give the model a truthful way out, such as "other", "unknown" or null, so it does not invent values to fill a field.
- Validate every answer in layers, covering parse, shape, rules and cross-field logic, before any other code uses it.
- Cap retries at a small number, send the error back in the retry, and fall back to a person or a queue when it still fails.
- Measure the share of failures, because a small rate at high volume is a real cost, and use a labelled test set to catch wrong labels that a schema cannot.
Check yourself
1. Why does a model's free-text answer cause trouble for a program? Because the wording and shape vary from one answer to the next, while code needs a fixed, predictable format every time.
2. What is the difference between a format failure and a content failure? A format failure is a wrong shape or type, such as a missing field, while a content failure is a well-formed but wrong answer, such as a label that is not correct.
3. What does schema-constrained output guarantee, and what does it not? It guarantees the answer follows the schema's shape, types and allowed values, but not that the values are true or correct.
4. Why should a schema offer "other" or null? So that the model has a truthful option when the information is missing, instead of inventing a value to fill a required field.
5. If one attempt succeeds 90 times in 100 and you allow three attempts in total, with independent failures, how many in 1,000 still fail? About 1, because all three must fail, and 0.1 × 0.1 × 0.1 is 0.001.
6. Why are retries limited to a few, and what should happen after they run out? Returns fall sharply after the first retry or two and each one costs money and time, so after the cap the item should go to a person or a review queue and be logged.
Try this
This exercise takes 30 to 45 minutes. You need a text editor and, if you have one, access to any model through a chat window or an API. You do not need to write a program: you can do the checking by eye.
1. Pick a small real task (5 minutes). Choose something where you read short text and pull out a few facts, such as customer emails, job postings or product reviews. Collect 10 real examples in a file.
2. Write the schema (10 minutes). List three to five fields, with a type and a one-line description for each. Give each category field a closed list that includes "other", and let numbers and dates be null. Write the instruction that says what to do when a fact is missing.
3. Run the 10 examples (10 minutes). Ask the model to answer each in JSON that follows your schema. If you use a chat window, paste the schema in and ask for JSON only.
4. Validate by hand (10 minutes). For each answer, tick four boxes: it parses as JSON, all fields are present with the right types, the values are in the allowed ranges, and a quote from the text really supports it. Mark every answer that fails any box and write why.
5. Count and improve (5 minutes). Work out the share of answers that failed, and find the most common reason. Change the schema or the instruction to fix that one reason, and rerun the failures.
A good result is a short table with 10 rows, a count of format failures and a count of content failures, the single change you made, and what happened after it. If you found at least one answer that was valid in shape and wrong in meaning, you have understood the lesson. If every answer passed on the first try, make the task harder with messier text, because the aim is to see where the schema is not yet enough.
Tags: AI Engineering Structured outputs LLM APIs JSON Validation