Lesson 5.4: Structured outputs, tools and agent instructions

Learn to get output that software can read, describe tools so a model uses them correctly, and write agent instructions with limits, checks and approval points.

By the end of this lesson you will be able to ask a model for output in a fixed structure that a spreadsheet or a program can read, validate that output and decide how many retries are worth paying for, write a description of a tool so that a model chooses and uses it correctly, and draft instructions for an agent that has a goal, limits, checks and points where a person must approve. You will see why an agent that is right 95 per cent of the time at each of eight steps completes the whole job correctly only about two times in three, how a single retry can lift a 92 per cent format success rate to over 99 per cent, and what a runaway loop can cost. You should have read Lessons 3.2, 4.4 and 5.1, since agents are chains with decisions in them, and everything here should be tested the way the testing lesson describes. The concepts are explained in plain words, and no programming is required, although builders will recognise the patterns. How tools are connected in software is covered in the AI Engineering track.

Until now the reader of the model's output has been a person. When the reader is another program, a spreadsheet import or a database, the rules change. A person forgives a missing comma or an extra friendly sentence. A program does not. And when the model can do things as well as write, such as search a system, send a message or update a record, a mistake is no longer a bad paragraph but an action. This lesson covers the three steps from text to software: structure, tools and agents.

Structured outputs

A structured output is a reply in a fixed format that follows rules, such as a table with set columns or a block of data in a standard notation. The most common notation is JSON, a way of writing labelled values, as in {"category": "Billing", "urgent": true}. You do not need to master it. What matters is the idea: every reply has the same fields, in the same form, so that software can pick them out without guessing.

Structure is useful whenever the output goes anywhere automatically: into a spreadsheet, a ticketing system, a report template, or the next step of a chain. It also helps people, because a consistent layout is easier to scan and compare.

How to ask for it

A reliable request has five parts, and each guards against a specific failure.

1. State the fields, with a short definition of each. 2. State the type and the allowed values of each field: a number, a date in a stated format, a choice from a fixed list. 3. Say what to do when information is missing, such as a null value or the word "not stated". Without this, a model may invent a value to fill the field. 4. Show one filled-in example, which fixes the format better than any description. 5. Say what to output and nothing else: "Reply with the JSON object only, with no explanation and no text before or after it."

text Read the customer message and return one JSON object with exactly these fields:

- category: one of "Billing", "Fraud", "Account", "Other" - urgent: true if the customer reports money missing or an account locked, otherwise false - amountinr: the amount mentioned as a number with no commas, or null if none - summary: one sentence of at most 20 words

If a field cannot be determined from the message, use null. Do not guess. Reply with the JSON object only.

Example: Message: "I was charged Rs 1,200 twice for the same order." Output: {"category": "Billing", "urgent": false, "amountinr": 1200, "summary": "Customer was charged twice for one order."}

Message: """[PASTE MESSAGE]"""

Predict first: you run this prompt on 1,000 messages and it returns a reply a program can read in 92 per cent of cases. The other 8 per cent contain an extra sentence, a missing bracket or a wrong value type. How many of the 1,000 would fail, and what could you do cheaply? Decide before reading on.

Eighty would fail. A cheap response is to validate and retry: check each reply against the rules, and if it fails, send the same request again, perhaps with the error attached. If failures are independent, a retry fixes most of them. The chance that a message fails twice in a row is 0.08 × 0.08 = 0.0064, so one retry lifts the success rate from 92 to 99.4 per cent, and two retries to 99.95 per cent. The cost is a few more calls: on average each message needs about 1.08 calls with one retry allowed, since only the failures are repeated.

Validation is not optional

A well-formed reply is not necessarily a correct one. Validation has two levels. Format checks confirm that the reply parses, that every field is present, that categories come from the allowed list and that numbers are numbers. A program can do this exactly. Content checks ask whether the values are right: is the amount actually in the message, is the category sensible? These need either a rule, such as "the amount must appear as a number in the source", or a sample check by a person, as in Lesson 4.2. Build both. A reply that passes the format check and carries a wrong value is the dangerous kind, because it flows into the system looking perfectly normal.

Practical tips

Keep it small. Ask only for fields you will use. Every extra field is another chance of error. Use enumerated choices for categories and flags, and list the options in the prompt. Describe each field in one line. Ambiguous names such as "type" or "status" invite varied answers. Use consistent formats for dates and amounts, and say them: "dates as DD-MM-YYYY, amounts as numbers". Allow an "other" or null, so the model is not forced into a wrong choice. Watch for extras: models often add a polite sentence or wrap the data in decoration. "JSON only" cuts it, and a validator strips what remains. Use the platform's own feature if there is one. Many tools offer a mode that forces replies to follow a given structure, which is more dependable than asking in words. Features differ by tool and change over time, so test yours.

Tools: letting the model do things

A tool is a capability the model can ask to use: search a database, look up an order, run a calculation, send an email, create a ticket. The model does not perform the action itself. It writes a request, such as "look up order 4471", a program carries it out and returns the result, and the model continues using it. This pattern is often called tool use or function calling, and it is the mechanism behind assistants that can look things up and act.

For you as a prompt writer, the central fact is that a tool description is a prompt. The model decides whether to use a tool, which one and with what inputs, based only on the text that describes it. A vague description leads to the wrong tool, or to the right tool with wrong inputs.

Writing a good tool description

A good description answers six questions. Question · ExampleWhat is it called? · lookuporder What does it do? · "Returns the status, items and delivery date of one order" When should it be used? · "Use when the customer asks about an existing order" When should it not be used? · "Do not use for refunds or for orders not yet placed" What does it need? · "orderid: the 6-digit number, as text. Ask the customer if it is missing" What does it return, and what if it fails? · "Returns a JSON object. If the order is not found, returns an error message" Two tools with overlapping purposes confuse a model. If searchcustomers and findcustomer both exist, state clearly how they differ, or merge them. Keep the number of tools small, since a model chooses more reliably among five than among fifty. Give names that describe the action in plain words. And include one short example of a good call, which teaches the input format.

text Tool: lookuporder Purpose: returns the status, items and delivery date for one existing order. Use when: the customer asks where an order is, or what it contains. Do not use for: refunds, cancellations or orders the customer has not yet placed. Input: orderid, a 6-digit number written as text, such as "104471". If the customer has not given it, ask for it. Never guess an order number. Output: a record with status, items and deliverydate. If no order is found, returns "NOTFOUND". If NOTFOUND, tell the customer and ask them to check the number. Do not try other numbers.

Notice the line "Never guess an order number". Without it, a model that lacks an input may make one up, and the tool will return somebody else's order, which is a privacy failure as well as an error.

Agents: a model that decides what to do next

An agent is a model that works towards a goal over several steps, choosing its own actions: reading, using tools, checking the results and deciding what comes next, until it judges the job done. It is the chain of Lesson 3.2 with the model, not you, deciding the next link. That freedom is what makes agents powerful, and what makes them risky. Each step is another place for an error, and an error that triggers an action can be hard to undo.

The arithmetic from the chaining lesson applies directly. If each step succeeds 95 per cent of the time, an eight-step task succeeds completely only 0.95 to the power 8, which is about 66 per cent of the time, and a twelve-step task about 54 per cent. If a check after each step catches 80 per cent of errors, each step ends up right 0.95 + 0.05 × 0.8 = 99 per cent of the time, and eight steps succeed about 92 per cent of the time.

What to put in agent instructions

An agent's instructions are its standing orders, in the sense of Lesson 3.3, with extra parts for autonomy. A good set has eight.

1. The goal, in one or two sentences, and what counts as done. 2. The role and boundaries: what the agent is for and what it must never do. 3. The tools it may use, each described as above, and any it may not. 4. A way of working: "Plan briefly, take one step, check the result, then decide the next step." 5. Rules for uncertainty: "If you are unsure or the information is missing, ask the user or stop. Do not guess." 6. Approval points: actions that need a person's confirmation before they happen, such as sending a message, spending money, deleting or changing records. 7. Limits: a maximum number of steps, a time limit and a spending cap. 8. What to report: "At the end, list what you did, what you found and anything you could not do."

text Goal: answer the customer's question about their order and, if the order is delayed more than 3 days, draft an apology for a human agent to approve. Done when: the customer has an answer, or you have asked for the missing detail.

You may use: lookuporder. You may not use any other tool, and you may not change any record.

Work step by step. After each tool result, check it answers the question before moving on.

If the order number is missing or lookup returns NOTFOUND, ask the customer and stop. Do not guess numbers.

Never promise a refund or compensation. Never send a message yourself: write the draft and mark it "FOR APPROVAL".

Stop after 6 steps. At the end, report what you did and what you could not do.

The three most important lines are the ones that limit harm: no guessing of inputs, no actions beyond the task and a human approving anything that reaches a customer. Compare the cost of adding them, a few sentences, with the cost of an agent that emails a wrong apology to a thousand customers.

Cap the cost

An agent that loops can run up a bill. Each step re-sends the conversation so far, so costs grow with every step. Suppose a step uses 3,000 tokens and the example price is ₹800 per million tokens. A planned eight-step task costs 8 × 3,000 ÷ 1,000,000 × 800 = ₹19.20. If a bug sends the agent round in circles for 100 steps, that is ₹240 for one task, and at 1,000 tasks a day it is ₹2.4 lakh a day. A step limit, set at perhaps twice the normal number, turns a disaster into a nuisance.

Testing agents and tools

An agent cannot be judged from a few chats. Apply the method of Lesson 5.1, with scenarios in place of single inputs. Build a set that includes normal tasks, tasks with missing information, where the right move is to ask, tasks that tempt the agent to overreach, such as "also refund the customer", tool failures, such as NOTFOUND or a timeout, and hostile inputs, where the text it reads contains instructions that try to redirect it, which Lesson 6.2 covers. For each, define what the agent should do and what it must not, and record whether it did. Read the trace, the record of the steps it took, since the way it reached an answer matters as much as the answer: a correct result reached by an action it should not have taken is a failure.

Worked examples

Example 1: a form-filling extractor. A team extracts four fields from invoices into JSON. First-try validity is 91 per cent, mostly from a missing amount field and stray text. They add the null rule, one example and "JSON only", and validity rises to 97 per cent. A single retry with the validator's error message attached brings it above 99.5. A sample check of 50 outputs finds two wrong amounts, which pass the format check, so they add a content rule that the amount must appear in the invoice text.

Example 2: two tools that clash. An assistant has searchorders and getorder, with descriptions both reading "find an order". It uses the wrong one a third of the time. Rewriting them, with searchorders for "when the customer does not have an order number, finds orders by name and date" and getorder for "when you have an order number", with a "do not use" line each, brings the errors down to a few per hundred.

Example 3: a bounded agent. A support agent drafts replies for delayed orders. Its instructions allow one tool, forbid changing records, require a human to approve every draft and cap the steps at six. In testing, a scenario in which the customer demands a refund shows the agent drafting "we will refund you". That breaks the rule, so they add an explicit example of the right response and a line in the approval checklist. The next run passes.

Common mistakes

Asking for structure without defining it. State fields, types, allowed values and a null rule, and show an example. Skipping validation. A reply that parses is not necessarily right. Check format and content. Vague tool descriptions. Say when to use a tool, when not to and what it needs. Too many overlapping tools. Keep the set small and distinct. Letting the agent guess inputs. Add "ask, do not guess". No approval point for actions that reach people or change records. A human confirms anything irreversible. No step or cost limit. Set caps near twice the normal. Judging an agent on a few chats. Test scenarios, including failures and hostile inputs, and read the traces.

Key takeaways

Structured outputs let software read replies. Define fields, types and allowed values, say what to do when information is missing, show an example and ask for the data only. Validate both format and content. With a 92 per cent first-try validity, one retry gives about 99.4 per cent and two about 99.95 per cent, at roughly 1.08 calls per message. A tool description is a prompt: name, purpose, when to use, when not to, inputs, outputs and failure behaviour. Keep tools few and distinct. An agent works toward a goal over steps it chooses. Reliability drops with length: at 95 per cent per step, eight steps complete correctly about 66 per cent of the time. Agent instructions need a goal, boundaries, tools, a way of working, rules for uncertainty, approval points, limits and a report. Never let it guess inputs, and keep a human on irreversible actions. Cap steps and cost, test with scenarios including failures and overreach, and read the traces.

Check yourself

1. Why add "use null if the field cannot be determined"? Without it, a model may invent a value to fill the field. 2. If a reply is valid 85 per cent of the time and you allow one retry, what is the chance of a valid reply? 1 − 0.15 × 0.15 = 0.9775, about 97.75 per cent. 3. What six things should a tool description say? Its name, purpose, when to use it, when not to, the inputs it needs and what it returns, including failure. 4. An agent is right 90 per cent of the time at each of 6 steps. What is the chance the whole job is right? 0.9 to the power 6 = 0.531, about 53 per cent. 5. Name two actions that should need human approval. Sending a message to a customer, spending money, deleting or changing records, or anything irreversible.

Try this

You need an assistant and about sixty minutes.

1. Choose a small extraction task, such as pulling the category, urgency and amount from a message. Define four fields with types, allowed values and a null rule. Add one example. 2. Run it on 15 realistic inputs, including some with missing information. Count how many replies are valid and how many are right. Check the content, not just the format. 3. Add "JSON only", a null rule and a second example, rerun and compare. Use the retry simulator with your first-try rate to see what one retry would give. 4. Write a description for one tool, such as order lookup, with all six parts. Rewrite it for a second, overlapping tool and state how they differ. 5. Write agent instructions with the eight parts for a task of your own, including approval points and a step limit. 6. List eight test scenarios: normal, missing information, a tempting overreach, a tool failure and a hostile input. State the correct behaviour for each. Run them by hand with an assistant, role-playing the tools if you have none, and record what it did. 7. Use the step-limit simulator with your own figures to set a cap. Scenario · Correct behaviour · What happened · Pass? A good result is a validated output format with measured reliability, a clear tool description, agent instructions with limits and a table of scenario tests. With this lesson Module 5 is complete. You can test prompts, choose models, ground answers and write instructions for structured output, tools and agents. Module 6 turns to making all of this safe and reliable: consistency, attacks through untrusted text, privacy and a capstone prompt library.

Tags: Prompt Engineering Structured output Tool use Agents Advanced