Lesson 3.3: Frameworks, SDKs and building without them
Learn what agent frameworks and SDKs do for you, what they cost, how to choose between them and your own small harness, and how to avoid lock-in.
By the end of this lesson you will be able to say what an agent framework provides, recognise the main families of tools without needing their brand names, weigh the benefits against the costs of depending on one, read a working harness of about sixty lines that shows what any framework does underneath, and use a checklist to evaluate a framework before committing to it. You will compare the effort of three approaches over three years and see that the one that is cheapest to start is not always the cheapest to keep, with illustrative totals of 20, 22 and 33 days of effort. You should have read Lessons 1.2 and 2.1, which describe the loop and tool calls, and some Python helps for the code example. The mathematics is addition.
A new builder meets a bewildering choice. Dozens of libraries, kits and platforms promise to make agents easy, and the list changes every few months. Which should you use? The honest answer is that it depends on what you are building and who will maintain it, and that the best preparation for choosing is to understand what they all do underneath. This lesson gives you that understanding and a method for deciding, and it deliberately avoids naming products, because the names change faster than the ideas.
What every agent system needs
Whatever you use, the same work has to happen. Lessons 1.2 and 1.4 described it: build the prompt, call the model, read the reply, validate and run tools, keep the history, enforce limits, record what happened. A framework is a collection of ready-made answers for these jobs. Here is the work, listed so that you can see what you are getting and what you are not.
Job · What it involvesThe loop · Call the model, handle its reply, repeat until done, stop on limits
Tool definition · Describe tools to the model, usually by turning a function into a schema
Validation and dispatch · Check requests, run the tool, turn errors into readable results
Conversation state · Keep the history, trim it, store it between turns
Memory and retrieval · Save and search facts across runs or from documents
Model access · Call one or several providers behind a common interface
Streaming · Show output as it is produced
Tracing · Record every step for debugging and audit
Guardrails and approvals · Check inputs and outputs, pause for a person
Multi-agent helpers · Start sub-agents, route between them, share state
Evaluation hooks · Run test sets and score the results
The loop, tool definition, validation and tracing are small. Even a beginner can write them in an afternoon, and the example later in this lesson does so in about sixty lines. The others, such as retrieval, streaming and multi-agent coordination, take more work, and a library can save real time there.
The families of tools
Rather than a catalogue, here are the main families. Any product you meet will fit one or more of them.
Family · What it gives you · Typical strength · Typical costDirect model API (no library) · Complete control; you write the loop · Simple, transparent, no dependency · You build everything else yourself
Model provider's agent kit · A loop, tool definitions and tracing designed for that provider's models · Fast start, tuned to the models · Ties you closer to one provider
General-purpose agent framework · Many integrations, memory helpers, multi-agent patterns, several providers · Breadth, community examples · Size, many abstractions, frequent change
Workflow or graph framework · You draw the steps and branches; models fill some of the nodes · Predictability, resumable runs, visible structure · More to learn, rigid for open-ended tasks
No-code or low-code platform · Visual building, hosting and monitoring included · Speed for non-engineers, prototypes · Limits on control, cost per use, platform lock-in
Protocol-based tool connectors · A standard way to expose tools so that any compatible agent can use them · Reuse a tool across many agents · Another layer to run and secure; Module 6 covers this
Many teams combine them: a provider's kit for the loop, a workflow framework for a fixed process, and a standard connector for shared tools.
Why frameworks help, and what they cost
The benefits
Speed. A working prototype in an hour, with tracing and tool definitions already in place.
Solved problems. Retries, streaming, state storage and integrations are written and tested by others.
Shared knowledge. Documentation, examples and other people who know the library.
Patterns. Ready-made ways to build sub-agents, hand-offs and approval steps.
The costs
Abstraction. The library sits between you and the model. When something goes wrong, you need to see the exact prompt and the exact reply, and some libraries make that hard. A good rule: if you cannot print precisely what was sent to the model at each step, you cannot debug it.
Hidden prompts. Some frameworks add their own text to your prompt. It may change between versions and affect behaviour in ways you did not choose.
Churn. The field moves quickly. Libraries change their interfaces, rename things and drop features. Code written against one version may need rewriting within months. Budget for upkeep.
Lock-in. The more of your logic lives in a library's special concepts, the harder it is to leave.
Weight. A large dependency brings its own dependencies, bugs and security updates.
Mismatch. A general library built for many uses may get in the way of your particular one. Simple jobs sometimes acquire complicated machinery.
Predict first: a team of two builds a single invoice agent that will run for years. They can start in a day with a large general framework, or spend a week on a small harness of their own. Over three years, which will probably cost less effort: the framework, the own harness, or about the same? Decide before reading on.
It depends on the numbers, and that is exactly the point. The framework has a low build cost and a recurring upkeep cost: updates, interface changes, learning its quirks. The own harness has a higher build cost and a low upkeep, since the code is small and the team knows every line. For a long-lived, simple agent, the harness often wins by the second or third year. For a short experiment, or an agent that needs many integrations, the framework wins. Count the effort over the life of the project, not just the first week.
Building without a framework
The best way to judge a framework is to know what it replaces. A minimal harness is short. The version below does the jobs in the first half of the table: it turns ordinary Python functions into tools with schemas, runs the loop, validates requests, turns errors into results, records a trace and enforces step and time limits. The model is any function that takes the goal, the tool schemas and the history and returns the next action. This makes it easy to swap a stand-in model for a real provider's by writing a small adapter, which Lesson 3.4 shows.
python
import inspect, json, time
REGISTRY = {}
def tool(fn):
"""Register a function as a tool and build its schema from the signature and docstring."""
params = {}
for name, p in inspect.signature(fn).parameters.items():
kind = {str: "text", int: "number", float: "number", bool: "true or false"}.get(p.annotation, "text")
params[name] = {"type": kind, "required": p.default is inspect.Parameter.empty}
REGISTRY[fn.name] = {"fn": fn, "description": (fn.doc or "").strip(), "params": params}
return fn
@tool
def addnumbers(a: float, b: float):
"""Adds two numbers exactly. Use for any arithmetic. Returns the sum."""
return {"sum": a + b}
@tool
def lookup(topic: str):
"""Looks up a short fact by topic. Returns the fact or says none was found."""
facts = {"due date": "Invoices are due 30 days after submission."}
return {"fact": facts.get(topic, f"No fact found for '{topic}'.")}
class Harness:
"""The whole loop: ask the model, validate, run the tool, record the trace, enforce limits."""
def init(self, model, maxsteps=6, maxseconds=20):
self.model, self.maxsteps, self.maxseconds = model, maxsteps, maxseconds
def schemas(self):
return {n: {"description": t["description"], "params": t["params"]} for n, t in REGISTRY.items()}
def run(self, goal):
history, started = [], time.time()
for step in range(1, self.maxsteps + 1):
if time.time() - started > self.maxseconds:
return {"status": "stopped: time", "trace": history}
action = self.model(goal, self.schemas(), history) # the model only decides
name, args = action["tool"], action.get("args", {})
if name == "finish":
return {"status": "done", "answer": args.get("answer"), "steps": step, "trace": history}
if name not in REGISTRY:
history.append({"tool": name, "error": f"no tool named {name}; available: {list(REGISTRY)}"})
continue
missing = [k for k, v in REGISTRY[name]["params"].items() if v["required"] and k not in args]
if missing:
history.append({"tool": name, "error": f"missing arguments: {missing}"})
continue
try:
runtool = REGISTRY[name]["fn"]
history.append({"tool": name, "args": args, "result": runtool(args)})
except Exception as err: # tool errors become readable results
history.append({"tool": name, "error": f"{type(err).name}: {err}"})
return {"status": "stopped: step limit", "trace": history}
def standinmodel(goal, schemas, history):
"""Pretend model: add the two numbers in the goal, then finish."""
nums = [float(x) for x in goal.split() if x.replace(".", "", 1).isdigit()]
if not history:
return {"tool": "addnumbers", "args": {"a": nums[0], "b": nums[1]}}
return {"tool": "finish", "args": {"answer": f"The total is {history[-1]['result']['sum']:g}."}}
print(json.dumps(Harness(standinmodel).schemas()["addnumbers"]))
print(Harness(standinmodel).run("Add 48000 and 12500")["answer"])
print(Harness(lambda g, s, h: {"tool": "lookup", "args": {"topic": "x"}}, maxsteps=3).run("loop")["status"])
Several things about this small program are worth noticing.
The tool schema comes from the function. The signature gives the parameter names and types, and the docstring gives the description. Writing good docstrings is therefore writing good tool descriptions, which Lesson 2.2 taught.
Every failure becomes a readable result. An unknown tool, a missing argument or an exception inside the tool does not crash the loop. The model is shown what went wrong.
The limits live in the harness. The step and time limits are enforced here, where the model cannot ignore them.
The model is replaceable. It is just a function. A framework often hides this fact behind a class hierarchy.
The trace is the history. Every call, argument and result is kept, which is all you need to debug.
What the sixty lines leave out matters too: streaming, storing state between conversations, calling tools in parallel, retries with backoff, approval pauses and sub-agents. Each of these can be added as you need it, and each is also something a framework may already provide. The decision is not between "magic" and "primitive" but between code you understand and maintain, and code you depend on.
A checklist for choosing
Whether you choose a library or write your own, check these points before committing. Answer them with a short test, not by reading the marketing.
Question · Why it mattersCan I see the exact text sent to the model and received back, at every step? · You cannot debug what you cannot see
Does it add its own prompt text, and can I turn that off? · Hidden text changes behaviour
Can I set step, time and budget limits in the loop? · Limits are essential safety controls
Can I validate tool requests and control errors myself? · Lessons 2.1 and 2.3 depend on it
Can I add approval steps before certain tools? · Needed for the autonomy levels of Lesson 1.3
How does it record traces, and can I export them? · Needed for audit and for evaluation
Can I swap the model provider without rewriting my agents? · Prices, quality and availability change
Can I test an agent with a stand-in model, offline? · Tests must be fast and repeatable
How often did it make breaking changes in the last year? · Predicts your upkeep
Who maintains it, and what is its licence? · Affects risk and legal use
How much of my code would be specific to it? · Measures lock-in
A practical protocol is to build the same small agent, such as the invoice agent of Lesson 3.4, in each candidate, and compare: lines of code, hours to first working version, how easy it was to see and test what happened, and how it behaved when you broke something on purpose. An afternoon of this beats a week of reading.
Reducing lock-in
Whatever you choose, structure your code to keep your options open.
1. Keep your tools as plain functions. Write them without library-specific types, and wrap them in a thin adapter for each framework. Then the valuable part, the business logic, moves with you.
2. Keep your instructions in your own files, as in Lesson 3.2, not scattered through framework configuration.
3. Put the model behind one function, so that changing provider means changing one adapter.
4. Keep your test cases and evaluation outside the framework. They are your most reusable asset.
5. Own your traces. Store a copy of every run in a form you control.
6. Review the choice on a schedule. The field changes monthly, and a quarterly look at whether your tool still fits is cheap insurance.
A sensible default for a beginner
If you are starting out, a good path is: learn on the direct model API or a small provider kit so that you see every moving part; write your own loop once, as in this lesson; then adopt a library for the pieces that you find yourself rebuilding, such as tracing or sub-agent routing. This path makes you a better user of any framework, because you will know what it is doing and be able to tell when it is doing something unwise.
Common mistakes
Choosing by popularity or novelty. Match the tool to your job and your team.
Not being able to see the prompts. If you cannot print what the model received, you cannot debug.
Putting business logic inside framework concepts. Keep tools and instructions as plain code and text.
Ignoring upkeep. Count effort over the life of the project, including updates and interface changes.
Building a framework when you needed an agent. Write only what you need.
Adopting a large library for a tiny job. Weight and churn have costs.
No offline tests. Make sure you can run an agent with a stand-in model.
Never revisiting the choice. Look again every few months.
Key takeaways
Every agent system needs a loop, tool definitions, validation, state, model access, tracing and limits. A framework is a set of ready answers to these jobs.
The loop, tool schemas, validation and tracing fit in about sixty lines. Reading and running them once teaches you what any framework does.
Frameworks give speed, solved problems and patterns. They cost abstraction, hidden prompts, churn, lock-in and weight.
Count effort over the whole life: build once and maintain every year. In the example, 20, 22 and 33 days over three years for three approaches.
Choose with a checklist and a small bake-off: can you see the prompts, set limits, validate, trace, swap models and test offline?
Reduce lock-in by keeping tools as plain functions, instructions and tests in your own files, the model behind one function and traces in your own storage.
Check yourself
1. Name three jobs that any agent system must do, whatever the tool.
For example run the loop, define and validate tools, keep the history, enforce limits and record a trace.
2. Why can a framework's hidden prompt text be a problem?
It changes the model's behaviour in ways you did not choose, and it may change between versions, so results shift without your doing anything.
3. An approach takes 5 days to build and 6 days a year to maintain. Another takes 2 days to build and 10 days a year. Which costs less over 2 years, and over 4?
Over 2 years: 17 days against 22, so the first. Over 4 years: 29 against 42, so again the first.
4. In the minimal harness, why are the step limit and time limit in the code and not in the instructions?
The model may forget or ignore an instruction. The harness runs outside the model and always enforces the limit.
5. Give two ways to reduce lock-in.
For example keep tools as plain functions behind a thin adapter, keep instructions and test cases in your own files, or put the model behind a single function.
Try this
Set aside ninety minutes.
1. Run the minimal harness. Add a third tool, with a clear docstring, and change the stand-in model so that it uses it.
2. Break it on purpose: make the model call an unknown tool, omit an argument, raise an error inside a tool, and never finish. Check that each is handled and that the trace shows it.
3. Add one feature yourself, such as retries with waiting from Lesson 2.3 or an approval step before one tool.
4. Pick one framework or kit you are curious about. Run the checklist on it with a small test, not by reading.
5. Estimate build and upkeep days for your own harness and for the library. Use the simulator to compare them over the life you expect.
6. Write a short list of what in your project must stay yours whichever you choose: tools, instructions, tests and traces.
7. Decide, and write down when you will review the decision.
A good result is a working harness you understand, a checklist run on a candidate library, a three-year effort comparison with your own numbers and a list of the assets you will keep portable. The next lesson puts everything together and builds a complete first agent, step by step.
Tags: Agentic AI Agent frameworks SDKs Harness Software architecture