Skip to content

Tool calling

Build · 45 min · hands-on

Before this: 0 Setup · After this: 3 The agent loop Overview version: What an agent is

Abstract

"Function calling" is a misleading name. The model never calls anything. It emits a structured request — a name and a JSON string of arguments — and your code decides whether to honour it. Understanding that this is a protocol rather than execution is what makes every agent framework legible afterwards.

Prerequisites: Setup — a model answering locally.

Verified as of 2026-08-21.

Arriving here first?

This module assumes you know what a language model is and roughly what an agent does. If either is new, read How models work and What an agent is first — about 70 minutes, no code — then come back. Everything after this point builds on both.

What you'll be able to do

Define a tool, recognise a tool-call request in a raw response, dispatch it yourself, and hand the result back so the model can use it.

The mechanism

Three things move between you and the model:

sequenceDiagram autonumber participant You as Your code participant M as Model You->>M: messages + tool schemas M-->>You: tool_calls: [{name, arguments}] Note over You: The model has run nothing.<br/>You choose whether to execute. You->>You: parse + validate arguments, run the function You->>M: messages + role:"tool" result M-->>You: content (or another tool_call)

A tool is two things that must agree: a JSON Schema the model reads, and a function you run. Nothing enforces the agreement. If they drift, the model sends arguments your function does not accept and you get an error the model never sees.

The three details that matter

Arguments arrive as a string, not an object. arguments is JSON text the model wrote. Parse it, then validate it. The schema guides the model; it does not bind it.

Descriptions are the interface. The model chooses a tool by reading its description. A vague description is a bug — it produces a model that reaches for the wrong tool, and no amount of prompt engineering elsewhere fixes it.

Errors belong in the conversation, not in a stack trace. Return {"error": "..."} as the tool result. A raised exception ends the run with a traceback the model never sees and cannot correct. A returned error gives it a chance to try again with different arguments.

A tool call is structured output

Worth naming, because it makes the rest of the field legible: a tool definition is a JSON Schema, and the model's request to call it is schema-constrained output. Same mechanism, two names.

Which means the guarantees are the same, and so are the limits:

Guaranteed Not guaranteed
The request names a tool you defined That it is the right tool for the task
The arguments match your schema's shape That the values are correct or safe
Required fields are present That a required field was filled from evidence rather than invented

A model that must supply order_id and does not know one will supply something that looks like an order ID. Constrained decoding removed your parse errors; it did not remove wrong answers, it made them well-formed. Validate the values, not just the shape.

Two practical consequences for the code you are about to write:

  • Give the model a way to not call a tool. If the only representable actions are tool calls, you have arranged for one to happen. An explicit "insufficient information" path — a tool, or simply a prompt that permits a plain answer — prevents an invented call.
  • Descriptions do the work in both directions. The tool description decides whether the right tool is chosen; the per-parameter descriptions decide whether the arguments are sensible. "ISO 8601 date, UTC" in a parameter description is worth more than a paragraph of system prompt.

The overview version of this, including JSON mode and where schema enforcement sits relative to asking politely, is in Prompting.

Build it

Lab 02 — tool dispatch · free, local, ~1 minute

python3 labs/02-tool-dispatch/lab.py

It asks the model a question it cannot answer from memory, prints the raw tool-call request, dispatches it by hand, and feeds the result back.

Verify

You should see a tool_calls block with "arguments": "{\"sku\":\"ABC-1\"}" — note the escaping; it is a string — followed by the model answering correctly once the result is in context.

What failure looks like: the lab exits non-zero if the model answers directly without requesting the tool. That is the quiet failure mode of weak tool support: the model invents a plausible number rather than admitting it needs to look one up. It is not an error you can catch — only an answer you can distrust.

In a framework

Every framework wraps exactly this. In Microsoft Agent Framework, the schema is generated from your Python type hints by a decorator, and the dispatch step is inside agent.run() — see tutorials/02-add-tools.

How it works in a real system

Tools in e-commerce-agents explains this concept as it is actually implemented there — what the design does, why, and where in the code to look. It is the bridge between this page and the source below.

In production

product_discovery/tools.py in e-commerce-agents — real tools with filtering, validation and clamped inputs. Note shared/tool_inputs.py alongside it: production tools re-validate arguments even though a schema was supplied, for exactly the reason above.

Go deeper

  • Writing effective tools for agents — Anthropic, Sep 2025. The best single piece on tool design: namespacing, token-efficient responses, actionable errors. Vendor-authored but concrete and non-promotional.
  • Advanced tool use — Anthropic, Nov 2025. What to do when the tool definitions themselves start costing more context than the task.

Next

The agent loop — what happens when one tool result creates the need for another.