Tools and Tool Calling

How a model requests a function call, who runs it, and how to define tools a model can use well.

A language model can only produce text. It can't run code, query a database, or send a request on its own. A tool is any external function the program running the model is willing to call on its behalf, when the model asks for it in a format the program recognizes. The model never executes anything. It emits a request, the program runs the function, and the result goes back into the transcript for the model's next step.

Two ways to request a tool

The simplest version is a text convention. The program watches the model's output for a line such as Action:, stops the model there, runs the function, and appends the result as an Observation: before asking the model to continue. Here is an agent solving an arithmetic question with a calculator tool:

User: What is 47 times 89, plus 12?

Thought: I need to multiply, then add. I'll use the calculator.
Action: calculator
Action Input: 47 * 89 + 12
Observation: 4195

Thought: I now have the answer.
Final Answer: 4195

Every line, including Action: calculator, is the model predicting text by the same mechanism as any other generation. The line labels are a convention, not a standard.

Hosted model APIs offer tool calling, also called function calling, which replaces the text convention with structure. You describe each tool with a name, a description, and a JSON Schema for its arguments:

{
  "name": "get_weather",
  "description": "Get the current weather for a given location.",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {"type": "string", "description": "City and state, e.g. San Francisco, CA"}
    },
    "required": ["location"]
  }
}

The schema field is called parameters in OpenAI's API and input_schema in Anthropic's. When the model wants the tool, the response carries a structured call instead of free text: an identifier, the tool name, and the arguments as JSON, such as {"location": "San Francisco, CA"}. The program runs the function and sends back a result that names the identifier of the call it answers. The loop is the one in The Agent Loop. The model can request several tools in one reply, and each needs its own result.

Who runs the tool

Anthropic's documentation separates tools by where the code executes. With client tools your application runs the function and returns the result. With server tools such as web search and code execution, the provider runs the operation and you read the outcome. Client tools are the common case for application-specific logic: database queries, internal APIs, file writes.

Tool results often carry content from outside your control, such as web pages, inbound email, or third-party APIs, and that content can contain instructions aimed at the model. Anthropic's documentation calls this indirect prompt injection and advises treating such content as untrusted.

Designing tools the model can use

Clear names and descriptions. The model decides when to call a tool from the user's request and the tool's description. OpenAI's guidance is to write clear and detailed function names, parameter descriptions, and instructions. Anthropic reports spending more time optimizing the tools than the overall prompt when building its coding agent for the SWE-bench benchmark.
A small set. Tool definitions count against the context window and are billed as input tokens. OpenAI suggests aiming for fewer than 20 functions at the start of a turn.
Constrained inputs. Types and enums in the schema keep arguments valid, and arguments the code already knows are better supplied by the code. Both OpenAI and Anthropic offer a strict mode that makes calls adhere to the schema.
Useful errors. A failure message that says what went wrong and what to try next lets the model recover.

A tool's schema also bounds what the agent can do. AI Agents in Production covers that as an authorization boundary.

Previous ← The Agent Loop

Next Planning and Reasoning →