LLM Function Calling and Tool Use, Explained

What LLM function calling actually does

Ask a model “what’s the weather in Austin right now” with no connection to the outside world, and it has two options: say it doesn’t know, or make something up. Function calling gives it a third option. It describes what it needs, hands that description to your code, and waits for a real answer before replying.

The name is slightly misleading. The model does not call anything. It emits a block of structured data saying which tool it wants and with what arguments, and your application decides whether to run it and what to do with the result. “Tool use” is the more accurate term, and it’s the one Anthropic’s docs use; OpenAI and Google mostly say function calling. The mechanism is the same under both names. The model only ever produces a request, never the side effect.

Why the model requests tools but never runs them

Picture a customer at a counter who can point at items on a menu and say how many they want, but never walks behind the counter. The model is the customer: it reads the menu you hand it (the tool definitions), picks an item, and says how many (the arguments). Your code is the person behind the counter who fills the order, and who can refuse one that doesn’t make sense.

That separation is the whole security model. A tool call is a suggestion your code can validate or reject, exactly like any other input from an untrusted source, because that’s what it is. A model that has been prompt-injected by a hostile web page can still only ask for a tool call. It cannot skip your validation and reach the database directly.

How to define a tool with JSON Schema

Every provider wants the same three things for each tool: a name, a description the model uses to decide when the tool applies, and a JSON Schema for the arguments. Here’s a weather lookup tool for the OpenAI Chat Completions API:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

weather_tool = {
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current temperature and conditions for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name, e.g. 'Austin, TX'"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["city"],
        },
    },
}

Spend more time on the description than on the schema. The model picks a tool by matching the user’s request against the descriptions in the list you sent. A vague one (“weather stuff”) either gets passed over for a better-described tool that does something else, or fires on requests it shouldn’t handle. Write it the way you’d explain the function to a teammate who will read exactly one sentence about it.

How the function calling request-response loop works

A tool call takes at least two requests. Here’s the full round trip against the OpenAI API:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

from openai import OpenAI
import json

client = OpenAI()
messages = [{"role": "user", "content": "What's the weather in Austin?"}]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=[weather_tool],
)
message = response.choices[0].message
messages.append(message)

if message.tool_calls:
    for call in message.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_weather(args["city"], args.get("unit", "celsius"))
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })

    final = client.chat.completions.create(model="gpt-4o", messages=messages)
    print(final.choices[0].message.content)

The first response doesn’t answer the question. It comes back with finish_reason set to tool_calls and no text content, because the model decided it needs get_weather before it can say anything useful. call.function.arguments arrives as a JSON string, not a parsed object, so json.loads is not optional. You run get_weather yourself (it’s a normal Python function; the model has no way to execute it), then append the result as a message with role: "tool" and the matching tool_call_id.

Only the second request, now carrying that result, gets you an actual sentence back. If you append the assistant message with its tool_calls but leave out the matching tool result, the API rejects the next request instead of guessing.

How Anthropic tool use differs from OpenAI function calling

Same mechanism, different field names. If you switch between providers or support both, this table is where your bugs will come from:

The trap is the arguments format. Anthropic hands you input already parsed into a Python dict or JS object; OpenAI hands you a string you must json.loads yourself. Code written against one and pointed at the other imports fine and then fails on the first tool call. If you support both, write one normalization function that both paths go through before your tool implementations ever see the arguments.

How parallel tool calls and multi-step loops work

Both APIs can return more than one tool call in a single response. Ask for the weather in three cities and you get three entries in tool_calls, not three separate round trips. Run them (concurrently, if they’re independent) and append all three results before the follow-up request. Every call ID the model emitted needs a result; send back one of three and the request fails.

Some tasks need several rounds: look up a user, then their order history, then flag the overdue order. Write this as a loop that keeps calling the model and running whatever it asks for until a response comes back with no tool calls. Then cap it. Nothing in the API stops a model from asking for the same tool nine times in a row when the results don’t move it closer to an answer. A hard limit, say 10 or 20 iterations, turns that into a clear error instead of a runaway bill.

Common function calling mistakes

Treating arguments as trusted input. args["city"] came out of a model, which means it came out of whatever text fed the conversation, including a scraped web page or a document the model was asked to summarize. Validate it before it touches a database query or a shell command. A tool that builds SQL by interpolating the model’s arguments has the same injection surface as one that builds SQL from a web form.

Trusting the schema as a guarantee. The model is good at following required and enum, but not perfect. Sooner or later it will pass "unit": "F" instead of "fahrenheit", or drop a required field. Validate, and return a clear error message as the tool result instead of crashing. Tell the model what was wrong and it can often retry with corrected arguments on the next round.

Letting the model pick destructive tools on its own. If a tool deletes data or sends an email, leaving tool_choice on automatic is the wrong default. Keep destructive tools out of the list until a human has approved the specific action, or make the effect reversible (a draft instead of a sent email) and let a separate step commit it.

When not to use function calling

Every tool call adds a round trip to the model plus whatever latency the tool itself has. If the “tool” would only ever return a static fact, put that fact in the prompt or in retrieved context and skip the round trip; that’s what RAG retrieval is for. And if the decision is really a classification with a small, fixed set of outcomes (route this ticket to one of four queues), a typed, non-generative call answers faster and cheaper than asking a general chat model to pick a tool. That’s the case system one models are built for.

Function calling earns its cost when the action depends on runtime data the model can’t have in its prompt, and the set of possible actions is too open to hardcode as a state machine. It’s also the loop underneath every CLI coding agent: read a file, run a test, decide what to do next, call another tool.

Start with one tool, a hardcoded Python function behind it, and a loop that stops after one round trip. Get argument validation and the error path right before you add a second tool. Most bugs in a multi-tool setup turn out to be the first tool mishandling arguments it wasn’t expecting.

Related articles

• What Is RAG: Retrieval-Augmented Generation Explained
• How CLI Coding Agents Work and How to Choose One
• What Is a System One Model and When to Use One
• What Are Embeddings and How Vector Search Works