ByteMind AI : Build. Break. Understand.

I had a chatbot that could answer questions about our product catalog. It was helpful — until someone asked “what’s the current stock for yoghurt in store 1234?” The model confidently hallucinated a number. It had no way to check. It just guessed.

The fix wasn’t a better prompt. It was giving the model the ability to call my APIs directly. One afternoon of wiring up function calling, and the chatbot went from a confident guesser to an assistant that checks real data before answering.

Function calling — also called tool use — is the mechanism that turns an LLM from a text generator into something that can interact with the real world. It’s the bridge between “I can write about your data” and “I can actually look it up.”

This post covers how function calling works under the hood, how to implement it properly, and the mistakes I made along the way so you don’t have to.

Related: This builds directly on the concepts from What Is an AI Agent? — if agents are the “what,” function calling is the “how.”

What Function Calling Actually Is

Here’s the misconception: people think the LLM executes code. It doesn’t. The LLM decides which function to call and with what arguments — then your code executes it, and you feed the result back.

The flow looks like this:

You: "What's the stock for product 45678 in store 1234?"
    ┌──────────────────────────────────────┐
    │  LLM receives message + tool list    │
    │                                      │
    │  Decides: I should call get_stock    │
    │  with spar_nummer=45678,             │
    │  store_id="1234"                     │
    └──────────────┬───────────────────────┘
                   │
                   ▼  (LLM returns structured tool call)
    ┌──────────────────────────────────────┐
    │  YOUR CODE executes the function     │
    │  result = get_stock(45678, "1234")   │
    │  → {"quantity": 23, "shelf": "A3"}   │
    └──────────────┬───────────────────────┘
                   │
                   ▼  (You send the result back to the LLM)
    ┌──────────────────────────────────────┐
    │  LLM incorporates the result         │
    │  "Store 1234 has 23 units of         │
    │   product 45678 on shelf A3."        │
    └──────────────────────────────────────┘

The LLM never touches your database. It never makes HTTP requests. It produces a structured JSON object that says “call this function with these arguments.” You execute it. You return the result. The LLM then uses that result to formulate its response.

This separation is critical for security, reliability, and debugging.

How It Works: The Three-Message Dance

Every function calling interaction follows the same three-step pattern, regardless of which provider you use. I’ll use the OpenAI SDK here, but Claude and Gemini work the same way conceptually.

Step 1: Define Your Tools

You describe your available functions as a JSON schema. The LLM reads these descriptions to decide when and how to call them.

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_stock",
            "description": "Returns the current stock quantity for a specific product in a specific store. Use this when someone asks about inventory, stock levels, or availability.",
            "parameters": {
                "type": "object",
                "properties": {
                    "spar_nummer": {
                        "type": "integer",
                        "description": "The unique product identifier (SPAR number)"
                    },
                    "store_id": {
                        "type": "string",
                        "description": "The store identifier, e.g. '1234'"
                    }
                },
                "required": ["spar_nummer", "store_id"]
            }
        }
    }
]

Notice how much detail goes into the description. The LLM doesn’t see your function implementation — it sees only the name, description, and parameter schema. If the description says “gets data,” the model won’t know when to use it. If it says “returns the current stock quantity for a specific product in a specific store,” it will.

Step 2: Send the Request

from openai import OpenAI
client = OpenAI()
messages = [
    {"role": "user", "content": "How much yoghurt do we have in store 1234?"}
]
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools
)

The model’s response won’t be text — it’ll be a tool call:

{
  "role": "assistant",
  "tool_calls": [
    {
      "id": "call_abc123",
      "type": "function",
      "function": {
        "name": "get_stock",
        "arguments": "{\"spar_nummer\": 45678, \"store_id\": \"1234\"}"
      }
    }
  ]
}

The model figured out that “yoghurt” maps to spar_nummer 45678 (from context or previous messages), chose the right function, and structured the arguments correctly.

Step 3: Execute and Return

You execute the function, then send the result back:

import json
# Execute the function call
tool_call = response.choices[0].message.tool_calls[0]
arguments = json.loads(tool_call.function.arguments)
# Your actual API call
stock_result = get_stock(
    spar_nummer=arguments["spar_nummer"],
    store_id=arguments["store_id"]
)
# Send the result back to the LLM
messages.append(response.choices[0].message)  # The assistant's tool call
messages.append({
    "role": "tool",
    "tool_call_id": tool_call.id,
    "content": json.dumps(stock_result)
})
# Get the final response
final_response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools
)
print(final_response.choices[0].message.content)
# → "Store 1234 currently has 23 units of yoghurt on shelf A3."

That’s the complete flow. Three messages: user request, tool call, tool result. The LLM then synthesizes a natural-language answer from real data.

Parallel and Sequential Tool Calls

Here’s where it gets interesting. Models don’t just call one function at a time.

Parallel Calls

Ask “what’s the stock for yoghurt and milk in store 1234?” and the model returns two tool calls in a single response:

{
  "tool_calls": [
    {
      "id": "call_001",
      "function": { "name": "get_stock", "arguments": "{\"spar_nummer\": 45678, \"store_id\": \"1234\"}" }
    },
    {
      "id": "call_002",
      "function": { "name": "get_stock", "arguments": "{\"spar_nummer\": 45679, \"store_id\": \"1234\"}" }
    }
  ]
}

You execute both, return both results, and the LLM combines them into one answer. This matters for performance — you can run these API calls concurrently.

import asyncio
async def execute_parallel_tools(tool_calls):
    tasks = [execute_tool(tc) for tc in tool_calls]
    return await asyncio.gather(*tasks)

Sequential Calls

Ask “find the cheapest product in store 1234 and check its stock” and the model will make two separate rounds — first calling search_products to find the cheapest item, then using that result to call get_stock. It can’t parallelize these because the second call depends on the first call’s output.

This is the reasoning loop from the AI agents post in action. The model reasons about what it learned and decides the next step.

Tool Description Quality: The Make-or-Break Factor

I spent two days debugging why my agent kept calling get_promotions when users asked about pricing. The function worked fine. The model just didn’t understand when to use it.

The problem was the description: "Get promotions for a product." What does “promotions” mean? Discounts? Marketing campaigns? Loyalty points?

I changed it to: "Returns active price promotions (bonus deals, temporary discounts) for a product. Use this when someone asks about deals, discounts, special offers, or bonus pricing — NOT for regular shelf price." The misrouting stopped immediately.

Here’s my checklist for tool descriptions:

• State what it returns, not just what it does
• Include when to use it — and when NOT to use it
• Use the vocabulary your users would use — if they say “deals,” mention “deals” in the description
• Describe edge cases — “Returns an empty array if no promotions are active”
• Add parameter descriptions — don’t assume the model knows what spar_nummer means

Bad:

{ "description": "Gets product data" }

Good:

{
  "description": "Returns detailed product information including name, category, brand, price, and nutritional data for a specific product identified by its SPAR number. Use this when someone asks about a product's details, ingredients, or price. Does NOT return stock levels — use get_stock for inventory."
}

The Anthropic (Claude) Way: Same Concept, Different Shape

Claude’s function calling works identically in principle, with slightly different syntax. Here’s the same example with Claude:

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-sonnet-4-6-20250514",
    max_tokens=1024,
    tools=[
        {
            "name": "get_stock",
            "description": "Returns current stock quantity for a product in a store",
            "input_schema": {
                "type": "object",
                "properties": {
                    "spar_nummer": {"type": "integer", "description": "Product identifier"},
                    "store_id": {"type": "string", "description": "Store identifier"}
                },
                "required": ["spar_nummer", "store_id"]
            }
        }
    ],
    messages=[
        {"role": "user", "content": "Stock for product 45678 in store 1234?"}
    ]
)

# Claude returns tool_use content blocks
for block in response.content:
    if block.type == "tool_use":
        result = execute_tool(block.name, block.input)
        # Send result back as tool_result

The key difference: OpenAI uses parameters, Claude uses input_schema. OpenAI returns tool_calls, Claude returns tool_use content blocks. The mental model is identical.

What Didn’t Work: Lessons Learned

The Model Invented Arguments

I had a tool that accepted an optional date_range parameter. When users didn’t mention dates, the model would sometimes invent one — passing "last_30_days" or "2026-01-01" as if the user had asked for it. The fix: remove optional parameters that the model shouldn’t guess. If the default behavior is what you want 90% of the time, don’t expose the parameter at all. Handle the default in your implementation.

Too Many Tools Caused Confusion

I started with 3 tools. Worked great. I added 15 more. The model started calling the wrong tools, especially when descriptions overlapped. With 5 tools for different types of “product data,” it frequently picked the wrong one.

The sweet spot I found: 7-10 well-described tools. Beyond that, group related tools into a single tool with a action parameter, or use a router pattern where one LLM call picks the tool category and a second call picks the specific tool.

Structured Output Isn’t Always Structured

About 2-3% of the time, the model’s arguments JSON was malformed — missing a closing brace, or including a trailing comma. Always parse tool call arguments in a try/except block:

try:
    args = json.loads(tool_call.function.arguments)
except json.JSONDecodeError:
    # Ask the model to try again with a correction prompt
    messages.append({
        "role": "tool",
        "tool_call_id": tool_call.id,
        "content": "Error: your function arguments were not valid JSON. Please try again."
    })

This self-correction pattern recovers from parse errors gracefully without crashing the entire flow.

When to Use Function Calling vs. RAG vs. Fine-Tuning

Function calling and RAG are complementary, not competing. My production agents use both: RAG for policy questions, function calling for live data. The model decides which approach to use based on the question.

Final Thoughts: Tools Are Your API Contract With the LLM

Function calling is deceptively simple — define a schema, let the model call it, execute the result. The hard part isn’t the mechanism. It’s the design: which tools to expose, how to describe them, and how to handle the inevitable edge cases.

The single most impactful thing I did was treat tool descriptions like public API documentation. Clear names. Explicit descriptions. Documented parameters. When-to-use and when-not-to-use notes. The model is your API consumer — and it reads the docs more carefully than most developers do.

If you’ve been building chatbots that guess at data, give function calling thirty minutes. Wire up one read-only endpoint. Watch the model call it correctly on the first try. That moment — when the model stops guessing and starts checking — is when LLM integration clicks.

Previously: What Is an AI Agent?. Next: Multi-Agent Systems: How I Built AI Agents That Delegate, Retry, and Collaborate.