Multiple, parallel and forced calls

The message arrived at 10:02 on a Saturday: “Is the eggless truffle cake in stock at Koregaon Park and Baner, and what’s the delivery fee to Aundh? Need it by 4.”

Arjun’s assistant now had three tools: get_order_status from the last lesson, plus two new ones Priya had helped him write, check_stock (asks a store’s inventory system) and get_delivery_fee (asks the delivery partner for a quote). The answer it gave was perfect: in stock at Baner, sold out at Koregaon Park, ₹60 to Aundh, and a suggestion to order from Baner.

It took eleven seconds. The customer had typed “hello??” twice while waiting.

Arjun opened the logs, expecting the model to be the slow part. It wasn’t. The model had done something clever: in its first response it had written all three slips at once. Then his code had run them one after another: Koregaon Park stock (2.1 seconds, the store’s system is slow), Baner stock (2.0 seconds), delivery fee (3.0 seconds). Seven seconds of his own code standing in line.

And there was a second problem waiting on his desk. Meera wanted every refund conversation to end with a record in the refunds log, no exceptions. Arjun had read about a setting that forces the model to call a particular tool. He tried it. The API answered with a 400 error.

“Two lessons in one morning,” Priya said. “How to handle many calls at once, and how to make sure a call happens when this model won’t let you force it.”

One message, three slips

Here is what Claude’s first response to the 10:02 message actually contained, after its (empty-by-default) thinking block:

ToolUseBlock(id='toolu_01Fq...', name='check_stock',      input={'product': 'eggless truffle cake', 'store': 'Koregaon Park'})
ToolUseBlock(id='toolu_01Hb...', name='check_stock',      input={'product': 'eggless truffle cake', 'store': 'Baner'})
ToolUseBlock(id='toolu_01Zt...', name='get_delivery_fee', input={'area': 'Aundh'})
stop_reason: tool_use

Three tool_use blocks in one response. This is called parallel tool use: when the model can see that several lookups do not depend on each other, it requests them together. It is on by default for Claude and for OpenAI models.

“Back to the restaurant,” Priya said. “The waiter comes to the hatch with three slips: a soup, a salad, a dessert. The kitchen has three stations. How should they cook?”

“All at once. Soup station, salad station, dessert station.”

“What did your kitchen do?”

Arjun looked at his loop. “Made the soup. Then made the salad. Then made the dessert. One cook, three stations idle.”

“The model did its part. It handed you three independent slips at once. Your code threw that away.”

Following the 10:02 message through two kitchens

Priya had Arjun write out the timeline for both versions.

Kitchen A, tools run one after another (what happened):

Time	What happens
0.0 to 1.9 s	Model call 1: returns three tool_use blocks
1.9 to 4.0 s	check_stock Koregaon Park (2.1 s)
4.0 to 6.0 s	check_stock Baner (2.0 s)
6.0 to 9.0 s	get_delivery_fee Aundh (3.0 s)
9.0 to 11.2 s	Model call 2: writes the answer

Kitchen B, tools run at the same time:

Time	What happens
0.0 to 1.9 s	Model call 1: returns three tool_use blocks
1.9 to 4.9 s	All three tools run together; the slowest (delivery fee, 3.0 s) decides
4.9 to 7.1 s	Model call 2: writes the answer

From 11.2 seconds to 7.1, without touching the model, the prompts or the bill. Tool time went from the sum of the tools (7.1 s) to the slowest tool (3.0 s). This is the same overlap-the-waiting idea as the batching lesson in Module 05, now inside a single conversation.

What did the model receive in call 2? One user message holding all three results, each tagged with the id of the slip it answers:

{"role": "user", "content": [
    {"type": "tool_result", "tool_use_id": "toolu_01Fq...", "content": '{"store": "Koregaon Park", "in_stock": 0}'},
    {"type": "tool_result", "tool_use_id": "toolu_01Hb...", "content": '{"store": "Baner", "in_stock": 3}'},
    {"type": "tool_result", "tool_use_id": "toolu_01Zt...", "content": '{"area": "Aundh", "fee_inr": 60, "eta_minutes": 45}'},
]}

The order of the results inside that message does not matter. The ids do the matching.

Opening the hood: parallel versus chained

Not every multi-tool question can run in parallel. There are two shapes, and the model handles both through the same loop you wrote last lesson.

Parallel calls are independent: stock at Baner does not depend on stock at Koregaon Park. They arrive as several tool_use blocks in one response.

Chained calls (also called sequential calls) depend on each other: the output of one is the input to the next. Consider: “Can I add a box of kaju katli to my order CR-1043?” The model first needs to know which store is preparing CR-1043, and only then can it check stock at that store. So it requests get_order_status alone, waits for the result ("store": "Aundh"), and in its next response requests check_stock for Aundh. Two rounds of the loop, one tool each.

flowchart LR
    Q["Customer question"] --> M1["Model turn 1"]
    M1 -->|"Independent lookups"| P["Several tool_use blocks at once"]
    M1 -->|"Needs a result first"| S["One tool_use block"]
    S --> M2["Model turn 2 uses the result"]
    M2 --> S2["Next tool_use block"]
    P --> M3["Model writes the answer"]
    S2 --> M3

You do not tell the model which shape to use. It works it out from the question and your tool descriptions. Your loop’s job is simply to run whatever it is handed, as efficiently as possible, and keep going while stop_reason == "tool_use".

Show the answer

(a) Parallel: two independent get_delivery_fee calls in one response. (b) Chained: the model must check the order status first; whether a refund question even arises depends on the answer. © One call: a single check_stock. Note that the model may occasionally choose differently from you, for example doing (a) in two rounds. That is still correct, just slower; clear descriptions make the efficient choice more likely.

Kitchen B in code

Here is the upgraded tool-running step. Two new tools first, with sleeps standing in for slow real systems:

import json
import time
from concurrent.futures import ThreadPoolExecutor

def check_stock(product: str, store: str) -> dict:
    time.sleep(2)   # pretend the store's inventory system is slow
    stock = {("eggless truffle cake", "baner"): 3, ("eggless truffle cake", "koregaon park"): 0}
    return {"store": store, "product": product, "in_stock": stock.get((product.lower(), store.lower()), 0)}

def get_delivery_fee(area: str) -> dict:
    time.sleep(3)   # pretend the delivery partner's quote API is slow
    fees = {"aundh": 60, "baner": 40, "wakad": 80}
    fee = fees.get(area.lower())
    if fee is None:
        return {"error": f"We don't deliver to {area} yet. Areas: {', '.join(fees)}"}
    return {"area": area, "fee_inr": fee, "eta_minutes": 45}

TOOL_FUNCTIONS = {
    "get_order_status": get_order_status,   # from the previous lesson
    "check_stock": check_stock,
    "get_delivery_fee": get_delivery_fee,
}

A dictionary from tool name to function replaces the if chain in last lesson’s run_tool. Now the part that changes, running every requested tool at once:

def safe_run(name: str, tool_input: dict) -> dict:
    """Run one tool; never let an exception escape into the loop."""
    func = TOOL_FUNCTIONS.get(name)
    if func is None:
        return {"error": f"Unknown tool {name}"}
    try:
        return func(**tool_input)
    except Exception as err:                      # a bug or outage inside the tool
        return {"error": f"{name} failed: {type(err).__name__}"}

def run_tools(content) -> list[dict]:
    calls = [b for b in content if b.type == "tool_use"]
    # Run all requested tools at the same time; the slowest one decides the wait
    with ThreadPoolExecutor(max_workers=8) as pool:
        outputs = list(pool.map(lambda c: safe_run(c.name, c.input), calls))
    results = []
    for call, output in zip(calls, outputs):
        result = {"type": "tool_result", "tool_use_id": call.id, "content": json.dumps(output)}
        if "error" in output:
            result["is_error"] = True               # tell the model this is a failure, not data
        results.append(result)
    return results

Line by line, the new ideas:

• ThreadPoolExecutor runs ordinary (non-async) functions in several threads at once. pool.map(f, calls) calls f on every item concurrently and returns the outputs in the same order as the inputs, so zip(calls, outputs) pairs each result with its slip. (If your tools are async def, asyncio.gather from Module 05 does the same job.)
• safe_run guarantees every slip gets an answer, even if a tool crashes. A missing tool_result would make the next API call fail with a 400.
• "is_error": True marks a failed tool so the model reads it as “this lookup failed”, not as data, and explains it honestly.

The loop from last lesson changes in only one line: results = run_tools(response.content). Everything still goes back in one user message.

Arjun ran the 10:02 question again: 7.3 seconds. He added a “Checking stock and delivery…” message in the chat while tools ran, and the “hello??” messages stopped.

Steering the model with tool_choice

So far the model has decided entirely on its own whether to call tools. The tool_choice parameter lets you steer that decision. Across providers there are four ideas:

Idea	Claude tool_choice	OpenAI tool_choice	Meaning
Model decides	{"type": "auto"} (default when tools are given)	"auto"	Call tools if useful, or just answer
No tools	{"type": "none"}	"none"	Answer in text only, even though tools are listed
Must call some tool	{"type": "any"}	"required"	Must call at least one tool
Must call this tool	{"type": "tool", "name": "..."}	{"type": "function", "function": {"name": "..."}}	Must call exactly this one

Two more switches control parallelism. On Claude, add "disable_parallel_tool_use": True inside tool_choice to allow at most one tool call per response. On OpenAI, pass parallel_tool_calls=False. You want this when calls must happen one at a time, for example two refund requests where the second should see the first.

none is more useful than it looks. At the end of a long support conversation you might ask for a plain-text summary for the human team. none guarantees a text answer without you having to change the tool list your code sends on every turn.

The 400 error: when you cannot force a call

Now Meera’s requirement. Every refund conversation must produce a record with the order id, the amount and the reason. Arjun defined a log_refund_request tool and, to make it unmissable, forced it:

response = client.messages.create(
    model="claude-opus-5-5", max_tokens=2000,
    tools=[LOG_REFUND_TOOL],
    tool_choice={"type": "tool", "name": "log_refund_request"},   # force this exact tool
    messages=[{"role": "user", "content": conversation}],
)

The API refused:

anthropic.BadRequestError: Error code: 400 - ... tool_choice: type "tool" and "any" are not supported for this model.

This is a real, documented difference. On claude-opus-5-5 (and some other recent Claude models), forced tool choice, both any and a named tool, is not supported. Older Claude models accept it. OpenAI models accept their equivalents ("required" and a named function). The same idea, the same English word “force”, works on one model and fails on another from the same provider.

“This is the real lesson,” Priya said. “Every provider, and sometimes every model, supports a slightly different set of features. Tutorials written six months ago will show code that 400s today. When something fails, read the error message first. It usually tells you exactly what’s unsupported. Then check the docs for your model, and design so you don’t depend on the missing feature.”

And there are three good ways not to depend on it.

Option 1: auto plus a clear instruction, then check. Tell the model what you expect, and verify in code that it happened. If it didn’t, ask once more.

LOG_REFUND_TOOL = {
    "name": "log_refund_request",
    "description": "Record a customer's refund request in Crumb's refunds log.",
    "strict": True,                                   # arguments must match the schema exactly
    "input_schema": {
        "type": "object",
        "properties": {
            "order_id": {"type": "string", "description": "Order id like CR-1042"},
            "amount_inr": {"type": "number", "description": "Refund amount requested, in rupees"},
            "reason": {"type": "string", "enum": ["late_delivery", "damaged_item", "wrong_item", "quality", "other"]},
        },
        "required": ["order_id", "amount_inr", "reason"],
        "additionalProperties": False,
    },
}

def extract_refund(conversation: str, attempts: int = 2) -> dict:
    messages = [{"role": "user", "content":
                 "Read this support conversation and record the refund request by calling "
                 "the log_refund_request tool.\n\n" + conversation}]
    for _ in range(attempts):
        response = client.messages.create(
            model="claude-opus-5-5", max_tokens=2000,
            tools=[LOG_REFUND_TOOL], tool_choice={"type": "auto"},
            messages=messages,
        )
        call = next((b for b in response.content
                     if b.type == "tool_use" and b.name == "log_refund_request"), None)
        if call:
            return call.input                          # already matches the schema, thanks to strict
        # No call this time: show the model its reply and ask directly
        messages += [{"role": "assistant", "content": response.content},
                     {"role": "user", "content": "Please call log_refund_request now with the details."}]
    raise RuntimeError("Model did not record the refund")

"strict": True on the tool definition (with additionalProperties: False and every property listed in required) means that when the model calls the tool, the arguments are guaranteed to match the schema exactly: right keys, right types, a reason from the allowed list. auto controls whether it calls; strict controls what shape the call has.

In Arjun’s tests on 200 refund conversations, the model called the tool on the first try in 198 and on the second try in the other two.

Option 2: if you only wanted JSON, use structured outputs. Look at what Meera actually needed: a record with three fields. Nothing about it requires a tool. Forcing a tool call was, in older code, a popular trick for getting guaranteed JSON. Structured outputs from Module 05 do that job directly, and they are guaranteed: messages.parse(..., output_format=RefundRecord) always returns an object matching the schema (or tells you clearly why it couldn’t). The next-but-one lesson goes deep on this.

Option 3: take the decision away from the model. If something must happen every time, it is not a decision. It is a step in your code.

Show the answer

Option 2 plus Option 3. “Every refund conversation must be logged” is a business rule, so your code should run it unconditionally: when a conversation is tagged as a refund, extract the details with structured outputs, then call your own log_refund(record) function. Option 1 leaves a mandatory step to the model’s judgement and then patches the gaps with retries. Tools are for when the model should decide whether and when to act, like looking up stock only when the customer asks. Use the model for judgement and your code for rules.

Where it breaks

Too many tools. Each tool definition costs input tokens on every request: a typical definition is 100 to 200 tokens, so 30 tools add 3,000 to 6,000 tokens to every call, about ₹1 to ₹2 extra per request on Claude Opus 5.5. Worse, similar tools confuse the choice: get_order, get_order_status and lookup_order_details will be mixed up. Keep tool sets small, names distinct, and descriptions clear about when each applies. Module 13 shows patterns for agents with many tools.

Parallel calls with side effects. Two parallel issue_refund calls for the same order is a real way to lose money. For any tool that changes something, consider disable_parallel_tool_use, make the tool idempotent (safe to repeat; Module 19), and confirm with a human before irreversible actions.

Splitting results across messages. Arjun once “tidied” his loop so each tool_result went into its own user message. It worked, but over the following week the model made noticeably fewer parallel calls. Anthropic’s guidance is explicit: return all results for one turn in a single user message.

Thread-unsafe tools. Running tools in threads is safe for independent network calls. If two tools write to the same file or in-memory structure, running them concurrently can corrupt it. Keep shared writes behind a lock, or in a database designed for concurrent use.

Try it yourself

1. Add check_stock and get_delivery_fee to your bot from the last lesson, with the run_tools function above.
2. Ask the 10:02 question. Time it with time.perf_counter() around ask(...).
3. Replace pool.map with a plain for loop that runs tools one by one, and time it again.
4. Try tool_choice={"type": "any"} once with claude-opus-5-5 and read the error.

Expected: about 7 seconds with the thread pool, about 11 without; the logs show three tool calls in one model turn; step 4 raises BadRequestError with a message about tool_choice.

Stuck? Hint

If the model makes the three calls across separate turns instead of one, check that every result goes back in a single user message, and that your descriptions make the tools clearly independent. Run it a few times; models vary slightly from run to run.

Common mistakes

• Running parallel tool calls one by one. The model gave you independent work; run it concurrently.
• Dropping a result. Every tool_use id needs a tool_result, even on failure (is_error: True), or the next call fails.
• Splitting results into separate messages. Return all of a turn’s results in one user message.
• Copying tool_choice settings between models or providers. Forced choice is unsupported on claude-opus-5-5; check your model’s docs and read the error.
• Using a forced tool call just to get JSON. Structured outputs do that directly.
• Leaving business rules to the model. If it must always happen, do it in code.

Explaining it to Meera

“Two things changed,” Arjun told Meera. “When a customer asks three things at once, the assistant already asked for all three lookups together. My code was then doing them one after another, like one cook making soup, then salad, then dessert. Now all three happen at once, so the answer comes in seven seconds instead of eleven.”

“And my refunds log?”

“I tried to order the AI to fill in the log every time, and this model doesn’t allow that setting; other models do. But it made me realise it shouldn’t be the AI’s decision at all. Now, whenever a conversation is marked as a refund, our own system fills in the log. The AI’s job is only to read the conversation and pull out the order number, amount and reason, in a fixed format.”

“So the rule lives with us, and the reading lives with the AI.”

“Exactly.”

Recap

• Models can request several independent tools in one response (parallel) or one after another across turns (chained); the same loop handles both.
• Run parallel calls concurrently and return every result, tagged by id, in one user message; mark failures with is_error.
• tool_choice steers tool use: auto, none, and, on models that support it, forced any or a named tool; disable_parallel_tool_use limits a turn to one call.
• claude-opus-5-5 rejects forced tool choice with a 400: use auto plus an instruction and a check, strict: true for exact arguments, or structured outputs when you only need JSON.
• Use the model for judgement and your code for rules.

Two days later the monsoon arrived early. Deliveries across Pune ran late, and the chat filled with “will my cake reach before the party?” The honest answer needed two things Crumb did not have inside its own walls: the delivery partner’s live tracking, and the weather. “Time to let the model reach outside the building,” said Priya.

Go deeper

• Anthropic docs (tool use, tool choice, parallel tool use): https://docs.anthropic.com
• OpenAI docs (function calling): https://platform.openai.com/docs
• Building effective agents: https://www.anthropic.com/research/building-effective-agents