Anatomy of an LLM API call

It is Diwali week, and Arjun’s script now drafts replies that the support team reviews and sends. On Tuesday morning, Kavita from support forwards a complaint. A customer received this, word for word:

“Dear Mrs Kulkarni, we are truly sorry that your Diwali hamper arrived late. We understand how important”

That was the whole email. It ended mid-sentence. Kavita had been clearing forty drafts in a hurry and pressed send without reading to the end.

Arjun opens his code. Two days earlier, to keep replies short and cheap, he had set the reply limit to 60 tokens. Then he checks the logs from that morning and finds something worse: three drafts that were completely empty, and one run where his script crashed with an error he had never seen, StopIteration.

“It just gives up randomly,” he tells Priya.

“It never gives up randomly. Every response tells you exactly why it stopped and exactly what it cost. Your code threw the receipt away and kept only the cake.”

Ordering a cake, and reading what comes back

Priya uses the shop counter, because Crumb takes custom cake orders every day.

“When a customer orders a custom cake, what goes on the order form?”

“Which baker. What to write on top. Size, flavour, ‘no nuts’, deadline.”

“And what comes back with the cake?”

Arjun thinks. “The cake. A receipt. And if something went wrong, a note: ‘ran out of time, icing unfinished’.”

“An API call is exactly that. The order form is the request: which model, what messages, what limits. The box that comes back is the response: the content, a note on why it stopped, and a receipt in tokens. You read the cake and ignored the note that said icing unfinished.”

Where the analogy breaks: a baker who runs out of time tells you. An API puts the note in a field that your code must go and read. If you never read it, you never know.

Following one call over the wire

Before the SDK conveniences, look at what actually travels. Every SDK call becomes a plain web request: an HTTPS POST carrying a JSON body. Here is one of Arjun’s drafting calls to Claude, exactly as sent:

POST https://api.anthropic.com/v1/messages
x-api-key: sk-ant-...your key...
anthropic-version: 2023-06-01
content-type: application/json

{
  "model": "claude-opus-5-5",
  "max_tokens": 1000,
  "system": "You are Crumb's support assistant. Reply warmly in under 100 words.",
  "messages": [
    {"role": "user", "content": "My Diwali hamper (order CR-20931) arrived two days late. Not happy."}
  ]
}

Line by line:

• The address (/v1/messages) says which API operation you want: “generate a message”.
• The headers carry your key and the API version. The SDK adds these for you.
• model: which LLM runs your request.
• max_tokens: the most tokens the model may produce in this reply. Required on Claude.
• system: the system prompt, standing instructions that shape every reply.
• messages: the conversation so far, each with a role saying who is speaking. Here there is one message, from the user.

And here is the response that came back (the long signature string shortened):

{
  "id": "msg_01Hk...",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-5-5",
  "content": [
    {"type": "thinking", "thinking": "", "signature": "EuYB..."},
    {"type": "text", "text": "Dear customer, we're truly sorry your Diwali hamper (CR-20931) arrived two days late. ..."}
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {"input_tokens": 512, "output_tokens": 186}
}

Three parts to read every single time:

1. content: what the model produced, as a list of blocks. There are two here. The first is a thinking block: Claude Opus 5.5 always reasons privately before it answers, and by default the API returns that block with its text left empty. The second is the text block, the actual reply. This is why Claude code picks out the text blocks rather than taking the first one.
2. stop_reason: why the model stopped. end_turn means it finished naturally.
3. usage: the receipt. 512 tokens read, 186 tokens written.

Now turn the receipt into money, at Claude Opus 5.5’s prices at the time of writing ($4permillioninputtokens,$20 per million output tokens):

	Tokens	Calculation	Cost
Input	512	512 × $4 ÷ 1,000,000	$0.0020
Output	186	186 × $20 ÷ 1,000,000	$0.0037
Total			$0.0058, about ₹0.51

Why 512 input tokens for a short complaint? The system prompt, the message, and a little formatting the API wraps around each message. And why 186 output tokens for a reply of about 80 words, which should be roughly 105 tokens? Because output_tokens includes the hidden thinking. You never see it, but you pay for it.

Why Arjun’s replies broke: three stop reasons

The stop_reason field (OpenAI and Gemini call it the finish reason) is the note in the box. These are the values you will meet most:

What happened	Claude	OpenAI	Gemini
Finished naturally	end_turn	stop	STOP
Hit your max_tokens limit: reply is cut off	max_tokens	length	MAX_TOKENS
Hit one of your stop sequences	stop_sequence	stop	STOP
Declined or blocked for safety	refusal	content_filter	SAFETY

Now Arjun’s three failures make sense.

The cut-off email. With max_tokens at 60, the model reached its 60th token in the middle of “We understand how important” and the API stopped it. stop_reason was max_tokens. His code never looked.

The empty drafts. Remember that thinking counts as output. On harder emails, the model spent all 60 tokens thinking and had none left for the reply. The response contained a thinking block and no text block. stop_reason was max_tokens again.

The crash. Arjun’s code pulled out the text with next(b.text for b in response.content if b.type == "text"). With no text block, next() had nothing to return and raised StopIteration. One malformed draft crashed the whole batch.

The settings you will use again and again

With the response understood, here are the request settings in full. Not every model accepts every one.

max_tokens (OpenAI: max_completion_tokens, Gemini: max_output_tokens). The hard ceiling on tokens produced in this reply, including hidden thinking. Set it generously. On Claude it is required; on the others it is optional, and if you leave it out the model can write up to its own maximum.

Stop sequences (OpenAI: stop, Gemini and Claude: stop_sequences). A list of strings that end the reply as soon as the model writes one. The stop sequence itself is not included in the returned text. Useful when you want only part of a pattern: ask for a numbered list and stop at "4." to get exactly three items.

Temperature and top-p. How adventurously the model samples its next token, from the previous lesson. Claude Opus 5.5 and OpenAI’s GPT-5 reasoning models do not accept custom values. Gemini Flash does.

Effort (Claude: output_config={"effort": "low" | "medium" | "high"}). How much the model thinks before answering. On Claude Opus 5.5 the default is medium. For simple jobs like a short apology, low is faster and uses fewer output tokens. OpenAI’s reasoning models have a similar setting, reasoning_effort.

The model name. The biggest setting of all. It decides quality, speed and price. The lesson after next is about choosing one.

Same idea, three different shapes

All three providers do the same job, but their SDKs present it differently. Here is the same call on each, printing the text, the stop reason and the receipt.

Claude:

import anthropic
from dotenv import load_dotenv

load_dotenv()
client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=1000,
    system="You are Crumb's support assistant. Reply warmly in under 100 words.",
    messages=[{"role": "user", "content": "My Diwali hamper (CR-20931) arrived two days late."}],
)
print(next((b.text for b in response.content if b.type == "text"), ""))
print("stop reason:", response.stop_reason)
print("tokens in/out:", response.usage.input_tokens, response.usage.output_tokens)

OpenAI (Chat Completions):

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
# Model names change; check the provider's models page for the current list.
MODEL = os.getenv("OPENAI_MODEL", "gpt-5-mini")
client = OpenAI()  # reads OPENAI_API_KEY

resp = client.chat.completions.create(
    model=MODEL,
    messages=[
        {"role": "system", "content": "You are Crumb's support assistant. Reply warmly in under 100 words."},
        {"role": "user", "content": "My Diwali hamper (CR-20931) arrived two days late."},
    ],
    max_completion_tokens=1000,
)
choice = resp.choices[0]
print(choice.message.content)
print("finish reason:", choice.finish_reason)
print("tokens in/out:", resp.usage.prompt_tokens, resp.usage.completion_tokens)
print("of which reasoning:", resp.usage.completion_tokens_details.reasoning_tokens)

Gemini:

import os
from dotenv import load_dotenv
from google import genai
from google.genai import types

load_dotenv()
client = genai.Client()  # reads GEMINI_API_KEY

resp = client.models.generate_content(
    model=os.getenv("GEMINI_MODEL", "gemini-2.5-flash"),
    contents="My Diwali hamper (CR-20931) arrived two days late.",
    config=types.GenerateContentConfig(
        system_instruction="You are Crumb's support assistant. Reply warmly in under 100 words.",
        max_output_tokens=1000,
    ),
)
print(resp.text)
print("finish reason:", resp.candidates[0].finish_reason)
u = resp.usage_metadata
print("tokens in/out:", u.prompt_token_count, u.candidates_token_count, "thinking:", u.thoughts_token_count)

Library APIs change quickly. If a name has moved, check the docs linked below.

Side by side, the differences are mostly names and nesting:

	Claude	OpenAI	Gemini
System prompt	Top-level system	A message with role system	system_instruction in the config
The conversation	messages	messages	contents
Length ceiling	max_tokens (required)	max_completion_tokens	max_output_tokens
Stop sequences	stop_sequences	stop	stop_sequences
The text	text blocks in response.content	resp.choices[0].message.content	resp.text
Why it stopped	stop_reason	choices[0].finish_reason	candidates[0].finish_reason
Tokens used	usage.input_tokens, output_tokens	usage.prompt_tokens, completion_tokens	usage_metadata.prompt_token_count, candidates_token_count
Hidden thinking	Inside output_tokens	completion_tokens_details.reasoning_tokens	thoughts_token_count

Two shapes deserve a word.

• Why is OpenAI’s text inside choices[0]? You can ask OpenAI for several alternative replies in one call, so answers come back as a list. You nearly always want the first.
• Why is Claude’s text inside a list of blocks? A Claude response can mix kinds of content: thinking, text, and requests to use a tool (Module 06). Picking out the text blocks works no matter what else is there.

OpenAI also offers a newer Responses API (client.responses.create(model=MODEL, input="..."), then r.output_text), designed for agents and built-in tools. This course uses Chat Completions for OpenAI because its messages shape lines up most closely with the other two, which makes comparisons easier.

Opening the hood: one shape for all three

Comparing three shapes in every script gets tiring. Priya suggests Arjun write a small adapter: three functions that call each provider and return the same simple object. Every later piece of Crumb’s code can then ignore which provider answered.

Step 1: decide what you care about. A dataclass (a lightweight Python class for holding data) with the five things every response must report:

# llm.py
import os
from dataclasses import dataclass
from dotenv import load_dotenv

load_dotenv()

@dataclass
class Reply:
    provider: str
    text: str
    finish: str          # "done", "cut_off", "blocked" or "other"
    input_tokens: int
    output_tokens: int

Normalising the stop reason to four words means the rest of your code needs only one if, not twelve.

Step 2: the Claude adapter. Note the "" default in next(), which fixes Arjun’s crash.

import anthropic

def call_claude(system: str, user: str, max_tokens: int = 1000) -> Reply:
    client = anthropic.Anthropic()
    r = client.messages.create(
        model="claude-opus-5-5",
        max_tokens=max_tokens,
        system=system,
        messages=[{"role": "user", "content": user}],
    )
    finish = {"end_turn": "done", "stop_sequence": "done",
              "max_tokens": "cut_off", "refusal": "blocked"}.get(r.stop_reason, "other")
    text = next((b.text for b in r.content if b.type == "text"), "")  # "" if no text block
    return Reply("claude", text, finish, r.usage.input_tokens, r.usage.output_tokens)

Step 3: the OpenAI and Gemini adapters, same pattern:

from openai import OpenAI
from google import genai
from google.genai import types

def call_openai(system: str, user: str, max_tokens: int = 1000) -> Reply:
    client = OpenAI()
    r = client.chat.completions.create(
        model=os.getenv("OPENAI_MODEL", "gpt-5-mini"),
        messages=[{"role": "system", "content": system}, {"role": "user", "content": user}],
        max_completion_tokens=max_tokens,
    )
    c = r.choices[0]
    finish = {"stop": "done", "length": "cut_off", "content_filter": "blocked"}.get(c.finish_reason, "other")
    return Reply("openai", c.message.content or "", finish,
                 r.usage.prompt_tokens, r.usage.completion_tokens)

def call_gemini(system: str, user: str, max_tokens: int = 1000) -> Reply:
    client = genai.Client()
    r = client.models.generate_content(
        model=os.getenv("GEMINI_MODEL", "gemini-2.5-flash"),
        contents=user,
        config=types.GenerateContentConfig(system_instruction=system, max_output_tokens=max_tokens),
    )
    reason = getattr(r.candidates[0].finish_reason, "name", "OTHER") if r.candidates else "OTHER"
    finish = {"STOP": "done", "MAX_TOKENS": "cut_off", "SAFETY": "blocked"}.get(reason, "other")
    u = r.usage_metadata
    out = (u.candidates_token_count or 0) + (u.thoughts_token_count or 0)  # count thinking too
    return Reply("gemini", r.text or "", finish, u.prompt_token_count, out)

Step 4: use it, and refuse to ship a broken reply.

SYSTEM = "You are Crumb's support assistant. Reply warmly in under 100 words."
EMAIL = "My Diwali hamper (CR-20931) arrived two days late."

for call in (call_claude, call_openai, call_gemini):
    reply = call(SYSTEM, EMAIL)
    if reply.finish != "done" or not reply.text.strip():
        print(f"[{reply.provider}] NOT SAFE TO SEND: finish={reply.finish}")
        continue
    print(f"[{reply.provider}] {reply.input_tokens} in / {reply.output_tokens} out")
    print(reply.text, "\n")

You should see three apologies, each with its token counts. Now change max_tokens to 40 in the loop call (call(SYSTEM, EMAIL, max_tokens=40)) and run it again. Instead of half-written emails, you should see three NOT SAFE TO SEND: finish=cut_off lines. That one if would have saved Mrs Kulkarni’s email.

This little llm.py is the seed of something you will build properly in Module 19: a gateway that can switch between providers when one fails.

When no response comes back at all

Sometimes the request fails before any response exists, and the SDK raises an error instead. The number in the error is an HTTP status code, a standard number that summarises what went wrong:

Code	Meaning	What to do
400	Your request is malformed, for example a parameter this model rejects	Fix the code; retrying will not help
401	Missing or invalid API key	Check .env
404	Unknown model name or address	Check the models page
429	Too many requests, or out of credit	Wait and retry, or add credit
500 and above	The provider is having trouble	Wait and retry

The OpenAI and Anthropic SDKs already retry some of these (429 and 500-range errors) automatically, twice by default. You will build proper retry logic yourself in Module 05.

Where it breaks in production

• Reading only the text. A cut-off, empty or refused reply can all look like a normal string, or no string. Check the finish reason on every call.
• Using max_tokens as a length instruction. It is a guillotine, not a word count. Ask for brevity in the prompt; keep the ceiling generous.
• Starving reasoning models. Thinking spends the same budget. If you see empty text with a “cut off” reason, raise the ceiling or lower the effort.
• Stop sequences that appear naturally. A stop sequence of "\n\n" will end the reply at the first blank line, which might be after the greeting.
• Not logging usage. The usage numbers are your real cost data, request by request. Log them from day one. When Meera asks what Diwali week cost, you will have the answer.

Try it yourself

Use llm.py (or the Claude snippet) and run three experiments, printing text, finish reason and token counts each time.

1. Cut it off. Ask for “a 300-word welcome note for new Crumb staff” with max_tokens=150.
2. Use a stop sequence. On Claude, add stop_sequences=["4."] and ask: “List six ideas for Christmas cupcakes, numbered 1. to 6.”
3. Measure thinking. Ask Claude “What is 2 + 2?” twice: once as normal, once with output_config={"effort": "low"}. Compare output_tokens.

Show what you should expect

1. finish is cut_off (stop_reason is max_tokens). The note is truncated mid-sentence, or the text is empty if the thinking used up the budget. Either way the finish reason tells the truth.
2. You get ideas 1 to 3 and then the text ends. stop_reason is stop_sequence, and response.stop_sequence is "4.". The stop sequence itself is not in the text.
3. Both answers are 4, but the low-effort call usually reports noticeably fewer output tokens, because it thinks less. Fewer tokens means cheaper and faster. Some OpenAI reasoning models reject stop; if you try experiment 2 there and see a 400 error, that is why.

Common mistakes

• “The text is the response.” The text is one part. The stop reason and usage are just as important.
• “A low max_tokens saves money.” You pay for what is produced. A low ceiling only produces broken replies.
• Taking content[0].text on Claude. The first block may be thinking. Pick out the text blocks.
• Ignoring hidden thinking in cost estimates. Output token counts include reasoning you never see.
• Retrying a 400. A malformed request fails the same way every time. Fix it instead.

Recap

• A request names a model, sends messages and an optional system prompt, and sets limits like max_tokens and stop sequences.
• A response contains the content, a stop or finish reason, and token usage.
• “Hit the limit” means the reply is cut off or even empty; always check before using the text.
• Reasoning models spend hidden thinking tokens, billed as output and counted against the ceiling.
• Claude, OpenAI and Gemini share the same ideas with different names, so a small adapter can give your code one shape.

Arjun ships the fix that afternoon: a generous ceiling, “under 100 words” in the instructions, and a check that holds back any draft whose finish reason is not done. He explains it to Meera: “Every reply comes with a note saying why it stopped and a receipt saying what it cost. We were reading only the reply. Now nothing goes out unless the note says it finished, and I can tell you to the paisa what Diwali week cost.”

Meera is impressed for about ten seconds. Then she forwards another draft from the review queue. A customer had asked whether order CR-20931 had shipped. The draft said, confidently, that it had left the Baner store yesterday and would arrive on Thursday.

“We still don’t have a Baner store,” says Meera. “And it doesn’t know anything about that order. How did it write that?”

Go deeper

• Claude docs, the Messages API and stop reasons: https://docs.anthropic.com
• OpenAI API reference and guides: https://platform.openai.com/docs
• Gemini API docs, text generation and config: https://ai.google.dev/gemini-api/docs