Structured outputs and custom formats

After the support drafts, Meera had a new wish. “Every morning, I want a spreadsheet of yesterday’s reviews. One row per review: which product, a score out of five, and whether it mentions an allergy. I’ll read it with my chai.”

Arjun wrote a nightly job. It pulled the day’s reviews from the website, sent each one to the model, and wrote the answers into a CSV file. He tested it on ten reviews on Tuesday afternoon. Perfect.

On Wednesday, Meera opened the spreadsheet. It had 36 rows. Yesterday there had been 212 reviews.

Arjun opened the logs. The job had crashed at 2:14 a.m. on review number 37 with json.decoder.JSONDecodeError: Expecting value: line 1 column 1. His code had expected JSON. The model had replied:

Sure! Here's the analysis you asked for:
{"product": "plum cake", "score": 3, "mentions_allergy": true}
Let me know if you need anything else!

One friendly sentence before the data and one after, and the whole night’s work stopped. Priya looked at the log and laughed, kindly. “You asked a very chatty writer to be a database. Let’s give it a form to fill in instead.”

Arjun’s first fix, and why it was not one

Here is the heart of the job that crashed:

import json

PROMPT = """Analyse this review and give me the product, a score out of 5,
and whether it mentions allergies, as JSON.

Review: {review}"""

def analyse(review: str) -> dict:
    resp = client.chat.completions.create(
        model=MODEL, messages=[{"role": "user", "content": PROMPT.format(review=review)}])
    return json.loads(resp.choices[0].message.content)   # crashes on any extra text

Arjun’s first instinct was to clean up the reply before parsing. He grabbed everything between the first { and the last }:

import re

def analyse(review: str) -> dict:
    text = ...  # same call as before
    match = re.search(r"\{.*\}", text, re.S)   # find the JSON-looking part
    return json.loads(match.group(0))

It ran all Wednesday night without crashing. Thursday morning, Meera’s spreadsheet had 198 rows, and some of them were strange. Arjun went through the week’s logs and counted the problems across 1,040 reviews:

Thirty-three bad rows out of 1,040, about 3%. Most did not crash; they quietly put wrong data into Meera’s sheet. A spreadsheet that is wrong 3% of the time, without telling you which 3%, is hard to trust at all.

“The regex fixed the crash,” Priya said, “but not the problem. What’s the actual problem?”

Arjun thought. “The model doesn’t know exactly what I want. I said ‘a score out of 5’, and it gave me ‘4/5’, which is a perfectly good score out of five.”

“Right. You described the answer in English. English is generous. Code isn’t.”

Forms, not letters

Priya drew two things on the whiteboard: a blank page, and a form with labelled boxes.

“When the tax office wants to know your income, do they ask you to write them a letter about it?”

“No, there’s a form.”

“Why?”

“Because then everyone’s answer is in the same place, in the same format. They can process millions of them.”

“And on the online version of that form, what happens if you type ‘about four lakh’ into the income box?”

“It won’t let you. The box only takes numbers.”

“That’s the whole lesson. Three levels. First, you can ask for a form-shaped answer in the prompt. Second, you can use a form that refuses anything that doesn’t fit: the online form. Third, someone still checks the numbers make sense, because a form will happily accept a wrong number as long as it’s a number.”

The technical name for the form-shaped answer is a structured output: a reply in a fixed, machine-readable shape. The usual shape is JSON (JavaScript Object Notation), a plain-text format of keys and values like {"score": 4} that nearly every programming language can read. The description of the form itself, which boxes exist and what each accepts, is called a schema.

Level one: ask for the exact form

The first improvement is a prompt that spells out the form, box by box:

Extract data from the review inside <review> tags.

Return ONLY a JSON object with exactly these keys and no other text:
{
  "product": string,           // the product named, lowercase, or "unknown"
  "score": integer 1-5,        // your estimate of the customer's rating
  "mentions_allergy": boolean, // true if any allergy or allergen is mentioned
  "allergens": string[]        // e.g. ["walnuts"], or [] if none
}

<review>
{review}
</review>

Three things changed. Every key and type is named. Every field says what to do when the information is missing ("unknown", []). And the reply is told to contain nothing else.

Arjun reran last week’s 1,040 reviews. Bad rows dropped from 33 to 6. Much better, and still not zero, because a prompt is a request. At Crumb’s volume, 6 a week is fine to fix by hand. At ten thousand a day it would be sixty broken rows every day.

Following one review through a real schema

Level two is the online form that refuses wrong input. Before the code, Priya and Arjun traced a single review all the way through, so every step is visible.

The review, from Thursday:

“Loved the plum cake but it had walnuts and nobody told us. My son is allergic. Three stars because the taste was honestly great.”

Step 1: describe the form in Python. Pydantic is a Python library that lets you describe data as a class and then checks real data against it. Each field has a name, a type and, optionally, limits and a description:

from typing import Literal
from pydantic import BaseModel, Field

class ReviewAnalysis(BaseModel):
    product: str = Field(description="Product named, lowercase, or 'unknown'")
    score: int = Field(ge=1, le=5, description="The customer's rating out of 5")
    mentions_allergy: bool
    allergens: list[str] = Field(description="Allergens mentioned, or []")
    sentiment: Literal["positive", "mixed", "negative"]

ge=1, le=5 means “greater than or equal to 1, less than or equal to 5”. Literal[...] means “one of exactly these three words”. That one is a menu, not a text box.

Step 2: Pydantic turns the class into a schema. A JSON Schema is the standard, language-neutral way to describe a valid JSON shape. Calling ReviewAnalysis.model_json_schema() produces it. Trimmed, it looks like this:

{
  "properties": {
    "product": {"type": "string", "description": "Product named, lowercase, or 'unknown'"},
    "score": {"type": "integer", "minimum": 1, "maximum": 5},
    "mentions_allergy": {"type": "boolean"},
    "allergens": {"type": "array", "items": {"type": "string"}},
    "sentiment": {"enum": ["positive", "mixed", "negative"], "type": "string"}
  },
  "required": ["product", "score", "mentions_allergy", "allergens", "sentiment"]
}

You never write this by hand. The SDK sends it to the provider for you.

Step 3: the model generates, inside the form. The reply comes back as exactly this text, nothing before or after:

{"product": "plum cake", "score": 3, "mentions_allergy": true, "allergens": ["walnuts"], "sentiment": "mixed"}

Step 4: the SDK parses and validates it into a ReviewAnalysis object. result.score is the integer 3, not the string "3". result.allergens is a real Python list.

Step 5: your code writes the row. plum cake,3,True,walnuts,mixed. One clean line in Meera’s spreadsheet.

Step 6: the cost. The schema adds about 150 tokens of input to each call, and the tidy JSON reply is about 40 tokens, far shorter than the chatty version, which ran to 70. At illustrative small-model prices ($0.25permillioninputtokens,$2 per million output), the whole call costs about $0.00014, or ₹0.012. All 212 of Wednesday’s reviews: about ₹2.60. Structured outputs often make answers cheaper, because the model stops writing “Sure! Here’s…”

Opening the hood: how a schema can be enforced

“Hang on,” Arjun said. “How can the provider force the model? It’s still predicting tokens one at a time.”

“Exactly that way. One token at a time.” Priya wrote a half-finished reply on the board: {"product": "plum cake", "score":

“What tokens could legally come next, if the score must be an integer from 1 to 5?”

“A digit. 1, 2, 3, 4 or 5.”

“So the provider checks every possible next token against the schema and blocks the ones that would break it. " is blocked, because a string is not allowed. S of ‘Sure’ is blocked. The letter f is blocked. Only the five digits remain, and the model picks the most likely of those.”

This technique is called constrained decoding. At each step, the model still produces probabilities for every token in its vocabulary, but tokens that would make the output invalid are masked out before one is chosen. That is why, with strict structured outputs on, “Sure! Here’s your JSON” is literally impossible: the very first token must be {.

It also explains what structured outputs cannot do. The constraint is on shape. Among the valid digits 1 to 5, the model still picks whichever it thinks is likely. If it misreads the review, it will pick the wrong digit with perfect formatting.

The real code, step by step

Step 1: install and set up. uv add pydantic anthropic openai python-dotenv. Keep the ReviewAnalysis class from above in a file called schemas.py.

Step 2: structured output with Claude. The Anthropic SDK accepts the Pydantic class directly and gives you a parsed object back:

import anthropic
from dotenv import load_dotenv
from schemas import ReviewAnalysis

load_dotenv()
claude = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

def analyse_with_claude(review: str) -> ReviewAnalysis:
    response = claude.messages.parse(
        model="claude-opus-5-5",
        max_tokens=2000,
        system="You analyse customer reviews for Crumb & Co., a bakery in Pune.",
        messages=[{"role": "user", "content": f"<review>\n{review}\n</review>"}],
        output_format=ReviewAnalysis,   # the form the answer must fit
    )
    return response.parsed_output       # a ReviewAnalysis object, not a string

r = analyse_with_claude("Loved the plum cake but it had walnuts and nobody told us. "
                        "My son is allergic. Three stars because the taste was great.")
print(r)

You should see product='plum cake' score=3 mentions_allergy=True allergens=['walnuts'] sentiment='mixed'.

Step 3: the same idea with OpenAI. OpenAI’s chat endpoint has a JSON mode, response_format={"type": "json_object"}, which guarantees valid JSON (though not your exact keys). Pair it with the form-describing prompt and validate with Pydantic yourself. One quirk: JSON mode requires the word “JSON” to appear somewhere in your messages.

import os
from openai import OpenAI
from pydantic import ValidationError

oai = OpenAI()
MODEL = os.getenv("OPENAI_MODEL", "gpt-5-mini")  # model names change; see the models page

def analyse_with_openai(review: str) -> ReviewAnalysis:
    resp = oai.chat.completions.create(
        model=MODEL,
        response_format={"type": "json_object"},   # always valid JSON
        messages=[
            {"role": "system", "content": "Return JSON matching this schema: "
                + str(ReviewAnalysis.model_json_schema())},
            {"role": "user", "content": f"<review>\n{review}\n</review>"},
        ],
    )
    return ReviewAnalysis.model_validate_json(resp.choices[0].message.content)

model_validate_json reads the text, checks every field against the class, and either returns a clean object or raises a ValidationError that names exactly which field broke which rule. The OpenAI SDK also has parse helpers that accept a Pydantic class and enforce your exact schema, like the Claude example. Library APIs change quickly. If a name has moved, check the docs linked below.

Step 4: check meaning, and retry with the error. Pydantic can hold your own rules too. Here is a check that the score and sentiment do not contradict each other:

from pydantic import model_validator

class CheckedReview(ReviewAnalysis):
    @model_validator(mode="after")
    def score_matches_sentiment(self):
        if self.sentiment == "negative" and self.score >= 4:
            raise ValueError("negative sentiment with a score of 4 or 5")
        if self.mentions_allergy and not self.allergens:
            raise ValueError("mentions_allergy is true but allergens is empty")
        return self

When validation fails, the most effective retry tells the model what went wrong:

def analyse_safely(review: str, attempts: int = 2) -> CheckedReview | None:
    note = ""
    for _ in range(attempts):
        raw = ask_model(review + note)          # your call from step 2 or 3, returning text
        try:
            return CheckedReview.model_validate_json(raw)
        except ValidationError as err:
            note = f"\n\nYour previous answer was invalid: {err.errors()[0]['msg']}. Fix it."
    return None                                  # give up: send to a human queue

Returning None and flagging the review for a person is a feature, not a failure. Module 05 builds this into a full error-handling strategy.

Step 5: write the rows. With objects instead of strings, the CSV step is a few lines:

import csv

with open("reviews.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["product", "score", "mentions_allergy", "allergens", "sentiment"])
    for r in results:
        writer.writerow([r.product, r.score, r.mentions_allergy, ";".join(r.allergens), r.sentiment])

Forms for human readers, too

Not every reader is code. Meera also wanted a weekly summary she could scan in thirty seconds, in the same layout every week. The same principle applies: show the skeleton, do not describe it.

Before: “Write a short structured summary of this week’s reviews.” Every week came back with different headings, different order, different length.

After:

Summarise this week's reviews in exactly this format, nothing else:

## Top praise
- (one line each, max 3)

## Top complaints
- (one line each, max 3)

## Allergy mentions
(count) reviews. Products: (comma-separated, or "none")

## One thing to fix this week
(one sentence)

Other human-friendly formats follow the same rule: a Markdown table (“columns: Product, Reviews, Average score”), CSV for pasting into a spreadsheet (say which separator, and whether text is quoted), or XML-style tags like <summary>...</summary><actions>...</actions> when your code needs to pull out sections but a strict schema would be overkill. Tags are forgiving: a regular expression can extract them even if the model adds a stray line elsewhere.

Where it breaks

Structured outputs fixed the crashes. Over the next month, Arjun hit subtler problems, each worth knowing.

The forced invention. Every field in his schema was required. A review in Marathi that only said “छान!” (“lovely!”) still had to have a product. With no product mentioned, the model picked the most common one: “chocolate cake”. Required fields make the model invent when the information is absent. The fix: an explicit escape hatch ("unknown") or an optional field (str | None), plus a line in the description saying when to use it.

The menu that was too short. He briefly made product a Literal of Crumb’s eight best-sellers. Then a review mentioned the new pumpkin loaf. The model, forced to choose from eight, picked “banana bread”. Closed lists are powerful, but always include an "other" option, and track how often it is chosen.

The cut-off reply. One day a long review produced an answer that stopped mid-way, because max_tokens was set too low and the reply hit the limit. Schema enforcement cannot finish an answer the model was not allowed to finish. Set max_tokens with room to spare, and treat a truncated reply as a failure to retry.

The deep, clever schema. An ambitious version nested products inside orders inside customers, five levels deep. Accuracy dropped and debugging got miserable. Flat schemas with clear field names work best, for models and for people.

Try it yourself

Crumb’s WhatsApp ordering line gets messages like this:

“hi can i get 2kg black forest for saturday 6pm, pickup from koregaon park store, write happy 60th papa on it. eggless pls”

Write a Pydantic model and a prompt that extracts: product, weight in kg, pickup or delivery, store or address, date and time as written, the message for the cake, and an eggless flag. Test it on this message and on one that is missing the time.

from typing import Literal
from pydantic import BaseModel, Field

class CakeOrder(BaseModel):
    product: str
    weight_kg: float
    fulfilment: Literal["pickup", "delivery", "unknown"]
    location: str | None = Field(description="Store for pickup, address for delivery, or null")
    when: str | None = Field(description="Date and time exactly as the customer wrote it, or null")
    cake_message: str | None = Field(description="Text to write on the cake, or null")
    eggless: bool

Common mistakes

• Trusting “return only JSON” in production. It works most of the time. At scale, “most” means daily breakage. Use the API’s structured output feature.
• No escape hatch for missing data. Required fields without an “unknown” or null option force the model to make something up.
• Closed lists without “other”. New products, new complaint types and new languages will arrive. Leave room for them.
• Validating shape but not meaning. Add rules for contradictions, and measure accuracy against hand-labelled examples.
• Tight max_tokens. A truncated answer cannot be valid. Leave headroom and treat truncation as an error.

Recap

• Code readers need structured outputs; a description in English is too generous for code.
• A schema is the form; Pydantic lets you write it as a Python class and generates the JSON Schema for you.
• Constrained decoding enforces the schema token by token, so the shape is guaranteed but the content is not.
• Validate meaning in code, retry with the specific error, and send stubborn cases to a human.
• Give the model escape hatches (unknown, null, other) so it never has to invent.
• For human readers, show a literal skeleton instead of describing a format.

The next Monday, Meera’s spreadsheet had 1,163 rows and not one of them was weird. She asked Arjun what he had changed. “Before, I asked the AI to write me a letter and then tried to cut the numbers out of it,” he said. “Now it fills in a form, and the form physically will not accept anything in the wrong box. Then my code double-checks that the answers make sense, like a 5-star review that calls us terrible, and anything odd goes to Kavita.”

“So it can still be wrong?”

“It can still misjudge a review. But it can’t break the spreadsheet any more, and we’ll know which rows to double-check.”

Meera nodded, then added, “Lovely. Now I need the same thing for the Diwali hamper reviews, and slightly different for wholesale cafés.” Arjun copied the prompt into two new files. Priya watched him do it and said nothing, which, he was learning, meant he was about to learn something.

Go deeper

• Pydantic documentation: https://docs.pydantic.dev
• OpenAI documentation, structured outputs: https://platform.openai.com/docs
• Anthropic documentation, structured outputs: https://docs.anthropic.com