FastAPI basics for AI applications

Crumb’s WhatsApp ordering line is run by a small team with its own service, written in Node.js. It handles about 900 customer messages a day. Their lead, Farah, messaged Arjun on Monday: “We love the ticket triage you built. Can our service use it? We’d like every incoming WhatsApp message classified before a human sees it.”

Arjun sent back the path to his Python file, triage.py, and a note: “Just import triage().”

Farah’s reply came in seconds: “We can’t import a Python file into Node. Different language, different server, different machine. Give us a URL we can POST to.”

Priya laughed when he showed her. “Right now your AI lives in a kitchen with no counter. It cooks beautifully, but nobody outside the kitchen can order from it. You need a front counter: a menu of what can be ordered, a slip format, and a promise about what comes back. On the web, that counter is an API. In Python, the quickest way to build a good one is FastAPI.”

What Farah’s service actually needs

An API (application programming interface) is a way for one program to ask another to do something. A web API does it over HTTP, the same protocol your browser uses to load pages. You met HTTP in the Developer Toolkit; here is the part that matters now.

Every HTTP request has a method (the verb: GET to fetch, POST to send something new), a path (like /triage), some headers (labels such as “this body is JSON”), and optionally a body (the data). Every response has a status code (200 OK, 404 not found, 422 invalid input, 500 server crashed), headers, and a body.

Each combination of method and path you offer is an endpoint. Farah needs one: “POST a message to /triage, get back a category and urgency as JSON.” Her Node code does not care that the other side is Python, or that it calls an LLM. It only cares about the contract: what to send, what comes back, and what the status codes mean.

Arjun’s first attempt

Arjun had seen FastAPI in a tutorial. Twenty minutes later he had this:

from fastapi import FastAPI
from triage import triage   # his function from the JSON lesson

app = FastAPI()

@app.post("/triage")
def triage_endpoint(body: dict):
    message = body["message"]
    if len(message) == 0:
        return {"error": "empty message"}
    return triage(message).model_dump()

It worked when he tested it. Then Farah’s team started integrating, and three things happened on the first afternoon:

1. Their code sent {"msg": "..."} instead of {"message": "..."}. The line body["message"] raised a KeyError, and FastAPI returned 500 Internal Server Error: “the server crashed”, with no hint that the request was the problem.
2. An empty message got back {"error": "empty message"} with status 200 OK. Farah’s code checked the status, saw success, and stored “error” as a ticket category.
3. One customer forwarded an entire PDF menu as text, 38,000 characters. It went straight to Claude. That one call cost about ₹4, and the reply took 25 seconds.

“Notice the pattern,” Priya said. “Every one of those is a slip that should have been handed back at the counter. Instead it went into the kitchen, and the kitchen either caught fire or cooked something expensive.”

The counter, the menu and the slip

“Walk me through ordering at the Aundh counter,” Priya said.

“There’s a menu board. You can only order what’s on it. You fill in a slip: item, quantity, name. The person at the counter checks it. If it’s wrong, they hand it back and tell you what’s wrong. If it’s right, it goes to the kitchen, and you get a receipt with a number.”

“Map it.”

Arjun tried. “The menu board is the list of endpoints. The slip is… the request body?”

“The slip format is the request model: which boxes, what goes in each one. The person checking it?”

“Validation. And handing it back with a reason is like… a 4xx error that says which box was wrong.”

“And the receipt?”

“The response model. A promise about what I give back.”

“Last one. In your code, who checks the slip?”

Arjun looked at body: dict. “Nobody. dict means ‘accept anything’.”

That is the key misconception to drop: a web endpoint is just a function you can call over the internet. It is a function plus a contract plus a bouncer. FastAPI’s whole design is that you write the contract once, as Pydantic models and type hints, and it builds the bouncer, the error messages and the documentation for you.

Where the analogy breaks: a human at a counter can be flexible (“you wrote ‘choc cake’, I know what you mean”). A validator is not, by design. Strictness at the boundary is what lets the code behind it stay simple.

Following one request through FastAPI

Here is what happens, step by step, when Farah’s service sends a valid request to the finished endpoint you will build below. This is the actual text that travels over the network:

POST /triage HTTP/1.1
Host: crumb-ai.internal:8000
Content-Type: application/json

{"message": "My hamper CR-2210 hasn't arrived and guests come at 7"}

1. Routing. FastAPI matches method POST and path /triage to your function.
2. Parsing. It reads the body and parses the JSON into a Python dictionary.
3. Validation. It checks the dictionary against your request model: is there a message key, is it a string, is it between 1 and 4,000 characters? Yes.
4. Your function runs, receiving a TriageRequest object, not a raw dictionary. It calls the model, which takes about 3 seconds and costs about ₹0.35.
5. Serialising. FastAPI checks your return value against the response model and converts it to JSON.
6. Response.

HTTP/1.1 200 OK
content-type: application/json

{"categories":["late_delivery"],"urgency":"high","order_id":"CR-2210","summary":"Hamper CR-2210 not delivered before a 7 pm party."}

Now the {"msg": ...} request from the first afternoon. Steps 1 and 2 are the same. At step 3 validation fails, and FastAPI stops right there. Your function never runs, Claude is never called, nothing is billed, and the response is:

HTTP/1.1 422 Unprocessable Content
content-type: application/json

{"detail":[{"type":"missing","loc":["body","message"],"msg":"Field required","input":{"msg":"My hamper..."}}]}

loc says exactly where the problem is: in the body, the field message is missing. Farah’s developer fixed their side in two minutes, without messaging Arjun. That is the counter handing the slip back with a reason. The 38,000-character PDF gets the same treatment: a 422 saying “String should have at most 4000 characters”, in about a millisecond, for ₹0.

flowchart LR
    A["Request arrives"] --> B["Route by method and path"]
    B --> C["Parse JSON body"]
    C --> D{"Valid for the request model?"}
    D -->|"No"| E["422 with details, your code never runs"]
    D -->|"Yes"| F["Your function"]
    F --> G["Check against response model"]
    G --> H["JSON response"]

Building it, one piece at a time

Step 1: set up the project

Create a fresh project with uv and add FastAPI. The [standard] extra brings the development server and the fastapi command.

uv init crumb-api
cd crumb-api
uv add "fastapi[standard]" python-dotenv

If you use plain pip instead: pip install "fastapi[standard]" python-dotenv inside a virtual environment.

Replace the generated main.py with the smallest useful app:

# main.py
from fastapi import FastAPI

app = FastAPI(title="Crumb API")

@app.get("/health")
def health() -> dict:
    """A tiny endpoint other services can call to check we are alive."""
    return {"status": "ok"}

Run it:

uv run fastapi dev main.py

Behind that command is Uvicorn, a small, fast web server that speaks HTTP and hands each request to your FastAPI app. fastapi dev also reloads the app whenever you save a file. In production you use fastapi run, which skips the reloader.

Open http://127.0.0.1:8000/health and you see {"status":"ok"}. Now open http://127.0.0.1:8000/docs. FastAPI has built an interactive documentation page where you can try every endpoint with a button. Farah’s team used this page as their integration guide. You wrote no documentation code.

Step 2: routes with path and query parameters

A route connects a method and path to a function. Parts of the path in curly braces are path parameters; extra function arguments become query parameters (the ?key=value part of a URL). Type hints tell FastAPI what to expect and convert.

from fastapi import FastAPI, HTTPException

app = FastAPI(title="Crumb API")

# A pretend database. In real life this is your order database.
ORDERS = {
    "CR-1042": {"status": "cancelled", "store": "Baner", "items": ["Plum cake"]},
    "CR-1043": {"status": "out_for_delivery", "store": "Aundh", "items": ["Kaju katli box"]},
}

@app.get("/orders/{order_id}")
def get_order(order_id: str):
    order = ORDERS.get(order_id.upper())
    if order is None:
        # HTTPException becomes a proper error response, not a crash
        raise HTTPException(status_code=404, detail=f"No order {order_id}")
    return {"order_id": order_id.upper(), **order}

@app.get("/orders")
def list_orders(store: str | None = None, limit: int = 10):
    # Query parameters: /orders?store=Baner&limit=5
    rows = [{"order_id": k, **v} for k, v in ORDERS.items()]
    if store:
        rows = [r for r in rows if r["store"].lower() == store.lower()]
    return rows[:limit]

Try /orders/CR-1043, /orders/CR-9999 (a 404 with a clear detail), and /orders?limit=abc. The last one returns a 422 explaining that limit must be an integer. FastAPI checked the type before your function ran, exactly like the body check above.

HTTPException fixes Arjun’s second bug. Instead of return {"error": ...} with a misleading 200, you raise an exception with the right status code, and FastAPI turns it into a proper error response.

Step 3: request and response models

Now the counter slip. For POST requests, describe the body with a Pydantic model. Pydantic, which you used in this module’s first lesson to validate model output, now validates incoming requests. Declare a response_model too: it documents what you return and filters out anything extra.

from pydantic import BaseModel, Field

class FeedbackIn(BaseModel):
    """What a client must send."""
    store: str = Field(min_length=2, examples=["Baner"])
    rating: int = Field(ge=1, le=5, description="1 to 5 stars")
    comment: str = Field(max_length=2000)

class FeedbackOut(BaseModel):
    """What we promise to send back."""
    id: int
    message: str

FEEDBACK: list[FeedbackIn] = []

@app.post("/feedback", response_model=FeedbackOut, status_code=201)
def create_feedback(feedback: FeedbackIn) -> FeedbackOut:
    FEEDBACK.append(feedback)
    return FeedbackOut(id=len(FEEDBACK), message=f"Thanks! We'll share this with the {feedback.store} team.")

ge=1, le=5 means “greater than or equal to 1, less than or equal to 5”. status_code=201 is the conventional code for “created”. Test it with curl, a command-line tool for sending HTTP requests:

curl -X POST http://127.0.0.1:8000/feedback \
  -H "Content-Type: application/json" \
  -d '{"store": "Baner", "rating": 5, "comment": "Loved the plum cake"}'

You get {"id":1,"message":"Thanks! We'll share this with the Baner team."} with status 201. Send "rating": 9 and you get a 422 naming the field and the rule it broke.

Step 4: the AI endpoint’s contract

Now Arjun could rewrite Farah’s endpoint properly. In this lesson we define the contract and keep the model call simple; the next lesson makes it production-grade.

from triage import triage, TicketTriage   # TicketTriage is the Pydantic model from the JSON lesson

class TriageRequest(BaseModel):
    message: str = Field(min_length=1, max_length=4000, description="The customer's message")

@app.post("/triage", response_model=TicketTriage)
def triage_endpoint(req: TriageRequest) -> TicketTriage:
    return triage(req.message)

Compare it with the first attempt. The KeyError is gone, because a missing message never reaches the function. The empty-message if is gone, because min_length=1 handles it with a proper 422. And the 38,000-character menu is stopped at the door by max_length=4000. For an AI endpoint, that one argument is three things at once: input validation, a cost cap (no request can send more than about 1,000 tokens of message to the model), and a latency cap.

Configuration and secrets

Your AI endpoints need secrets, starting with the API key. An environment variable is a named setting the operating system passes to a program. It is the standard place for secrets and for settings that differ between your laptop, a test server and production. Locally, you keep them in a .env file, which python-dotenv loads, and which must never be committed to Git.

# .env  (add ".env" to .gitignore before your first commit!)
ANTHROPIC_API_KEY=sk-ant-...
CLAUDE_MODEL=claude-opus-5-5
MAX_INPUT_CHARS=4000

Load settings once, in one place, and fail loudly at startup if something essential is missing:

# config.py
import os
from dataclasses import dataclass
from dotenv import load_dotenv

load_dotenv()

@dataclass(frozen=True)
class Settings:
    anthropic_api_key: str
    claude_model: str
    max_input_chars: int

def load_settings() -> Settings:
    key = os.getenv("ANTHROPIC_API_KEY")
    if not key:
        # Crash at startup, not on the first customer's request
        raise RuntimeError("ANTHROPIC_API_KEY is not set. Add it to .env or the server environment.")
    return Settings(
        anthropic_api_key=key,
        claude_model=os.getenv("CLAUDE_MODEL", "claude-opus-5-5"),
        max_input_chars=int(os.getenv("MAX_INPUT_CHARS", "4000")),
    )

settings = load_settings()

Then from config import settings anywhere. frozen=True makes the settings read-only, so no code can quietly change the model halfway through a request. Changing the model for a test server becomes a one-line change in that server’s environment, not a code change and a redeploy.

Why fail at startup? Because the alternative is an app that starts happily, passes its health check, and then fails on the first real customer at 9 a.m. A server that refuses to start with a clear message is the kindest kind of failure.

A sensible project layout

As the app grows, split it up. Arjun’s ended up like this:

crumb-api/
  main.py          # creates the FastAPI app and the routes
  config.py        # settings from the environment
  schemas.py       # Pydantic request and response models
  services/
    llm.py         # every call to the model lives here
  .env             # secrets (never committed)
  pyproject.toml   # dependencies, managed by uv

The rule that pays off fastest: every call to the model lives in services/llm.py. Retries, timeouts, logging, token counting and the error handling from earlier lessons then live in one place, instead of being copy-pasted (and slowly drifting apart) across every route.

Where it breaks

A few things that tripped Arjun up in the first week:

• Port already in use. Running two copies of the app gives “address already in use”. Stop the first one, or pass --port 8001.
• Forgetting Content-Type: application/json. Without it, some clients send the body as form data and you get a 422 that looks baffling. Read the loc in the error.
• /docs in production. The interactive docs page is wonderful internally and an invitation on the public internet. You can turn it off with FastAPI(docs_url=None) or protect it; Module 24 covers API security properly.
• Slow AI calls block a plain def endpoint’s worker. FastAPI runs plain def endpoints in a thread pool, which is fine at low traffic, but each waiting call occupies a thread for several seconds. The next lesson switches to async def with the async client, and that is where the real fix lives.

Try it yourself

Add an endpoint GET /stores/{store}/orders/count that returns {"store": "...", "count": N} using the ORDERS dictionary. Return a 404 if no orders exist for that store. Then check it in /docs.

Expected: /stores/baner/orders/count returns {"store": "baner", "count": 1}; /stores/kharadi/orders/count returns a 404 with a helpful detail.

@app.get("/stores/{store}/orders/count")
def count_orders(store: str):
    count = sum(1 for o in ORDERS.values() if o["store"].lower() == store.lower())
    if count == 0:
        raise HTTPException(status_code=404, detail=f"No orders for store {store}")
    return {"store": store, "count": count}

Common mistakes

• Taking dict as a request body. You lose validation, error messages and docs. Define a Pydantic model.
• Returning errors with status 200. return {"error": ...} tells the caller it worked. Raise HTTPException with the right code.
• No length limits on text fields. On an AI endpoint, every unbounded string is an unbounded bill. Add max_length.
• Reading os.environ all over the code. Centralise settings so missing keys fail at startup, in one obvious place.
• Committing .env. Add it to .gitignore before the first commit. If a key leaks, rotate it immediately.

Explaining it to Meera

Meera asked what “the API” Farah kept mentioning actually was.

“Think of the counter at Aundh,” Arjun said. “Before, my AI was a kitchen with no counter. Only I could use it, from my laptop. Now it has a counter with a menu board: other systems, like the WhatsApp service, can walk up and order ‘classify this message’. There’s a slip they have to fill in properly. If they get it wrong, the counter hands it back with a note saying exactly which box is wrong, and the kitchen never even sees it.”

“Why does that matter to me?”

“Because last Monday a customer forwarded a whole menu PDF, and it went straight to the AI and cost four rupees in one go. Now anything longer than a page is handed back at the counter for free. And when the WhatsApp team made a mistake, they could see what was wrong themselves, instead of calling me.”

Recap

• A web API exposes endpoints (method plus path) that any program, in any language, can call over HTTP.
• FastAPI maps routes to Python functions and converts path and query parameters using your type hints.
• Pydantic request models reject bad input with a 422 before your code runs, which for AI endpoints also means before you pay.
• Use the right status codes: HTTPException(404) for missing things, 4xx for caller mistakes, 5xx only for your own failures.
• Secrets and settings come from environment variables, loaded once and checked at startup.

Farah’s service switched to the new endpoint that afternoon, and the “error” category disappeared from her dashboard. But within an hour she was back. “We got two bare 500 errors at lunch with no explanation. And Meera’s asking whether the website chat can type its answers out live, like ChatGPT.” Priya grinned. “The counter’s built. Now we make the AI behind it production-grade.”

Go deeper

• FastAPI documentation and tutorial: https://fastapi.tiangolo.com
• Pydantic documentation: https://docs.pydantic.dev
• uv documentation: https://docs.astral.sh/uv/