Project: Working with the OpenAI API

By December, Arjun had built a lot of small AI tools for Crumb: the Monday review summary, the menu translator, the tone fixer for store replies, the ticket router, the SQL helper. Each one had started as a quick script. Each one lived in a different folder. Priya asked him to list how they read the API key. There were four different ways, one of them a key pasted straight into the file, which he had not noticed until that moment.

Then, on the Monday before Christmas, the provider had a twenty-minute slowdown. The summary script crashed on review 140 and had to be restarted from scratch. The translator hung forever, because it had no timeout. The router, which did have a retry loop, retried instantly forty times in a row, hit the rate limit, and got Crumb’s key throttled for the rest of the morning.

Meera’s question from the last lesson was fair: they were held together with tape. Priya’s suggestion was to stop writing scripts and build one small, proper package: shared configuration, one well-behaved client, the text utilities as tested functions, and a command-line tool that anyone on the team could run.

The brief

Crumb’s team (Arjun, Kavita, Rohan in logistics, and occasionally Meera) needs a single tool for everyday text jobs. It must:

• Run from a terminal with clear commands: crumbkit summarise reviews.txt --reader "Meera" --words 250.
• Keep secrets out of the code, and settings (model, timeouts) in one place.
• Survive the provider having a bad day: time out instead of hanging, retry sensibly instead of hammering, and fail with a clear message when it gives up.
• Return validated, typed results where code depends on them (ticket routing), and checked results where people do (tone changes must not alter numbers).
• Be testable without network access, so tests are fast, free and deterministic.

This is the “production-style” version of everything in this module. The model calls are the easy part. The engineering around them is the project.

What you will build

Must-have

• A src/ layout package managed with uv, installable as a command.
• config.py: settings from environment variables and a .env file, with defaults and validation.
• client.py: one LLM class wrapping the OpenAI SDK with a timeout, retries with exponential backoff and jitter, and token-usage logging.
• text.py: summarise (with map-reduce), translate (with a keep-list and glossary), adjust_tone (with a number check), improve_writing (with a change list).
• tickets.py: route_ticket returning a validated Pydantic Ticket, with one corrective retry and a safe fallback.
• code.py: generate_code, propose_diff, explain_code.
• cli.py: subcommands for each utility, reading files or standard input.
• A pytest suite using a fake LLM, plus one optional live smoke test.

Nice-to-have

• A --json flag to output machine-readable results.
• A running cost estimate printed at the end of each command.
• Concurrency for map-reduce summaries.

Architecture

flowchart LR
    U[Terminal user] --> CLI["cli.py (argparse)"]
    CLI --> T["Task modules (text, tickets, code)"]
    T --> P["Prompt files"]
    T --> L["LLM client"]
    C["config.py (.env)"] --> L
    L --> R["Retry with backoff"]
    R --> API["OpenAI API"]
    TS["Tests"] --> F["FakeLLM"]
    F --> T

Terminal user and cli.py. The front door. It parses arguments, reads input, calls one task function, and prints the result. It contains no prompt text and no API logic, so it stays small.

Task modules. Plain Python functions, one per job. Each one takes an llm argument, builds a prompt from a template file, calls llm.complete(...), and validates or checks the result. Because the LLM is passed in rather than created inside, tests can pass in a fake.

Prompt files. Text templates in src/crumbkit/prompts/, versioned in git, as in Module 03’s templates lesson. Changing a prompt is a reviewed change to a file, not an edit buried in code.

LLM client. The only place that talks to the SDK. It sets the timeout, applies the retry policy, records token usage and turns SDK errors into one clear LLMError for callers.

config.py. Reads OPENAI_API_KEY, OPENAI_MODEL, timeouts and retry counts from the environment. Fails fast with a helpful message if the key is missing.

Retry with backoff. A small, generic function that retries only errors worth retrying, waiting longer each time.

Tests and FakeLLM. A stand-in with the same complete method that returns canned answers and records the prompts it received. Tests check your logic (prompt building, validation, fallbacks, the CLI) without any network calls.

Tech stack

Piece	Tool	Why
Language	Python 3.12	The course standard; good typing support
Project and packaging	uv	Fast installs, lockfile, uv init --package gives a src layout and a command entry point
Model API	OpenAI Python SDK	The brief; widely used; JSON mode for structured results
Settings	python-dotenv plus a dataclass	Secrets in .env, never in code; no extra framework needed
Validation	Pydantic	Typed models for tickets and code answers, clear validation errors
CLI	argparse (standard library)	Stable, no dependency, subcommands built in
Tests	pytest	Simple fixtures, capsys for CLI output, monkeypatch for environment

Build it, milestone by milestone

1. Set up the package, config and secrets

Why. Four ways of reading an API key means four places for a key to leak and four things to update when anything changes. One settings object, loaded once, removes the whole category of problem.

Create the project and add dependencies:

uv init --package crumbkit       # creates src/crumbkit/ and a [project.scripts] entry
cd crumbkit
uv add openai pydantic python-dotenv
uv add --dev pytest

(Without uv, pip install openai pydantic python-dotenv pytest inside a virtual environment works too.)

In pyproject.toml, point the command at your CLI’s main function:

[project.scripts]
crumbkit = "crumbkit.cli:main"

Then src/crumbkit/config.py:

import os
from dataclasses import dataclass
from dotenv import load_dotenv

@dataclass(frozen=True)
class Settings:
    api_key: str
    model: str
    timeout_s: float
    max_retries: int

def load_settings() -> Settings:
    load_dotenv()                                   # reads .env in the current folder, if present
    key = os.getenv("OPENAI_API_KEY")
    if not key:
        raise SystemExit("OPENAI_API_KEY is not set. Add it to .env or your environment.")
    return Settings(
        api_key=key,
        model=os.getenv("OPENAI_MODEL", "gpt-5-mini"),   # model names change; see the models page
        timeout_s=float(os.getenv("CRUMBKIT_TIMEOUT", "30")),
        max_retries=int(os.getenv("CRUMBKIT_MAX_RETRIES", "4")),
    )

Create .env with your key, and add .env to .gitignore before your first commit. Add a .env.example with the variable names and no values, so teammates know what to set.

Working looks like: uv run python -c "from crumbkit.config import load_settings; print(load_settings().model)" prints your model name. Deleting the key from .env prints the friendly error instead of a stack trace.

Most likely bug: committing .env. Run git status before the first commit and check it is not listed.

Stuck? Hint

If uv run crumbkit says the command is not found, check that [project.scripts] points at a module path that exists (crumbkit.cli:main needs src/crumbkit/cli.py with a main function), then run uv sync so the entry point is installed. A placeholder def main(): print("crumbkit") is enough for now.

2. Build a resilient client with retries and timeouts

Why. The December slowdown showed three failure modes: no timeout (hang forever), no retry (crash on a blip), and naive retry (hammer the API and get throttled). APIs fail in ordinary ways: a rate limit (the provider’s cap on how many requests or tokens you may send per minute, signalled by HTTP status 429), timeouts, dropped connections and occasional server errors (status 500 and above). Some are worth retrying. Others, like an invalid API key or a malformed request, will fail identically every time.

The standard policy is exponential backoff with jitter: after each failure, wait twice as long as last time (1 s, 2 s, 4 s, 8 s), plus a small random amount so that many clients failing together do not all retry in the same instant.

src/crumbkit/client.py:

import logging
import random
import time
from typing import Callable, TypeVar

import openai
from openai import OpenAI
from .config import Settings

log = logging.getLogger("crumbkit")
T = TypeVar("T")

RETRYABLE = (openai.RateLimitError, openai.APITimeoutError,
             openai.APIConnectionError, openai.InternalServerError)

class LLMError(Exception):
    """Raised when a call fails for good. Callers handle one error type."""

def with_retries(fn: Callable[[], T], attempts: int, retry_on: tuple = RETRYABLE,
                 base_delay: float = 1.0, sleep: Callable[[float], None] = time.sleep) -> T:
    for attempt in range(1, attempts + 1):
        try:
            return fn()
        except retry_on as err:
            if attempt == attempts:
                raise LLMError(f"Gave up after {attempts} attempts: {err}") from err
            delay = base_delay * 2 ** (attempt - 1) + random.uniform(0, 0.5)
            log.warning("attempt %d failed (%s); retrying in %.1fs", attempt, type(err).__name__, delay)
            sleep(delay)
    raise AssertionError("unreachable")

class LLM:
    def __init__(self, settings: Settings):
        # max_retries=0: our policy is in charge, so retries are not doubled up with the SDK's own
        self._client = OpenAI(api_key=settings.api_key, timeout=settings.timeout_s, max_retries=0)
        self._model = settings.model
        self._attempts = settings.max_retries
        self.tokens_in = self.tokens_out = 0

    def complete(self, system: str, user: str, json_mode: bool = False) -> str:
        kwargs = {"response_format": {"type": "json_object"}} if json_mode else {}
        def call():
            return self._client.chat.completions.create(
                model=self._model,
                messages=[{"role": "system", "content": system},
                          {"role": "user", "content": user}],
                **kwargs)
        try:
            resp = with_retries(call, self._attempts)
        except openai.APIStatusError as err:            # 400, 401, 403...: retrying will not help
            raise LLMError(f"Request rejected ({err.status_code}): {err.message}") from err
        self.tokens_in += resp.usage.prompt_tokens
        self.tokens_out += resp.usage.completion_tokens
        return resp.choices[0].message.content or ""

Notice sleep is a parameter. In tests you will pass a function that records delays instead of waiting, so a test of four retries runs in milliseconds. The OpenAI SDK also has built-in retries (max_retries on the client). Using the SDK’s is perfectly reasonable; the point here is to choose one policy deliberately and not stack two. Library APIs change quickly. If a name has moved, check the docs linked below.

Working looks like: a quick script calling LLM(load_settings()).complete("Reply in one word.", "Say hello") prints a greeting, and tokens_in and tokens_out are non-zero. Setting CRUMBKIT_TIMEOUT=0.001 produces three warning lines with growing delays, then a clean LLMError.

Most likely bug: retrying everything, including 401 (bad key) and 400 (bad request). Those fail instantly forever, so retrying only wastes time and hides the real message. Only the four retryable types go in RETRYABLE.

Stuck? Hint

If retries never trigger during your timeout test, check the order of the except clauses. APITimeoutError and APIConnectionError are not APIStatusErrors, so they reach with_retries correctly; but if you wrap the call in a broad except Exception anywhere, it will swallow them before your retry logic sees them.

3. Add summarise, translate and tone utilities

Why. These are the rewriting tasks from this module’s first lesson, now as reusable, checked functions. The guardrails you learned there (named reader, counts not adjectives, keep-lists, number checks) become code that every caller gets for free.

Put the briefs in prompt files, for example src/crumbkit/prompts/summarise_reduce.txt:

Combine these partial summaries for {reader}. Add up the counts.
Under {words} words: a one-line verdict, the top three complaints with
counts, the top two praises with counts. Use only information given.

A small loader in src/crumbkit/templates.py keeps file access in one place:

from importlib.resources import files

def prompt(name: str, **values) -> str:
    return (files("crumbkit") / "prompts" / f"{name}.txt").read_text().format(**values)

Then src/crumbkit/text.py:

import re
from .client import LLM
from .templates import prompt

RULES = ("You rewrite text for Crumb & Co., a family bakery in Pune. Never add facts, "
         "promises, dates or numbers not in the source. Use Western digits. "
         "Return only the result.")

def summarise(llm: LLM, items: list[str], reader: str, words: int = 250, batch: int = 50) -> str:
    partials = [llm.complete(RULES, prompt("summarise_map", n=len(chunk)) + "\n\n" + "\n".join(chunk))
                for chunk in (items[i:i + batch] for i in range(0, len(items), batch))]
    return llm.complete(RULES, prompt("summarise_reduce", reader=reader, words=words)
                        + "\n\n" + "\n\n".join(partials))

def translate(llm: LLM, text: str, language: str, keep: list[str], glossary: dict[str, str]) -> str:
    out = llm.complete(RULES, prompt("translate", language=language, keep=", ".join(keep),
                       glossary="\n".join(f"{k} -> {v}" for k, v in glossary.items()))
                       + f"\n\n<text>\n{text}\n</text>")
    lost = [w for w in keep if w in text and w not in out]
    if lost:
        raise ValueError(f"Translation changed protected terms: {lost}")
    return out

def _numbers(s: str) -> set[str]:
    return set(re.findall(r"\d+", s))

def adjust_tone(llm: LLM, text: str, formality: int, warmth: int, firmness: int, words: int = 80) -> str:
    out = llm.complete(RULES, prompt("tone", formality=formality, warmth=warmth,
                       firmness=firmness, words=words) + f"\n\n<text>\n{text}\n</text>")
    if missing := _numbers(text) - _numbers(out):
        raise ValueError(f"Rewrite dropped or changed numbers: {missing}")
    return out

Add improve_writing the same way, with a prompt that asks for the improved text followed by a Changes: list.

Working looks like: summarising a file of 120 reviews makes three map calls and one reduce call, and the result contains counts like “14 of 120”. Tone-adjusting “Refunds take 7 days” returns a warmer sentence containing “7”.

Most likely bug: a KeyError from .format because a prompt file contains literal braces, for example a JSON example. Double them ({{ and }}) in the template, as in Module 03’s templates lesson.

Stuck? Hint

If files("crumbkit") / "prompts" cannot find your templates after installing, make sure the prompts folder is inside src/crumbkit/ (not the project root), so it is packaged with the code. uv run from the project folder uses the source directly, which hides this until someone installs the package elsewhere.

4. Route tickets into validated JSON

Why. Routing output is read by code, so a malformed answer must never become a wrong route. This milestone combines JSON mode (always valid JSON) with Pydantic validation (your exact fields and values), one corrective retry, and a safe fallback, as in Module 03’s structured outputs lesson and this module’s routing lesson.

src/crumbkit/tickets.py:

from typing import Literal
from pydantic import BaseModel, Field, ValidationError
from .client import LLM
from .templates import prompt

Category = Literal["delivery", "food_safety", "billing", "order_change", "wholesale", "general"]

class Ticket(BaseModel):
    categories: list[Category]
    primary: Category
    urgency: Literal["low", "normal", "high"]
    order_id: str | None = None
    summary: str = Field(max_length=200)
    safety_flag: bool
    needs_human: bool = False

FALLBACK = Ticket(categories=["general"], primary="general", urgency="normal",
                  summary="Could not triage automatically.", safety_flag=False, needs_human=True)

def route_ticket(llm: LLM, message: str) -> Ticket:
    system = prompt("route_system", schema=Ticket.model_json_schema())
    user = f"<message>\n{message}\n</message>"
    for _ in range(2):                                       # one try plus one corrective retry
        raw = llm.complete(system, user, json_mode=True)
        try:
            ticket = Ticket.model_validate_json(raw)
            if ticket.safety_flag and "food_safety" not in ticket.categories:
                ticket.categories.append("food_safety")      # protocol: the two signals always agree
            return ticket
        except ValidationError as err:
            user += f"\n\nYour previous JSON was invalid: {err.errors()[0]['msg']}. Return corrected JSON."
    return FALLBACK.model_copy(update={"summary": message[:180]})

Remember that the word “JSON” must appear somewhere in your messages for JSON mode; the system prompt template should say “Return JSON matching this schema”.

Working looks like: the walnut-brownie email from the routing lesson returns a Ticket with both food_safety and billing in categories and safety_flag=True. A deliberately broken fake response (in milestone 7) returns the fallback with needs_human=True.

Most likely bug: the schema in the prompt and the Pydantic model drift apart after someone edits one of them. Generating the prompt’s schema from the model, as above, makes that impossible.

Stuck? Hint

If validation keeps failing on categories, print the raw JSON. Models often return a single string ("categories": "billing") when a list is expected. Make the schema description explicit (“a JSON array, even for one category”), and include one example in the system prompt.

5. Generate, modify and explain code

Why. The code-generation lesson’s method, packaged: give real context, ask for a diff for changes, ask for assumptions, and never run anything inside the tool itself. crumbkit proposes; a person or a test decides.

src/crumbkit/code.py:

import re
from pydantic import BaseModel
from .client import LLM

class CodeAnswer(BaseModel):
    code: str
    assumptions: list[str]

def generate_code(llm: LLM, request: str, context: str, language: str = "python") -> CodeAnswer:
    system = (f"You write {language} for Crumb & Co.'s codebase. Use only names that appear in "
              "the context or the standard library. Return JSON with keys 'code' and 'assumptions'.")
    raw = llm.complete(system, f"<context>\n{context}\n</context>\n\nRequest: {request}", json_mode=True)
    return CodeAnswer.model_validate_json(raw)

def propose_diff(llm: LLM, path: str, source: str, request: str) -> str:
    out = llm.complete("You edit code. Return ONLY a unified diff. Change nothing else.",
                       f"File {path}:\n<file>\n{source}\n</file>\n\nChange: {request}")
    diff = re.sub(r"^```\w*\n|```$", "", out.strip(), flags=re.M)   # strip a code fence if present
    if not diff.startswith(("---", "diff")):
        raise ValueError("Model did not return a unified diff")
    return diff

def explain_code(llm: LLM, source: str) -> str:
    return llm.complete("Explain code for the developer who must maintain it.",
                        "1. What it does (3 sentences). 2. Step by step. "
                        f"3. Anything surprising or risky.\n<code>\n{source}\n</code>")

Working looks like: propose_diff on a small file returns text starting with --- that git apply --check accepts. generate_code returns code plus a non-empty assumptions list for any request that is even slightly ambiguous.

Most likely bug: diffs that do not apply because the model reproduced the surrounding lines slightly differently (changed whitespace, a reformatted line). Always run git apply --check before applying; if it fails, retry with “copy context lines exactly from the file”, or fall back to reviewing the whole file.

Stuck? Hint

If the JSON for generate_code fails to parse because the code contains quotes and newlines, check you are using JSON mode (the API escapes them properly). If you are not, the model may emit raw newlines inside strings, which is invalid JSON.

6. Wrap everything in a command-line tool

Why. Kavita and Rohan will not import Python modules. A command with clear subcommands and helpful --help text makes the toolkit usable by the whole team, and in scripts and scheduled jobs.

src/crumbkit/cli.py:

import argparse
import json
import sys
from .client import LLM, LLMError
from .config import load_settings
from . import text, tickets

def read_input(path: str) -> str:
    return sys.stdin.read() if path == "-" else open(path, encoding="utf-8").read()

def main(argv: list[str] | None = None, llm: LLM | None = None) -> int:
    parser = argparse.ArgumentParser(prog="crumbkit", description="Crumb's AI text utilities")
    sub = parser.add_subparsers(dest="command", required=True)

    s = sub.add_parser("summarise", help="Summarise one item per line")
    s.add_argument("file"); s.add_argument("--reader", default="Meera"); s.add_argument("--words", type=int, default=250)

    t = sub.add_parser("tone", help="Rewrite a reply in Crumb's voice")
    t.add_argument("file"); t.add_argument("--warmth", type=int, default=4)
    t.add_argument("--formality", type=int, default=2); t.add_argument("--firmness", type=int, default=3)

    r = sub.add_parser("route", help="Triage messages, one per line, to JSON lines")
    r.add_argument("file")

    args = parser.parse_args(argv)
    llm = llm or LLM(load_settings())                  # tests pass a fake here
    try:
        data = read_input(args.file)
        if args.command == "summarise":
            print(text.summarise(llm, [l for l in data.splitlines() if l.strip()], args.reader, args.words))
        elif args.command == "tone":
            print(text.adjust_tone(llm, data, args.formality, args.warmth, args.firmness))
        elif args.command == "route":
            for line in filter(str.strip, data.splitlines()):
                print(tickets.route_ticket(llm, line).model_dump_json())
    except (LLMError, ValueError) as err:
        print(f"crumbkit: {err}", file=sys.stderr)
        return 1
    print(f"[tokens in {llm.tokens_in}, out {llm.tokens_out}]", file=sys.stderr)
    return 0

Add translate, improve, code-gen, code-diff and explain subcommands in the same pattern. Note the token summary goes to standard error, so crumbkit route inbox.txt > triaged.jsonl produces a clean output file.

Working looks like:

uv run crumbkit --help
echo "We already told you, refunds take 7 days. Stop emailing." | uv run crumbkit tone - --warmth 5
uv run crumbkit route samples/inbox.txt > triaged.jsonl

The second command prints a kinder reply containing “7” and a token line. The third writes one JSON object per message.

Most likely bug: the entry point calls main() but the exit code is ignored, so failures look like success to scripts. In pyproject.toml the entry point calls main and uses its return value as the exit code, so make sure every path returns an integer.

Stuck? Hint

If Hindi or Marathi input prints as garbled characters, check that you open files with encoding="utf-8" (as above) and that your terminal uses UTF-8. On Windows, setting the environment variable PYTHONUTF8=1 helps.

7. Test it all without calling the API

Why. Tests that call the real API are slow, cost money and give different answers each run, so nobody runs them. The logic that breaks most often is yours: prompt building, validation, fallbacks, retries and argument parsing. A fake LLM tests all of it in milliseconds. A mock (or fake) is a stand-in object with the same interface as the real one, which returns answers you script.

tests/fakes.py:

class FakeLLM:
    def __init__(self, *responses: str):
        self.responses = list(responses)
        self.calls: list[tuple[str, str]] = []
        self.tokens_in = self.tokens_out = 0

    def complete(self, system: str, user: str, json_mode: bool = False) -> str:
        self.calls.append((system, user))       # so tests can inspect the prompts
        return self.responses.pop(0)

tests/test_crumbkit.py:

import pytest
from crumbkit import text, tickets
from crumbkit.client import with_retries, LLMError
from crumbkit.cli import main
from fakes import FakeLLM

GOOD = ('{"categories": ["billing"], "primary": "billing", "urgency": "normal", '
        '"order_id": "6620", "summary": "Double charge", "safety_flag": false}')

def test_tone_rejects_changed_numbers():
    llm = FakeLLM("Your refund will arrive within a week.")      # "7" became "a week"
    with pytest.raises(ValueError, match="7"):
        text.adjust_tone(llm, "Refunds take 7 days.", 2, 4, 3)

def test_route_retries_then_succeeds():
    llm = FakeLLM("not json at all", GOOD)
    t = tickets.route_ticket(llm, "I was charged twice for 6620")
    assert t.primary == "billing" and len(llm.calls) == 2
    assert "invalid" in llm.calls[1][1]                           # the retry explained the error

def test_route_falls_back_to_human():
    t = tickets.route_ticket(FakeLLM("{}", "{}"), "???")
    assert t.needs_human

def test_safety_flag_forces_category():
    raw = GOOD.replace('"safety_flag": false', '"safety_flag": true')
    assert "food_safety" in tickets.route_ticket(FakeLLM(raw), "rash after brownie").categories

def test_backoff_doubles_and_gives_up():
    delays = []
    def always_fails(): raise TimeoutError()
    with pytest.raises(LLMError):
        with_retries(always_fails, attempts=4, retry_on=(TimeoutError,), sleep=delays.append)
    assert len(delays) == 3 and delays[0] < delays[1] < delays[2]

def test_cli_route(tmp_path, capsys):
    f = tmp_path / "inbox.txt"; f.write_text("charged twice for 6620\n")
    assert main(["route", str(f)], llm=FakeLLM(GOOD)) == 0
    assert '"primary":"billing"' in capsys.readouterr().out

Add one live smoke test that is skipped unless a key is present, and run it before releases only:

import os
@pytest.mark.skipif(not os.getenv("OPENAI_API_KEY"), reason="needs a real key")
def test_live_tone_smoke():
    from crumbkit.client import LLM
    from crumbkit.config import load_settings
    assert "7" in text.adjust_tone(LLM(load_settings()), "Refunds take 7 days.", 2, 4, 3)

Working looks like: uv run pytest -q prints six passes and one skip in under a second, with no network access.

Most likely bug: ModuleNotFoundError: fakes. Add an empty tests/conftest.py (pytest then puts the tests folder on the import path), or move FakeLLM into conftest.py as a fixture.

Stuck? Hint

If test_backoff_doubles_and_gives_up is flaky, it is the jitter: with a random extra of up to 0.5 s, 1.0 + 0.5 could in theory exceed 2.0 + 0.0. It cannot here because doubling always outgrows the jitter, but if you change base_delay to something small, pass a fixed random seed or make jitter a parameter too.

Done means done

• uv run crumbkit --help `uv run crumbkit --help` lists every subcommand with a one-line description.
• No API key, password or customer data appears anywhere in the repository; .env is git-ignored and .env.example No API key, password or customer data appears anywhere in the repository; `.env` is git-ignored and `.env.example` exists.
• A missing key produces a one-line, friendly error, not a stack trace.
• Every API call has a timeout, and retries apply only to rate limits, timeouts, connection errors and server errors, with growing delays.
• Tone and translation outputs are checked in code (numbers, protected terms) and fail loudly when the check fails.
• Ticket routing always returns a valid Ticket, falling back to needs_human=True Ticket routing always returns a valid `Ticket`, falling back to `needs_human=True` rather than crashing.
• Code utilities never execute generated code; diffs pass git apply --check Code utilities never execute generated code; diffs pass `git apply --check` before anyone applies them.
• uv run pytest `uv run pytest` passes offline in under five seconds.
• A short README explains installation, configuration, each command and the cost per typical run.

Stretch goals

• Concurrency. Run the map step of summarise in parallel with asyncio and the SDK’s async client, with a cap on simultaneous requests. Module 05 covers batching.
• Cost reporting. Convert token counts to rupees using prices from config, and print them per command.
• Provider switch. Add an AnthropicLLM class with the same complete method, chosen by an environment variable. If your task modules only depend on complete, nothing else changes.
• Caching. Hash (model, system, user) and store results on disk, so rerunning a summary on the same reviews is free.
• Golden-file tests. Save the rendered prompts for fixed inputs and fail the test if they change unexpectedly, so prompt edits are always deliberate.

Common pitfalls

• Double retries. Leaving the SDK’s built-in retries on while adding your own multiplies attempts: four times four is sixteen calls during an outage.
• Retrying non-retryable errors. A 401 or 400 will never succeed on retry. Fail fast with the provider’s message.
• Prompts hidden in code. Long f-strings scattered across functions drift apart. Keep prompts in files and load them.
• Tests that hit the network. They will be skipped, then broken, then deleted. Inject the LLM and use a fake.
• Printing logs to standard output. It corrupts piped JSON output. Logs and token counts go to standard error.
• Hard-coded model names. Read the model from config so a model change is a one-line .env edit.

Show it off

• README. Lead with a three-line quick start (uv sync, copy .env.example, one example command), then a table of commands, then “How it handles failure” with the retry policy explained in two sentences. Engineers reading your repo will look for that section first.
• Demo. Record a 45-second terminal session: crumbkit tone on a blunt reply, crumbkit route on five messages including the walnut email, then pytest passing in under a second. A tool like asciinema or a screen recording converted to a GIF works well.
• LinkedIn post. Focus on the engineering, not the AI: “I turned five one-off LLM scripts into a tested Python package: one config, retries with exponential backoff and jitter, validated JSON for ticket routing, and code-level checks that tone rewrites never change numbers. The whole test suite runs offline in under a second using a fake LLM.” Link the repo and the demo.

Meera’s verdict, after Kavita had used crumbkit route on a full Monday inbox without asking Arjun for help once: “So it’s not tape any more?” “It’s one toolkit,” Arjun said, “with one key, one set of rules for when the AI service misbehaves, and tests that check our safety rules every time anyone changes anything.” Priya added one line to his review: “Next, put it to work on a real customer conversation.”

Go deeper

• OpenAI documentation, including error codes and rate limits: https://platform.openai.com/docs
• uv documentation: https://docs.astral.sh/uv/
• Pydantic documentation: https://docs.pydantic.dev
• Python tutorial (argparse and the standard library): https://docs.python.org/3/tutorial/