Generating, modifying and explaining code

Meera’s question was specific: “Baner had 211 late deliveries in week 46. I want them broken down by hour of day and by delivery partner. I’m meeting the partners on Thursday.”

Arjun knew the answer was in the order database. He also knew the database had forty-one tables, built up over seven years by three different developers, one of whom had named a column dlvry_ts2. Writing the query by hand would take him most of an afternoon. He had two other jobs waiting: Rohan, the logistics lead, wanted a change to the function that offers delivery slots, and nobody on the team understood the 2019 loyalty-points script that a freelancer had written and then vanished.

“Three code jobs,” Arjun said. “The model writes code all day for millions of people. This should be the easy part of the module.”

Priya smiled the smile he was learning to be wary of. “It can be. Let’s see what it writes when it’s never seen our database.”

Arjun asks for a query

His first prompt:

Write a SQL query to find late deliveries from the Baner store in
week 46, broken down by hour of day and delivery partner.

The model replied instantly with clean, well-commented SQL:

SELECT dp.partner_name,
       EXTRACT(HOUR FROM d.delivered_at) AS hour,
       COUNT(*) AS late_count
FROM deliveries d
JOIN delivery_partners dp ON d.partner_id = dp.id
WHERE d.store_name = 'Baner'
  AND d.is_late = TRUE
  AND d.week_number = 46
GROUP BY dp.partner_name, hour;

It looked like exactly what a good developer would write. Arjun ran it. Error: no such column: d.store_name. Then is_late. Then week_number. None of those columns existed. Crumb’s database had no is_late flag; lateness had to be worked out from timestamps.

The model had not looked anything up. It had written the most typical query for a typical delivery database. Every table and column name was a plausible guess.

So Arjun went to the other extreme. He dumped the entire schema, all forty-one tables, into the prompt: about 9,000 tokens. This time the query ran. It returned 311 late deliveries. Meera’s figure, from the delivery partner’s own report, was 211.

The contractor who has never seen your kitchen

Priya leaned back. “Suppose we hire a very good contract developer for one day. Never seen our code. What do you give them on their first morning?”

“Access to the repo?”

“All forty-one tables and seven years of code, and ‘good luck’?”

Arjun laughed. “No. I’d show them the three tables that matter. Tell them what ‘late’ means for us. Mention the weird stuff, like timestamps.”

“What weird stuff with timestamps?”

He stopped. “They’re stored in UTC. Pune is five and a half hours ahead.”

“Would the contractor know that?”

“Not unless I told them.”

“And at the end of the day, would you deploy whatever they wrote without looking?”

“I’d check it. Run it on something where I already know the answer.”

“So: the right context, the house rules, and a way to check. That’s the whole method, for queries, for changes and for explanations.”

The model is that contractor: very fast, very well-read, and completely new to your codebase. Generating code is a kind of writing it does extremely well, because it was trained on enormous amounts of public code, and code is highly structured. But it fills every gap in its knowledge with the most typical guess, exactly as with Mrs. Deshpande’s refund in the first lesson of Module 03. In code, those guesses are called hallucinated APIs, tables and columns: names that look right and do not exist, or exist and mean something else.

Following one query from question to verified answer

Priya and Arjun rebuilt the request properly, and traced it all the way to a number they could defend.

Step 1: pick the relevant context, not all of it. Three tables mattered. Arjun pulled their definitions (the DDL, data definition language, the CREATE TABLE statements) from the database itself, about 600 tokens instead of 9,000:

CREATE TABLE orders (id INTEGER PRIMARY KEY, store TEXT, status TEXT,
  created_at TEXT, promised_slot_end TEXT);          -- times stored in UTC
CREATE TABLE deliveries (id INTEGER PRIMARY KEY, order_id INTEGER,
  partner_id INTEGER, delivered_at TEXT);             -- UTC
CREATE TABLE delivery_partners (id INTEGER PRIMARY KEY, name TEXT);

Step 2: add the house rules the schema cannot express.

Database: SQLite (a nightly copy of the order database).
Definitions at Crumb:
- A delivery is late if deliveries.delivered_at is more than 30 minutes after
  orders.promised_slot_end.
- Only count orders with status = 'delivered'. Cancelled orders keep a deliveries row.
- All timestamps are stored in UTC. Report hours in Indian time (UTC + 5:30).
- Week 46 of 2026 is Monday 9 November to Sunday 15 November, Indian time.
Return the SQL, then list any assumptions you made.

Step 3: the model’s query.

SELECT dp.name AS partner,
       CAST(strftime('%H', d.delivered_at, '+5 hours', '+30 minutes') AS INTEGER) AS hour_ist,
       COUNT(*) AS late_deliveries
FROM orders o
JOIN deliveries d         ON d.order_id = o.id
JOIN delivery_partners dp ON dp.id = d.partner_id
WHERE o.store = 'Baner'
  AND o.status = 'delivered'
  AND d.delivered_at > datetime(o.promised_slot_end, '+30 minutes')
  AND datetime(o.promised_slot_end, '+5 hours', '+30 minutes') >= '2026-11-09'
  AND datetime(o.promised_slot_end, '+5 hours', '+30 minutes') <  '2026-11-16'
GROUP BY partner, hour_ist
ORDER BY late_deliveries DESC;

With its stated assumption: “I used the promised slot (converted to Indian time) to decide which week an order belongs to.” That is a real choice someone should confirm, and the model surfaced it instead of hiding it.

Step 4: check against a number you already trust. Before reading the breakdown, Arjun summed the late_deliveries column: 211. That matched the partner’s report exactly. A second check: total Baner deliveries that week, with the lateness condition removed, came to 1,480, which matched the store’s own dashboard. Two independent totals agreeing is strong evidence the joins and filters are right.

Step 5: read the answer. The top rows: QuickDrop at 7 p.m. (48 late), QuickDrop at 8 p.m. (41), RideKart at 7 p.m. (29). The evening rush, with one partner carrying most of the problem. That was Meera’s Thursday meeting, in one table.

The cost. About 1,100 tokens in and 400 out. On a mid-size model at illustrative prices of $3permillioninputtokensand$15 per million output, roughly $0.009, under one rupee. Arjun’s time: twenty minutes, mostly on checking, against an estimated afternoon.

Opening the hood: three jobs, three shapes of request

Code tasks come in three families, and each has its own best shape.

Generating new code from a description. The request needs: the language and version, the relevant existing code or schema, conventions (“we use httpx, not requests”), and the definition of done, ideally as tests or an example input and output. The model’s weak spots are names it has not been shown and library versions newer than its training data. Ask it to list assumptions, as above.

Modifying existing code. Here the danger is the model quietly rewriting things you did not ask it to touch: renaming variables, “tidying” logic, dropping a comment that explained a hack. Ask for the change as a diff: a listing of only the lines removed and added, marked with - and +. A diff is small to review and easy to apply with standard tools, and anything outside it is guaranteed unchanged.

Explaining code someone else wrote. Models are excellent at this, and it is the safest of the three, since nothing runs. The risk is a confident, plausible explanation that is wrong in one important detail. Ask for a summary, then a step-by-step walkthrough, then “anything surprising or risky”, and verify the claims that matter by running the code.

Across all three, one principle holds: the model proposes, a check disposes. For SQL, the check is a known total. For a function, it is a test. For an explanation, it is running the code on an input and seeing whether it does what the explanation says.

Building the code helpers, step by step

Step 1: gather only the relevant schema, from the database itself. Install with uv add openai pydantic python-dotenv. Crumb’s analysis copy is a SQLite file, so the standard library’s sqlite3 is enough.

import sqlite3

def schema_for(db_path: str, tables: list[str]) -> str:
    """Return the CREATE TABLE statements for only these tables."""
    con = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True)    # read-only connection
    rows = con.execute(
        f"SELECT sql FROM sqlite_master WHERE type='table' AND name IN ({','.join('?' * len(tables))})",
        tables).fetchall()
    con.close()
    return "\n".join(r[0] for r in rows)

print(schema_for("crumb_orders.db", ["orders", "deliveries", "delivery_partners"]))

Pulling the DDL from the database, rather than from memory or an old document, means the model sees the real column names.

Step 2: generate SQL as structured output, with assumptions.

import os
from dotenv import load_dotenv
from openai import OpenAI
from pydantic import BaseModel

load_dotenv()
client = OpenAI()
MODEL = os.getenv("OPENAI_MODEL", "gpt-5-mini")  # model names change; see the provider's models page

class SqlAnswer(BaseModel):
    sql: str
    assumptions: list[str]

HOUSE_RULES = open("prompts/crumb_sql_rules.txt").read()   # definitions of 'late', UTC note, etc.

def write_sql(question: str, schema: str) -> SqlAnswer:
    resp = client.chat.completions.create(
        model=MODEL,
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": "You write SQLite SELECT queries. Use ONLY the tables "
             "and columns in the schema. Return JSON with keys 'sql' and 'assumptions'.\n\n"
             f"{HOUSE_RULES}\n\n<schema>\n{schema}\n</schema>"},
            {"role": "user", "content": question},
        ])
    return SqlAnswer.model_validate_json(resp.choices[0].message.content)

Step 3: run it safely. The real protection is the database connection, not the prompt:

def run_readonly(db_path: str, sql: str, max_rows: int = 500) -> list[tuple]:
    if not sql.lstrip().lower().startswith(("select", "with")):
        raise ValueError("Only SELECT queries are allowed")
    con = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True)    # cannot write, whatever the SQL says
    try:
        return con.execute(sql).fetchmany(max_rows)
    finally:
        con.close()

answer = write_sql("Late Baner deliveries in week 46 of 2026 by partner and hour (Indian time).",
                   schema_for("crumb_orders.db", ["orders", "deliveries", "delivery_partners"]))
print(answer.assumptions)
rows = run_readonly("crumb_orders.db", answer.sql)
print(sum(r[-1] for r in rows), "late deliveries")    # compare with a figure you trust

The startswith check is a convenience, easy to fool. The read-only connection is the guarantee: even a DELETE hidden inside a clever query cannot change anything. On a production database server, the equivalent is a database user that only has read permissions.

Step 4: modify code with a diff, then let the tests decide.

def propose_change(file_path: str, request: str) -> str:
    source = open(file_path).read()
    prompt = (f"Here is {file_path}:\n<file>\n{source}\n</file>\n\n"
              f"Change request: {request}\n\n"
              "Return ONLY a unified diff against this file. Change nothing else.")
    resp = client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": prompt}])
    return resp.choices[0].message.content

diff = propose_change("crumb/slots.py",
    "No delivery slots after 21:00 IST. Cakes with a custom message need 2 hours' lead time, not 1.")
open("change.diff", "w").write(diff)

Then, in the terminal: git apply --check change.diff (does it match the real file?), git apply change.diff, and uv run pytest. If Rohan’s two requirements are not covered by existing tests, ask the model to write those tests first, read them yourself, and only then ask for the change. Tests you have read are a specification the model cannot quietly reinterpret.

Step 5: explain legacy code, then verify the explanation.

def explain(source: str) -> str:
    resp = client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content":
        "Explain this code for a developer who must maintain it.\n"
        "1. What it does, in three sentences.\n2. Step by step, how it works.\n"
        "3. Anything surprising, risky or likely to be a bug.\n"
        f"<code>\n{source}\n</code>"}])
    return resp.choices[0].message.content

print(explain(open("legacy/loyalty_points_2019.py").read()))

The explanation of the freelancer’s script said, under “surprising”: “Points are calculated on the order total before discounts, so customers earn points on money they did not pay.” That was a real business question, which Meera answered (“that’s a bug, but we’ll honour existing points”). Arjun confirmed the claim the cheap way: one test order of ₹1,000 with a ₹200 discount earned 100 points, not 80.

Where it breaks

The tools saved Arjun hours in the first week. They also produced the bugs below, each of which taught a rule.

The time-zone trap, in a different place. A follow-up query for “orders by hour” came back with the busiest hour at 1 p.m. Crumb is busiest in the evening. The house rules said to convert to Indian time, but the new question used a different column, created_at, and the model converted one timestamp and not the other. The data was right, the hours were 5.5 hours off. Sanity-check results against what you know about the business: if a bakery’s peak is at lunchtime, look again.

The confident library call. Asked to add retries to a script, the model used a function from a version of a library newer than the one in Crumb’s pyproject.toml. It looked fine and failed on import. Tell the model your versions, and let a quick run or test catch the rest.

SQL built from user text. An early idea was to let store managers type questions and run the generated SQL directly. Even with a read-only connection, someone could ask for “all customers’ phone numbers”. The model cannot know what each person is allowed to see; access rules belong in your application, not in the prompt.

The change that passed the tests. A diff for a discount function passed every test, and was still wrong for orders over ₹10,000, a case no test covered. Tests only check what they check. For code that touches money, a person reads the diff too.

Try it yourself

Create a tiny practice database and ask the model a question with a known answer.

1. With Python’s sqlite3, create the three tables from step 1 and insert ten orders by hand: six delivered on time, three delivered 45 minutes after the slot, and one cancelled order with a late deliveries row.
2. Ask write_sql for “the number of late deliveries”, once with no house rules, once with the rules from step 2.
3. Which version gives 3?

Show the answer

Without house rules, you will often get 4 (the cancelled order is counted) or a different number entirely, if the model invents its own definition of late or looks for an is_late column that does not exist. With the rules, you should get exactly 3: the rules define lateness as more than 30 minutes past promised_slot_end and exclude cancelled orders.

The exercise shows the full method in miniature: a small, known dataset is a test for generated SQL. Whenever you can, check generated code against data where you already know the answer.

Common mistakes

• No context. Without your schema or files, the model invents plausible names. Give it the relevant pieces, pulled from the real source.
• All the context. Forty-one tables of DDL bury the three that matter. Select what is relevant.
• Schema without meaning. Write down the business definitions (late, active, revenue) the schema cannot express.
• Whole-file rewrites. Ask for diffs, and apply them with tools that check they match.
• Running unchecked code. Use read-only connections, tests and known totals. The model proposes; a check decides.
• Pasting secrets and customer data. Strip them before they reach any prompt.

Recap

• Generating, modifying and explaining code are three tasks with different shapes: context plus definition of done, diffs, and explanation plus verification.
• The model is a brilliant contractor on day one: it fills gaps with typical guesses, so give it the real schema, the house rules and your versions.
• Check every result against something you trust: a known total, a test, a run on a sample input.
• Put safety in the environment, not the prompt: read-only connections, limited permissions, human review for code that touches money.
• Ask the model to list its assumptions; the assumptions are often where the real questions are.

On Thursday, Meera walked into the delivery-partner meeting with a one-page table: late deliveries by hour and partner, with QuickDrop’s 7 p.m. and 8 p.m. slots circled. QuickDrop agreed to add two riders in Baner for the evening rush.

Afterwards she asked Arjun how he had got the numbers so fast. “The AI wrote the query,” he said, “but only after I gave it the three tables that mattered and told it what ‘late’ means for us, and that our clocks are in UTC. Then I checked its answer against the partner’s own total. It matched to the delivery: 211. If it hadn’t, I wouldn’t have given you the table.”

“So it’s a fast new hire you double-check.”

“A very fast one. With no access to change anything.”

Meera nodded, then mentioned what had been on her mind. “Speaking of new hires: we’re taking on eight seasonal staff for Christmas. Kavita can’t train all of them on handling angry customers. Could your AI pretend to be an angry customer, so they can practise?”

Go deeper

• OpenAI documentation: https://platform.openai.com/docs
• Anthropic documentation: https://docs.anthropic.com
• Python sqlite3 and the rest of the standard library, via the tutorial: https://docs.python.org/3/tutorial/
• OWASP Top 10 for LLM applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/