Errors, logging and debugging

At 2:07 a.m., Arjun’s phone buzzes. The script that imports orders from Crumb’s WhatsApp line has crashed. In the morning he finds the cause: one customer typed “two” in the quantity field. The script tried to turn “two” into a number, failed, and stopped. The 299 perfectly good orders queued behind it never reached the kitchen. The 6 a.m. bake was planned from yesterday’s numbers, and three stores ran out of croissants by nine.

Meera is calm but direct. “One customer made a typo. Why did that stop everyone else’s breakfast?”

It’s a fair question, and it has two halves. Why did one bad order stop the rest? And, as Arjun is about to discover, what’s the right way to stop it without making things worse?

The report Arjun didn’t read

Here’s a simplified version of the importer, saved as process_orders.py:

orders = [
    {"id": "W-301", "item": "Croissant", "qty": "4"},
    {"id": "W-302", "item": "Plum cake", "qty": "two"},
]


def parse_qty(order):
    return int(order["qty"])


def process(orders):
    for order in orders:
        qty = parse_qty(order)
        print(f"{order['id']}: {qty} x {order['item']}")


process(orders)

Running it prints:

W-301: 4 x Croissant
Traceback (most recent call last):
  File "/home/arjun/crumb/process_orders.py", line 17, in <module>
    process(orders)
  File "/home/arjun/crumb/process_orders.py", line 13, in process
    qty = parse_qty(order)
          ^^^^^^^^^^^^^^^^
  File "/home/arjun/crumb/process_orders.py", line 8, in parse_qty
    return int(order["qty"])
           ^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'two'

That block is a traceback (also called a stack trace): Python’s report of the path the problem travelled before the program stopped. Beginners see a wall of red and panic. Priya’s advice: read it from the bottom up.

1. The last line is the answer. ValueError is the kind of error. After the colon is the message: int() was given 'two', which isn’t a number in base 10.
2. The lines directly above it say where. Line 8, inside the function parse_qty, in the expression int(order["qty"]). The ^^^^ markers point at the exact piece of code.
3. Further up is how you got there. Line 17 called process, which on line 13 called parse_qty.

Nine times in ten, the bottom three lines tell you everything.

Arjun’s first fix: make the errors go away

Arjun has a plan. If an error stops the loop, catch every error so nothing can stop the loop. He wraps the body in try and except (Python’s way to say “try this, and if anything goes wrong, do that instead”), with pass, which means “do nothing”:

orders = [
    {"id": "W-301", "qty": "4"},
    {"id": "W-302", "qty": "two"},
    {"id": "W-303", "qty": "2"},
    {"id": "W-304", "qty": "1"},
]
saved = []


def save_order(order_id, qty):
    if order_id == "W-304":                      # pretend the database drops here
        raise ConnectionError("database connection lost")
    saved.append(order_id)


for order in orders:
    try:
        save_order(order["id"], int(order["qty"]))
    except:            # catch EVERYTHING
        pass           # ...and say nothing

print("Saved:", saved)

Saved: ['W-301', 'W-303']

No crash. He deploys it. The next night, 412 WhatsApp orders arrive and the script finishes without a single error. Only 371 orders reach the kitchen.

Three of the missing 41 had words for quantities. The other 38 failed because the database connection dropped for four minutes at 1:40 a.m. Nobody knew until customers started calling at 8. The log was empty, because there was no log. The script had been told to hide every problem, and it obeyed.

“You turned a loud failure into a silent one,” Priya says. “Loud is better. Silent failures are the expensive kind.”

The kitchen chain of shouting

Priya explains with the kitchen during the Diwali rush.

“A line cook discovers there’s no saffron. They can’t finish the order. What do they do?”

“Shout to the sous-chef.”

“And if the sous-chef can handle it, say ‘use the kesar syrup instead’, the kitchen keeps going. If not?”

“The sous-chef shouts to the head chef.”

“And if nobody can handle it, the line stops, and everyone knows why. Now: what’s the worst thing the line cook could do?”

Arjun thinks. “Quietly skip the saffron and send the dish out anyway.”

“That’s except: pass.”

That’s exactly how errors work in Python. When something goes wrong, Python creates an exception: an object that describes the problem. It’s raised where the problem happened, and travels up through every function that called that code, until some code catches (handles) it. If nothing catches it, the program stops and prints the traceback. The rule that follows:

Catch an exception only where you can do something useful about it. Everywhere else, let it travel.

Following “two” up the chain

Let’s trace both of last night’s failures through a fixed importer, one step at a time. First the code:

orders = [
    {"id": "W-301", "qty": "4"},
    {"id": "W-302", "qty": "two"},
    {"id": "W-303", "qty": "2"},
]
saved, needs_human = [], []


def parse_qty(order):
    return int(order["qty"])          # may raise ValueError


def save_order(order_id, qty):
    saved.append((order_id, qty))     # in real life, may raise ConnectionError


for order in orders:
    try:
        qty = parse_qty(order)
    except ValueError:
        needs_human.append(order["id"])   # we CAN handle this: set it aside
        continue                          # and move to the next order
    save_order(order["id"], qty)          # NOT wrapped: an outage should stop us

print("Saved:", saved)
print("Needs a human:", needs_human)

Saved: [('W-301', 4), ('W-303', 2)]
Needs a human: ['W-302']

Now follow order W-302:

And the database outage? save_order sits outside the try. A ConnectionError raised there travels up, finds no matching except, and stops the program with a traceback. That sounds bad, but it’s what we want. A monitoring alert fires at 1:41 a.m., the on-call engineer restarts the job at 1:50, and no orders are silently lost. The importer can’t fix a database outage, so it shouldn’t pretend to.

Two kinds of problems, two different correct responses:

Opening the hood: exceptions are objects in a family tree

Remember classes from the last lesson? Exceptions are classes too. ValueError is a class, and when int("two") fails, Python creates a ValueError object and raises it. Like any classes, exceptions use inheritance, and they form a family tree:

flowchart TD
    E["Exception"] --> V["ValueError"]
    E --> T["TypeError"]
    E --> L["LookupError"]
    L --> K["KeyError"]
    L --> I["IndexError"]
    E --> O["OSError"]
    O --> F["FileNotFoundError"]
    O --> C["ConnectionError"]
    E --> A["AttributeError"]

This matters because except matches the class and all its children. except LookupError catches both KeyError and IndexError. except Exception catches nearly everything, which is why it’s dangerous.

print(issubclass(KeyError, LookupError))
print(issubclass(ConnectionError, OSError))

try:
    {"croissant": 85}["donut"]
except LookupError as error:            # a parent class catches the child
    print(type(error).__name__, "caught as LookupError:", error)

True
True
KeyError caught as LookupError: 'donut'

except ... as error gives you the exception object, so you can read its message or its type. These are the ones you’ll meet most often:

The full shape of try

A try statement has up to four parts:

def read_rating(text):
    try:
        rating = int(text)
    except ValueError as error:
        print(f"Bad rating {text!r}: {error}")
        return None
    else:
        print("Parsed fine")              # runs only if NO exception happened
        return rating
    finally:
        print("Done checking", repr(text))  # ALWAYS runs, error or not


print(read_rating("4"))
print(read_rating("great"))

Parsed fine
Done checking '4'
4
Bad rating 'great': invalid literal for int() with base 10: 'great'
Done checking 'great'
None

• try: the code that might fail. Keep it as small as possible, so you know exactly which line you’re protecting.
• except: what to do for a specific kind of failure. You can have several, and catch more than one type with a tuple: except (KeyError, ValueError):.
• else: what to do only if nothing failed.
• finally: clean-up that must happen no matter what, like closing a connection. Notice it ran even though both paths had already hit return.

({text!r} in an f-string shows the value with its quotes, so stray spaces become visible.)

Raising your own errors

Sometimes your code is the one that notices something is wrong. raise signals it:

def check_quantity(qty):
    if qty <= 0:
        raise ValueError(f"Quantity must be at least 1, got {qty}")
    if qty > 50:
        raise ValueError(f"Quantity {qty} is above the 50-item limit; route to bulk orders")
    return qty


for q in [3, 0, 120]:
    try:
        print("OK:", check_quantity(q))
    except ValueError as error:
        print("Rejected:", error)

OK: 3
Rejected: Quantity must be at least 1, got 0
Rejected: Quantity 120 is above the 50-item limit; route to bulk orders

Raise early, at the edge of your program where data comes in. Bad data that slips past the door travels deep into your app, where it’s much harder to trace.

Custom exceptions: errors that speak your language

ValueError is generic. In a real app you want errors named after your domain: “this order doesn’t exist”, “the model’s reply couldn’t be parsed”. A custom exception is a class that inherits from Exception:

class CrumbError(Exception):
    """Base class for every error our app raises on purpose."""


class OrderNotFoundError(CrumbError):
    def __init__(self, order_id):
        self.order_id = order_id
        super().__init__(f"No order with ID {order_id!r}. Check the ID on the receipt.")


class LLMResponseError(CrumbError):
    """The model replied, but not in the shape we asked for."""


ORDERS = {"CR-1042": "out for delivery"}


def order_status(order_id):
    if order_id not in ORDERS:
        raise OrderNotFoundError(order_id)
    return ORDERS[order_id]


try:
    print(order_status("CR-1042"))
    print(order_status("CR-9999"))
except OrderNotFoundError as error:
    print("Not found:", error)
    print("The ID was:", error.order_id)

out for delivery
Not found: No order with ID 'CR-9999'. Check the ID on the receipt.
The ID was: CR-9999

Two wins. The error carries useful data (error.order_id). And the family tree lets callers choose how specific to be. A web endpoint can catch OrderNotFoundError and reply “404 Not Found”, while a top-level handler catches any CrumbError and shows a friendly message.

This pays off in AI apps. Models sometimes reply with broken JSON, refuse a request, or run out of room mid-answer. Wrapping those cases in exceptions like LLMResponseError makes them easy to catch, log and retry. When you catch a low-level error and raise a clearer one, use raise ... from error, which keeps the original attached:

import json


class LLMResponseError(Exception):
    pass


def parse_model_json(text):
    try:
        return json.loads(text)
    except json.JSONDecodeError as error:
        raise LLMResponseError(f"Model did not return valid JSON: {text[:40]!r}") from error


try:
    parse_model_json("Sure! Here is the JSON: {item: croissant")
except LLMResponseError as error:
    print(error)
    print("Caused by:", type(error.__cause__).__name__)

Model did not return valid JSON: 'Sure! Here is the JSON: {item: croissant'
Caused by: JSONDecodeError

The kitchen logbook

Even the fixed importer has a gap. When it sets an order aside at 2 a.m., where does that information go? print is like shouting across an empty kitchen: nobody hears it, and nothing is written down.

Logging is the kitchen logbook. Every entry has a time, a severity level and a source, and the log can be saved to a file or sent to a monitoring service that wakes someone up. Python’s built-in logging module has five levels:

import logging

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s: %(message)s")
logger = logging.getLogger("crumb.orders")

logger.debug("Raw payload: {...}")          # hidden: below the INFO level we chose
logger.info("Processing 412 WhatsApp orders")
logger.warning("Order W-302 has an unreadable quantity, set aside for a human")
try:
    int("two")
except ValueError:
    logger.exception("Could not parse quantity for W-302")   # ERROR + full traceback

INFO crumb.orders: Processing 412 WhatsApp orders
WARNING crumb.orders: Order W-302 has an unreadable quantity, set aside for a human
ERROR crumb.orders: Could not parse quantity for W-302
Traceback (most recent call last):
  File "log_demo.py", line 10, in <module>
    int("two")
ValueError: invalid literal for int() with base 10: 'two'

What to notice:

• Call basicConfig once, when your program starts. level hides anything less serious. Switch it to logging.DEBUG when you’re hunting a bug, without touching any other line.
• In real modules, write logger = logging.getLogger(__name__). __name__ is the module’s own name, so every line says which file it came from.
• logger.exception(...) inside an except block records the full traceback. That’s what you’ll want at 8 a.m. when you’re piecing together what happened at 2.
• Add %(asctime)s to the format to timestamp every line.

Debugging like a detective

When something goes wrong and the traceback alone doesn’t explain it, work like a detective, not a gambler:

1. Read the error, bottom up. Most of the time it tells you the answer.
2. Reproduce it with the smallest input you can. One order, not 412.
3. Look at the actual values. print(repr(x), type(x)) reveals hidden spaces (' 3 '), Nones, and strings pretending to be numbers.
4. Pause the program and poke around. Put breakpoint() on a line. When Python reaches it, it pauses and opens pdb, the built-in debugger (a tool that stops a running program so you can inspect it). Type a variable’s name to see its value, n to run the next line, c to continue and q to quit. Editors like VS Code offer the same with buttons: click beside a line number to set a breakpoint.
5. Change one thing at a time, then run again.

The usual suspects:

• AttributeError: 'NoneType' object has no attribute ... Something you expected to be an object is None. Usually a function that forgot to return, or a .get() that found nothing.
• KeyError: 'choices' on an API response. The response isn’t the shape you assumed, often because it’s an error response. Print the whole thing and look.
• ModuleNotFoundError: No module named 'anthropic' You installed the package into a different environment from the one running your code. The next lesson fixes this for good.
• TypeError: can only concatenate str (not "int") to str You’re joining text and a number. Use an f-string.
• IndentationError Mixed tabs and spaces, or a missing indent after a :.

Error messages for tired humans

An error message is a note to a future, tired person, possibly you at 2 a.m. Compare:

A good message says what went wrong, which input caused it, and, when you know, what to do next.

There are two audiences. Developers need detail: IDs, values, tracebacks, all in the logs. Customers need kindness and a next step: “Sorry, we couldn’t read the quantity. Could you send it as a number, like 2?” Never show a customer a traceback. It’s confusing for them, and it can reveal file paths and internal details to anyone curious.

Where it breaks

Error handling has its own failure modes, and experienced engineers watch for these:

• Retrying the un-retryable. Some errors are transient: a timeout, a dropped connection, an API saying “too many requests”. Trying again in a few seconds often works. Others are permanent: int("two") will fail on the hundredth attempt too. Retrying a ValueError in a loop burns time and money. In Module 19 you’ll build proper retries with growing waits between attempts, for transient errors only.
• Catching too early. If a low-level function can’t do anything useful about an error, catching it there only hides it. Let it travel to code that can decide: retry, set aside, or stop.
• Catching too much. except Exception around a large block catches bugs in your own code (a NameError from a typo) along with the problem you meant to handle. Keep try blocks small and except clauses narrow.
• Logging too little, or too much. No log means 8 a.m. guesswork. Logging every value at INFO means millions of lines nobody reads, and real costs on hosted logging services. Log events and decisions at INFO, details at DEBUG.
• The one place catching everything is right. At the very top of a long-running worker, a final except Exception: that logs the full traceback with logger.exception, alerts someone, and then stops or moves on cleanly is reasonable. The difference from except: pass is that nothing is hidden.

Arjun explains it to Meera

The rebuilt importer runs that night: 418 orders. Four go to a “needs a human” list, each with a clear note. At 3:12 a.m., the database blips; the job stops, an alert fires, the on-call engineer restarts it at 3:20, and every order is in by 3:25. The log shows exactly what happened, minute by minute.

“So what changed?” Meera asks.

“The first version stopped completely when one customer made a typo,” Arjun says. “My first fix was worse. It hid every problem, including a database outage, so we lost 41 orders and didn’t know. Now the script sorts problems into two kinds. The ones it can handle, like a word instead of a number, it sets aside for a person and carries on. The ones it can’t, like the database going down, it stops and shouts about, so someone fixes them in minutes, not hours. And it writes everything in a logbook.”

Meera reads the four notes on the “needs a human” list herself and calls the customers before breakfast. “I like the version that tells me what’s wrong,” she says.

Try it yourself

WhatsApp customers type quantities in all sorts of ways. Write:

1. A custom exception InvalidQuantityError that inherits from ValueError and stores the original text.
2. A function parse_quantity(text) that strips spaces, accepts digits (“3”) and the words “one” to “five” in any capitalisation, and raises InvalidQuantityError for anything else, or for numbers under 1.
3. A loop over ["2", " 3 ", "Four", "a dozen", "0"] that prints each result or each error message.

Expected output:

'2' -> 2
' 3 ' -> 3
'Four' -> 4
Could not read quantity 'a dozen'. Please send a number like 2.
Could not read quantity '0'. Please send a number like 2.

WORDS = {"one": 1, "two": 2, "three": 3, "four": 4, "five": 5}


class InvalidQuantityError(ValueError):
    def __init__(self, text):
        self.text = text
        super().__init__(f"Could not read quantity {text!r}. Please send a number like 2.")


def parse_quantity(text):
    cleaned = text.strip().lower()
    if cleaned.isdigit():
        qty = int(cleaned)
    elif cleaned in WORDS:
        qty = WORDS[cleaned]
    else:
        raise InvalidQuantityError(text)
    if qty < 1:
        raise InvalidQuantityError(text)
    return qty


for raw in ["2", " 3 ", "Four", "a dozen", "0"]:
    try:
        print(f"{raw!r} -> {parse_quantity(raw)}")
    except InvalidQuantityError as error:
        print(error)

'2' -> 2
' 3 ' -> 3
'Four' -> 4
Could not read quantity 'a dozen'. Please send a number like 2.
Could not read quantity '0'. Please send a number like 2.

Common mistakes

• except: pass. It hides every error, including the ones you most need to see. If you catch something, handle it or log it.
• Reading tracebacks from the top. The top is where the program started. The bottom is what broke.
• Huge try blocks. Twenty lines inside one try means you don’t know which line failed. Wrap only the line that can fail in the way you expect.
• print debugging left in production. Use logging with levels, so detail can be turned up or down without editing code.
• Vague messages. “Something went wrong” helps nobody. Say what, which input, and what to do next.

Recap

• An exception is an object that travels up through the functions that called it until something catches it; if nothing does, the program stops with a traceback, which you read from the bottom up.
• Catch only the specific exceptions you can do something useful about; let everything else surface loudly. Silent failures are the expensive ones.
• Exceptions are classes in a family tree, so catching a parent catches its children; except Exception catches almost everything, which is rarely what you want.
• raise signals problems early with clear messages; custom exceptions speak your app’s language, carry data, and raise ... from error keeps the original cause.
• logging records events with levels, sources and timestamps; logger.exception captures the full traceback; never log secrets or personal data.

Arjun’s importer is now solid, and Priya asks to review his newer project: the prototype support bot. She clones his repository onto her laptop, runs it, and gets ModuleNotFoundError: No module named 'anthropic'. Then she opens bot.py and sees his API key, typed straight into the code. “Before this goes anywhere near GitHub,” she says, “we need to talk about files, secrets and environments.”

Go deeper

• Python tutorial: errors and exceptions
• Logging HOWTO, the official beginner-friendly guide to logging.
• pdb, the Python debugger