Error handling in AI applications

It was the Saturday before Diwali, the busiest day of Crumb’s year. By 11 a.m. the website had taken 1,300 hamper orders, and the support chat was answering a question every four seconds: “Is the kaju katli box eggless?”, “Can I move my delivery to tomorrow?”, “Do you deliver to Hinjewadi?”

At 11:40 a.m. Meera forwarded Arjun a screenshot from a customer. Where the chat assistant’s answer should have been, there was a wall of red text: anthropic.RateLimitError: Error code: 429 - {'type': 'error', 'error': {'type': 'rate_limit_error', ... followed by forty lines of Python traceback (the list of function calls Python prints when a program crashes).

Her message said: “Is this what customers see when something breaks? It looks like we’ve been hacked. Also, how many people saw this?”

Arjun checked. Between 11:30 and 11:50, 63 customers had seen some version of that red wall. Eleven of them had abandoned their carts, worth about ₹27,000 in hampers. The AI had worked perfectly for weeks. It had never been tested on the one day that mattered most.

Priya pulled a chair over. “Every call to a model is a call to someone else’s computer, over the internet, at their busiest hour, which is also your busiest hour. Things will fail. The question isn’t whether. It’s whether your customer notices.”

Arjun’s first attempt: no plan at all

The chat widget called one function. Here it is, exactly as it was that morning:

import anthropic
from dotenv import load_dotenv

load_dotenv()
client = anthropic.Anthropic()

def answer(question: str) -> str:
    response = client.messages.create(
        model="claude-opus-5-5",
        max_tokens=2000,
        system="You are Crumb & Co.'s friendly support assistant.",
        messages=[{"role": "user", "content": question}],
    )
    return response.content[0].text

Nothing in it is wrong for a demo. Everything in it is wrong for Diwali. When something goes wrong in Python, the code raises an exception: it creates an object describing the problem and stops normal execution until some code catches it with try / except. Nothing here catches anything. So the exception travelled up to the web framework, which helpfully printed it into the page.

Arjun’s first fix, typed in a panic, was this:

def answer(question: str) -> str:
    try:
        response = client.messages.create(...)
        return response.content[0].text
    except Exception:
        return "Sorry, something went wrong."

The red walls disappeared. Priya looked at it for a long moment. “Now tell me: in the last hour, how many of those failures were because the provider was busy, and how many were because of a bug in your code?”

“I… can’t tell anymore.”

“Exactly. You swapped a loud failure for a silent one. If your API key expires tonight, every customer gets ‘something went wrong’ and nobody knows why until Monday. And look at line two: response.content[0].text. What if the first block isn’t text?”

That last point needs a little background. A Claude response’s content is a list of blocks. On Claude Opus 5.5 the model thinks before answering, and that thinking can arrive as a block before the text. Index [0] is not guaranteed to be text, so [0].text can raise an AttributeError, a bug in your code, which except Exception would quietly hide among the rate limits.

Four reasons a dish never reaches the table

“Let’s think about the restaurant upstairs,” Priya said. “A customer orders. The dish doesn’t arrive. Give me reasons.”

Arjun thought. “The waiter wrote the wrong table number on the slip.”

“Will sending the same slip again help?”

“No. Someone has to fix the slip.”

“Next.”

“The kitchen’s slammed. Twenty tickets on the rail.”

“Will sending it again help?”

“Eventually. If you wait. Sending it ten times right now makes it worse.”

“Next.”

“The waiter tripped on the stairs and dropped it.”

“Send again?”

“Yes, straight away.”

“One more. The sneaky one.”

Arjun thought harder. “The dish arrives… but it’s the wrong dish? Or half a dish?”

“That’s the one nobody plans for,” Priya said. “Nothing ‘failed’. The kitchen says ‘order complete’. But you can’t serve it.”

She wrote four headings on the whiteboard, and those four families became the backbone of every AI app Arjun built afterwards.

The analogy breaks in one useful place: a real kitchen rarely tells you why the dish didn’t come. An API tells you a lot, through HTTP status codes, the three-digit numbers every web response carries. 200 means OK. 4xx codes mean “your request has a problem”. 5xx codes mean “the server had a problem”. And the SDK turns each one into its own exception class, so you never have to guess.

The four families, precisely

flowchart TD
    A["Call the model"] --> B{"What happened?"}
    B -->|"400, 401, 403, 404"| C["My bug: do not retry, alert a developer"]
    B -->|"429, 500, 529"| D["Their busy: wait, then retry"]
    B -->|"Timeout, connection error"| E["The network: retry"]
    B -->|"200 OK but unusable"| F["Success that isn't: check, then recover"]

Family	Status / exception	Retry?	What the customer sees
My bug	400 BadRequestError, 401 AuthenticationError, 403 PermissionDeniedError, 404 NotFoundError	Never: it fails the same way every time	A polite fallback. A developer gets paged
Their busy	429 RateLimitError, 500 InternalServerError, 529 overloaded	Yes, after a wait	Usually nothing: a retry succeeds
The network	APITimeoutError, APIConnectionError	Yes	Usually nothing
Success that isn’t	HTTP 200, but stop_reason is max_tokens or refusal, or the text is empty or invalid	Sometimes	A fallback answer or a human handover

Some words in that table, defined. A rate limit is a cap your provider puts on how many requests and tokens you can use per minute; you get 429 when you exceed it (the next lesson but one is all about them). 529 means the provider as a whole is overloaded, not just your account. A timeout is the longest you are willing to wait for one attempt before giving up. The stop reason is the API’s note on why the model stopped writing: end_turn (finished normally), max_tokens (hit your output limit mid-sentence), refusal (declined the request), and a few others you will meet in Module 06.

The fourth family is the one that is special to AI. A normal API either gives you the data or an error. A model can give you HTTP 200 with half an answer, or a polite refusal, or JSON your validator rejects (the previous lesson). Your code only catches these if it looks.

Show the answer

(a) My bug: a 404 NotFoundError, because that model name does not exist. Retrying will fail forever; fix the config. (b) Success that isn’t: HTTP 200 with stop_reason == "max_tokens". © Their busy: wait and retry. (d) The network: APIConnectionError, retry. Notice that (a) is the one that looks harmless and is actually the most serious: it breaks every request until a human fixes it.

Following one question through a bad minute

Theory is easier to trust once you watch it run. Here is one real question from 11:41 a.m., traced through the system after the fixes in this lesson, with the timings from the logs.

The customer asked: “Can I change my hamper delivery from Sunday to Monday? Order CR-4471.”

Time	What happened
0.0 s	messages.create sends the request: about 120 input tokens
0.3 s	Response: 429 rate limit, with header retry-after: 2
0.3 s	The SDK sees a retryable status and waits about 2 seconds, as the header asked
2.4 s	Attempt 2 sent
2.6 s	Response: 529 overloaded
2.6 s	The SDK waits again, a little longer this time
4.1 s	Attempt 3 sent
4.3 s	Response: 429 again. The SDK has used its 2 retries and raises RateLimitError
4.3 s	Arjun’s except anthropic.RateLimitError branch runs: logs a warning, returns the fallback
4.3 s	The customer sees: “Sorry, our assistant is busy right now. A team member will reply on WhatsApp within 15 minutes.”
4.3 s	The question is saved to a follow-up queue with the order number

Three things happened that Arjun did not write code for. The SDK retried twice, it waited between tries, and it read the retry-after header. The Anthropic and OpenAI SDKs both do this by default for 408, 409, 429 and 5xx responses and for network errors. Arjun’s except blocks only run after those built-in retries are used up. That is good news: most “their busy” and “network” blips never reach your code at all. Of the 63 failures that morning, the logs showed 51 would have been rescued by the SDK’s retries if he had not set max_retries=0 while debugging the week before.

And one thing happened that the SDK could never do: decide what Crumb says to a customer when the AI is unavailable. That is the part you own.

Building the handler, one family at a time

Here is the function Arjun shipped that afternoon, built up family by family.

Step 1: configure the client. Two settings matter on every production client:

import logging
import anthropic
from dotenv import load_dotenv

load_dotenv()
log = logging.getLogger("crumb.support")

# timeout: give up on a single attempt after 30 seconds (the default is 10 minutes!).
# max_retries: how many times the SDK retries 429s, 5xx and network errors. Default is 2.
client = anthropic.Anthropic(timeout=30.0, max_retries=2)

FALLBACK = "Sorry, our assistant is busy right now. A team member will reply on WhatsApp within 15 minutes."

The default timeout of ten minutes makes sense for long batch jobs. For a customer staring at a chat bubble it is absurd. With retries, the worst case wait is roughly timeout × (max_retries + 1), so 30 seconds and 2 retries means up to about 90 seconds. For live chat you might choose timeout=15.0, max_retries=1.

Step 2: catch the families, most specific first. The SDK’s exceptions form a family tree. RateLimitError is a kind of APIStatusError. APITimeoutError is a kind of APIConnectionError. Python checks except clauses top to bottom and uses the first match, so the specific ones must come first.

def answer(question: str) -> str:
    try:
        response = client.messages.create(
            model="claude-opus-5-5",
            max_tokens=2000,
            system="You are Crumb & Co.'s friendly support assistant.",
            messages=[{"role": "user", "content": question}],
        )
    # Family 1: my bug. Retrying will never help. Log loudly.
    except (anthropic.AuthenticationError, anthropic.PermissionDeniedError,
            anthropic.NotFoundError, anthropic.BadRequestError) as err:
        log.error("Request or configuration bug (%s): %s", type(err).__name__, err)
        return FALLBACK
    # Family 2: their busy, still failing after the SDK's own retries.
    except anthropic.RateLimitError as err:
        log.warning("Rate limited after retries: %s", err)
        return FALLBACK
    except anthropic.APIStatusError as err:
        log.warning("Provider error %s: %s", err.status_code, err)
        return FALLBACK
    # Family 3: the network. Timeout BEFORE connection error: a timeout is a kind of connection error.
    except anthropic.APITimeoutError:
        log.warning("Model call timed out")
        return FALLBACK
    except anthropic.APIConnectionError as err:
        log.warning("Network problem: %s", err)
        return FALLBACK

Notice the difference from except Exception. Every branch still returns the fallback to the customer, so the customer experience is identical. But the logs now say which family failed, at error level for bugs and warning level for transient trouble. And a genuine bug in Arjun’s own code, say a typo that raises NameError, is no longer swallowed: it crashes in testing, where he will see it.

Step 3: check the success. The try block only covers exceptions. Family 4 arrives as a normal response, so it needs its own checks after the try:

    # Family 4: HTTP 200, but is the answer usable?
    if response.stop_reason == "max_tokens":
        log.warning("Reply cut off at max_tokens")
        return FALLBACK
    if response.stop_reason == "refusal":
        log.info("Model declined the request")
        return "I can't help with that one here, but a team member can. I've passed it on."
    text = next((b.text for b in response.content if b.type == "text"), "").strip()
    if not text:
        log.warning("Empty reply")
        return FALLBACK
    return text

next((b.text for b in response.content if b.type == "text"), "") walks the content blocks, picks the first text block, and returns "" if there is none. That fixes the [0].text bug for good.

Show the answer

Every timeout now lands in the connection-error branch, because APITimeoutError is a subclass of APIConnectionError and Python uses the first matching clause. The timeout branch can never run again. The customer still gets the fallback, but the “Model call timed out” log line vanishes, so the team loses the signal that says “our timeout is too short” or “the provider is slow today”. Bugs like this are invisible in normal testing, which is exactly why the order rule is worth memorising: specific first, general last.

Graceful degradation: what does the shop do when the till breaks?

A friendly fallback message is the minimum. Graceful degradation means the app keeps doing something useful when one part fails. Priya asked Arjun Meera’s kind of question: “What would a good shop assistant do if the card machine broke?”

“Apologise. Take cash. Or write down the order and call them back.”

That maps directly onto options for an AI feature:

• Apologise and promise a follow-up. The fallback message, plus saving the question to a queue so a human replies. This is what saved most of the Diwali carts after the fix.
• Take cash instead. A pre-written answer for the top ten questions (“Delivery slots are 10 a.m. to 9 p.m.”, “All hampers are eggless”), matched by keywords, used when the model is unavailable. Not clever, but right 80% of the time for the most common questions.
• Try another machine. Switch to a backup model or a second provider. You will compare the OpenAI and Claude SDKs in Module 06 and build a full fallback gateway in Module 19.
• Do it later. For non-urgent work like nightly review tagging, save failed items and reprocess them. The next lesson shows how.

Which one you pick depends on the cost of a wrong answer. A canned “slots are 10 to 9” is harmless. A canned answer to “does this contain nuts?” is dangerous, because the right answer depends on the product. For allergy questions, the only acceptable fallback is a human.

Where it still breaks

Even with all four families handled, three things caught Arjun out in the following weeks.

Silent fallbacks hide outages. A fallback that hides an error from customers must not hide it from you. Arjun added one counter: fallbacks per hour, by family. On a normal day it showed 2 or 3 “their busy” fallbacks. One Tuesday it showed 400 “my bug” fallbacks in an hour: a teammate had rotated the API key and forgotten the production server. The counter caught it in 20 minutes instead of a weekend. Module 22 builds proper dashboards; even a log line you can count is enough to start.

Retrying can double-act. Retrying a question is harmless. Retrying something with a side effect, like “send the customer an SMS”, can send two SMS messages if the first one actually succeeded but the response was lost on the network. Module 19 covers idempotency keys, which make repeats safe.

The timeout is a product decision. Too short and slow-but-fine answers become fallbacks. Too long and customers stare at a spinner. Arjun’s first setting of 10 seconds caused fallbacks on long catering questions; 25 seconds plus streaming (coming in this module’s FastAPI lessons) turned out to be the right balance for Crumb.

A last misconception to drop: “Errors are rare, so I’ll add handling later.” A 1% failure rate sounds tiny. On Diwali Saturday Crumb’s chat handled about 5,000 questions. One percent is fifty customers, and the failures cluster at the busiest moments, exactly when each one costs the most.

Try it yourself

Break your own code on purpose, one failure at a time. Add logging.basicConfig(level=logging.INFO) at the top so log lines print, and check that answer() returns the fallback and logs the right family each time:

1. Wrong model name. Use model="claude-does-not-exist". Expect a NotFoundError logged at error level.
2. Bad key. Create the client with anthropic.Anthropic(api_key="sk-ant-wrong"). Expect AuthenticationError.
3. Impatient timeout. Use anthropic.Anthropic(timeout=0.01, max_retries=0). Expect “Model call timed out”.
4. Tiny output limit. Set max_tokens=20 and ask for a long story about a plum cake. Expect “Reply cut off at max_tokens”.

Show a hint for test 4

With a tiny max_tokens, the call succeeds with status 200, so none of the except clauses run. Only the stop_reason == "max_tokens" check catches it. If you comment that check out, you will see the half-sentence returned to the “customer”. That is the fourth family in action.

Common mistakes

• One giant except Exception. It hides the difference between “fix the code” and “try again later”, and swallows your own bugs. Catch specific SDK classes.
• Retrying 4xx errors. A bad request or a bad key fails the same way every time. Retrying wastes time and money.
• Reading response.content[0].text. The first block is not always text. Search for the first text block.
• Forgetting the 200-but-wrong family. Check stop_reason and the content even when there was no exception.
• Leaving the default 10-minute timeout on a live chat. Choose a timeout that matches how long a human will wait.
• Fallbacks nobody counts. Log the family of every fallback and watch the numbers.

Explaining it to Meera

That evening Arjun walked Meera through the change.

“Before, when the AI service hiccupped, the website printed the AI’s error message, the technical gibberish, straight to customers. Now every problem falls into one of four boxes. If it’s our mistake, like a wrong setting, customers get a polite message and my phone buzzes. If the AI company is busy, the system waits a second or two and tries again by itself; most customers never notice. If the internet blips, same thing. And if the AI answers but the answer is cut off or empty, we don’t show it.”

“And the people who still get the ‘sorry’ message?”

“Their question goes into a list for the WhatsApp team, with the order number. Nobody gets dropped.”

“How many today?”

Arjun checked. “Since 3 p.m., four. All four answered by a person within ten minutes.”

Recap

• AI calls fail in four families: my bug, their busy, the network, and success that isn’t.
• The SDKs already retry the busy and network families twice; your handler runs after that.
• Catch the SDK’s specific exception classes, most specific first; never parse error strings or catch everything blindly.
• Always check stop_reason and the content, even when the call returned 200.
• Show customers a friendly fallback, give humans the dropped questions, and count every fallback in your logs.

By Diwali evening the chat had answered over 5,000 questions and nobody saw a traceback. Meera was happy, and already thinking about the next thing. “After the festival,” she said, “I want the AI to read every review we got this season. All of them. By Tuesday.” Arjun opened a new file and typed for review in reviews:. Priya, passing by, winced. “Before you run that, let’s talk about doing three thousand things at once.”

Go deeper

• Anthropic docs (errors, rate limits and client options): https://docs.anthropic.com
• OpenAI docs (error codes): https://platform.openai.com/docs
• Python tutorial, errors and exceptions: https://docs.python.org/3/tutorial/