Designing complex agent graphs

Arjun had taken Meera’s instruction literally. “The whole support inbox,” she had said, so by Thursday his support agent had fourteen tools and a system prompt that scrolled for three screens. Order lookup, stock check, swap request, refund, recipe search, allergen lookup, store hours, delivery partner status, and six more. One agent, one loop, every tool in reach.

On Friday morning a mother wrote in on the WhatsApp line: “Does the walnut brownie box contain peanuts? My daughter is allergic.” The agent called get_order (there was no order), then check_stock at Baner, then replied that the brownies were “nut-free apart from walnuts”. The recipe archive says otherwise: the brownies are baked on shared equipment that also handles peanuts.

The same afternoon, someone asking where their plum cake was got offered a refund they had not asked for, because “refund” was one of the fourteen things the agent could do and it was feeling helpful.

Priya read both transcripts twice. “This is not a bad model,” she said. “It’s a bad org chart. You have hired one person and given them every job in the building.”

One person doing every job

“Picture our Baner store on a Saturday,” Priya said. “Who’s at the counter?”

“Whoever’s free,” said Arjun.

“And when someone asks if the plum cake has almonds?”

“They check the allergen binder. It’s behind the till.”

“When someone wants a refund?”

“They call the store manager. Counter staff can’t refund.”

“And when someone asks where their delivery is?”

“They look up the order on the tablet.”

Priya drew it. A front desk, three arrows, three different stations, all meeting again at the counter where the customer gets their answer. “Nobody at Crumb is expected to know everything. The counter’s only job is to send the question to the right place. Each station has a small, specific set of tools. The allergen binder never touches the refund system.”

Arjun’s fourteen-tool agent broke that rule twice. When a model sees fourteen tools, every tool is a possible next step, and every irrelevant one is a chance to go wrong. And because the allergy question went through a general agent, nothing forced it to read the recipe archive. It answered from what it believed, which is how hallucinations (confident, wrong answers) happen.

Where the analogy breaks: real staff remember the last customer and learn over time. Each branch of your graph knows only what is in the state you give it. If the allergy branch needs the order history, you have to put it there.

The shape: route, specialise, respond

The design Priya sketched is the routing pattern you met in Module 13’s workflow lesson, with an agent inside one of the branches:

flowchart LR
    S([START]) --> R[router]
    R -->|orders| O["orders (agent loop)"]
    R -->|allergy| A["allergy (retrieve + answer)"]
    R -->|refund| F["refund (triage graph)"]
    R -->|human| H[human handoff]
    O --> P[respond]
    A --> P
    F --> P
    H --> P
    P --> E([END])

Each box earns its place:

• router makes one judgement: what kind of message is this? It is an LLM call because customers phrase things in a thousand ways.
• orders is a real agent loop. Order questions need a variable number of lookups (“where is it, and can I swap it?”), so the model should choose the steps.
• allergy is not an agent. It always does the same two things: look up the recipe notes, then answer using only those notes. No choices, so no loop.
• refund is the triage graph you built last lesson, reused whole. Money rules stay in Python.
• human is the fallback for anything the router is unsure of.
• respond is the single exit. Every reply passes through it, so it is the one place to enforce rules that apply to all answers.

Notice what this design does to the two Friday failures. The allergy question can no longer reach get_order or the refund tool, because those tools do not exist in that branch. And the order branch’s model sees two tools, not fourteen.

A new tool belt: LangChain and LCEL

To build the branches compactly you will use a few pieces of LangChain, the library LangGraph grew out of. LangChain provides standard building blocks for LLM apps: chat model wrappers, prompt templates, output parsers and tools. LCEL (the LangChain Expression Language) is its way of snapping those blocks together with the | (pipe) character.

# lcel_demo.py
from dotenv import load_dotenv
from langchain_anthropic import ChatAnthropic
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

load_dotenv()
llm = ChatAnthropic(model="claude-opus-5-5", max_tokens=2000)

router_prompt = ChatPromptTemplate.from_messages([
    ("system", "You route messages for Crumb & Co., a bakery. Reply with exactly one "
               "lowercase word: orders (order status, delivery, swaps), allergy "
               "(ingredients, allergens, dietary needs), refund (money back, damaged "
               "items), or other. No punctuation."),
    ("human", "{message}"),
])
router_chain = router_prompt | llm | StrOutputParser()

print(router_chain.invoke({"message": "Does the walnut brownie box contain peanuts?"}))

allergy

Read router_prompt | llm | StrOutputParser() left to right, like a kitchen conveyor belt. Each piece is a Runnable (anything with an .invoke() method), and the pipe feeds one piece’s output into the next:

1. router_prompt takes the dictionary {"message": "Does the walnut..."} and fills in {message}, producing a list of two chat messages.
2. llm sends those messages to Claude and returns an AIMessage (LangChain’s object for a model reply).
3. StrOutputParser() pulls the plain text out of that message: "allergy".

The chain is itself a Runnable, so router_chain.invoke(...) runs all three.

Building the branches, one at a time

Put everything in one file, support_graph.py. First, the shared state. Keep it small: the conversation, plus the one fact every branch needs to know about, which route was chosen.

# support_graph.py
import json
import re
from typing import Annotated, TypedDict

from dotenv import load_dotenv
from langchain_anthropic import ChatAnthropic
from langchain_core.messages import AIMessage, SystemMessage
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.tools import tool
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode, tools_condition

load_dotenv()
llm = ChatAnthropic(model="claude-opus-5-5", max_tokens=2000)

class SupportState(TypedDict):
    messages: Annotated[list, add_messages]
    route: str

The router node. It wraps router_chain from above (paste the prompt and chain into this file) and validates the answer.

ROUTES = {"orders", "allergy", "refund"}

def router(state: SupportState) -> dict:
    question = state["messages"][-1].content
    label = router_chain.invoke({"message": question}).strip().lower().rstrip(".")
    return {"route": label if label in ROUTES else "human"}

"other", "Orders.", an empty string, a paragraph of apology: all land somewhere safe.

The orders branch: an agent loop as a subgraph. Last lesson you hand-wrote the tools node and the “did it ask for tools?” check. LangGraph ships both, ready-made, in langgraph.prebuilt:

• ToolNode(tools) reads the last message’s tool calls, runs each tool, catches errors, and returns one ToolMessage (a tool result linked to its request by ID) per call.
• tools_condition is a router that returns "tools" if the last message asked for tools, and END otherwise.

ORDERS = {
    "CR-1042": {"item": "Black Forest cake 1kg", "store": "Baner",
                "status": "delayed", "eta": "tomorrow 11:00"},
    "CR-1043": {"item": "Plum cake 500g", "store": "Aundh",
                "status": "out for delivery", "eta": "today 18:40"},
}
STOCK = {"Baner": ["Red velvet cake 1kg", "Truffle cake 1kg"],
         "Aundh": ["Plum cake 500g", "Walnut brownie box"]}

@tool
def get_order(order_id: str) -> str:
    """Look up a Crumb order by ID, like CR-1043. Returns item, store, status and ETA."""
    return json.dumps(ORDERS.get(order_id, "not found"))

@tool
def check_stock(store: str) -> str:
    """List cakes in stock right now at one Crumb store, like Baner or Aundh."""
    return json.dumps(STOCK.get(store, []))

order_tools = [get_order, check_stock]
order_llm = llm.bind_tools(order_tools)
ORDERS_SYSTEM = ("You are Crumb's order desk. Check facts with tools before answering. "
                 "Offer swaps only from in-stock items. Never promise refunds. Be brief.")

def order_agent(state: SupportState) -> dict:
    reply = order_llm.invoke([SystemMessage(ORDERS_SYSTEM), *state["messages"]])
    return {"messages": [reply]}

orders = StateGraph(SupportState)
orders.add_node("agent", order_agent)
orders.add_node("tools", ToolNode(order_tools))
orders.add_edge(START, "agent")
orders.add_conditional_edges("agent", tools_condition)   # "tools" or END
orders.add_edge("tools", "agent")
orders_graph = orders.compile()

orders_graph is a complete, runnable graph. You can test it alone with orders_graph.invoke({"messages": [("user", "Where is CR-1043?")]}). Shortly you will drop it into the main graph as a single node. A compiled graph used as a node inside another graph is called a subgraph.

One subtlety: inside the subgraph, tools_condition sends the finished conversation to END. That is the subgraph’s own END. It ends the orders branch, and control returns to the parent graph, which carries on along its own edges to respond. This is a big reason to use subgraphs: the inner loop’s exit cannot accidentally end the whole conversation.

The allergy branch: retrieve, then answer. A real system would search the recipe archive with embeddings (lists of numbers that capture meaning), as in Module 09. To keep the focus on graph design, this one scores recipe notes by shared words.

RECIPES = [
    "Walnut brownie box: flour, cocoa, butter, sugar, eggs, walnuts. Baked on shared "
    "equipment that also handles peanuts and almonds.",
    "Christmas plum cake: flour, butter, brown sugar, eggs, raisins, almonds, cashews, "
    "orange peel. Contains tree nuts.",
    "Eggless truffle pastry: flour, cocoa, condensed milk, cream, sugar. No eggs. "
    "Nut-free kitchen line at Aundh only.",
]

def words(text: str) -> set[str]:
    return set(re.findall(r"[a-z]+", text.lower()))

def retrieve(question: str, k: int = 2) -> list[str]:
    q = words(question)
    return sorted(RECIPES, key=lambda note: -len(q & words(note)))[:k]

allergy_prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer allergy questions for Crumb using ONLY these recipe notes. "
               "Always mention shared-equipment warnings. If the notes do not settle "
               "the question, say so and suggest calling the store.\n\n{notes}"),
    ("human", "{question}"),
])
allergy_chain = allergy_prompt | llm | StrOutputParser()

def allergy(state: SupportState) -> dict:
    question = state["messages"][-1].content
    notes = "\n".join(retrieve(question))
    return {"messages": [AIMessage(allergy_chain.invoke({"notes": notes,
                                                         "question": question}))]}

The refund branch: reuse last lesson’s graph. The triage graph uses different state keys (message, reply, log) from this one (messages, route). When the keys do not match, you cannot drop the compiled graph straight in. Instead, call it from inside an ordinary node and translate in both directions:

from triage import graph as triage_graph   # the graph from the previous lesson

def refund(state: SupportState) -> dict:
    result = triage_graph.invoke({"message": state["messages"][-1].content, "log": []})
    return {"messages": [AIMessage(result["reply"])]}

That gives you the two ways to use a subgraph:

Situation	How to add it
Subgraph shares the parent’s state keys (like orders_graph and messages)	builder.add_node("orders", orders_graph)
Subgraph has its own, different keys (like triage_graph)	Call triage_graph.invoke(...) inside a node and map the keys yourself

Human handoff and the shared exit.

def human(state: SupportState) -> dict:
    return {"messages": [AIMessage("Thanks for writing in! A teammate will reply "
                                   "within a few hours.")]}

def respond(state: SupportState) -> dict:
    last = state["messages"][-1]
    text = last.content
    # A rule for every branch, enforced in plain code, in one place.
    if state["route"] != "refund" and "refund" in text.lower():
        text = "A teammate will follow up about this shortly."
    # Same id as the last message, so add_messages REPLACES it instead of appending.
    return {"messages": [AIMessage(content=text + "\n\n- Team Crumb", id=last.id)]}

That id=last.id uses the replace-by-ID behaviour of add_messages from last lesson: the final message is swapped for the signed, checked version rather than duplicated.

Assemble the parent graph.

builder = StateGraph(SupportState)
builder.add_node("router", router)
builder.add_node("orders", orders_graph)       # a compiled graph as a node
builder.add_node("allergy", allergy)
builder.add_node("refund", refund)
builder.add_node("human", human)
builder.add_node("respond", respond)

builder.add_edge(START, "router")
builder.add_conditional_edges("router", lambda s: s["route"],
                              ["orders", "allergy", "refund", "human"])
for branch in ["orders", "allergy", "refund", "human"]:
    builder.add_edge(branch, "respond")
builder.add_edge("respond", END)
graph = builder.compile()

Passing a list to add_conditional_edges means “the router returns node names directly”. It still documents every possible destination, which LangGraph uses to draw the graph and to check the wiring.

Library APIs change quickly. If a name has moved, check the docs linked below.

Show the answer

Because there is nothing for the model to decide. Every allergy question needs the recipe notes, every time. As a tool, searching would be optional: the model could skip it and answer from memory, which is exactly the Friday failure. Making retrieval a fixed step guarantees it happens, costs one fewer model call, and is easier to test. Use an agent loop only where the number or order of steps genuinely varies.

Following one message through the whole graph

A customer writes: “Where is my order CR-1043? It was supposed to come at 6.”

result = graph.invoke({"messages": [("user", "Where is my order CR-1043? "
                                              "It was supposed to come at 6.")]},
                      config={"recursion_limit": 15})
for m in result["messages"]:
    print(type(m).__name__, "|", m.content if m.content else m.tool_calls)

Here is the journey, with the state after each step.

1. router. The chain fills the prompt (system rules plus the message, about 150 tokens), Claude replies orders, the parser returns "orders", validation passes. State: one HumanMessage, route: "orders". The conditional edge reads route and sends control to orders.

2. orders subgraph, agent node. The model sees the order-desk system prompt, the customer message and the two tool definitions (about 700 tokens in all, because each tool’s schema counts). It replies with an AIMessage whose tool_calls is:

[{"name": "get_order", "args": {"order_id": "CR-1043"}, "id": "toolu_01..."}]

tools_condition sees a tool call and returns "tools".

3. orders subgraph, tools node. ToolNode runs get_order("CR-1043") and appends:

ToolMessage(content='{"item": "Plum cake 500g", "store": "Aundh", '
                    '"status": "out for delivery", "eta": "today 18:40"}',
            tool_call_id="toolu_01...")

The plain edge goes back to agent.

4. orders subgraph, agent again. Now the model sees the tool result (about 830 tokens in) and writes a text answer with no tool calls: “Your plum cake left our Aundh store and is out for delivery. The latest ETA is 18:40 today, so it’s running about 40 minutes late. Sorry for the wait!” tools_condition returns END, which ends the subgraph.

5. respond. Back in the parent graph. route is "orders", and the text does not mention refunds, so the rule does not fire. The last message is replaced with the signed version.

The final messages list has four entries: HumanMessage, AIMessage (tool call), ToolMessage, AIMessage (signed answer). Three model calls in total: roughly 1,680 input tokens and 100 output tokens, which at $4and$20 per million is about $0.009, or under one rupee. It takes around three to five seconds end to end, most of it the two order-agent calls.

Now send the Friday question, “Does the walnut brownie box contain peanuts? My daughter is allergic.” The router says allergy. retrieve ranks the brownie note first (it shares “walnut”, “brownie” and “box”). The chain answers from that note, and the reply warns about shared equipment with peanuts. Two model calls, no order tools anywhere in sight.

Design principles that keep graphs sane

Priya wrote four rules on the whiteboard and made Arjun copy them into the repo’s README.

1. Keep the state small and typed. Every key in the state is something every node can read and might write. Arjun’s first draft had twenty keys, half of them flags like has_checked_stock. Flags in state recreate the four-hundred-line mess in dictionary form. Store facts (route, order_id), not progress markers; the graph’s position already says where you are.

2. One job per node. A node called classify_and_lookup_and_reply cannot be tested, retried or reused. If you cannot name a node with one verb, split it.

3. Deterministic code wherever possible. Routing labels, refund limits, sign-offs, “never mention refunds outside the refund branch”: these are rules, and rules belong in Python. Each model call adds cost, latency and a chance of being wrong. Use the model for language and judgement; use code for everything you could write as an if.

4. Explicit limits on every loop. The orders subgraph is a cycle. Always pass a recursion_limit you chose deliberately, and for loops you design yourself (like a “check and redraft” loop), keep a counter in state and route out when it hits a maximum.

And one rule about when to reach for a subgraph: when a piece of the graph has its own internal loop, or is reused in several places, or is owned by a different person. The orders loop is all three.

Show the answer

The router must pick one label, probably orders (or occasionally allergy). Whichever branch runs answers its half. The other half is silently dropped, because the orders agent has no recipe tool and the allergy chain has no order tool. The fixes, in rough order of effort: let the router return several labels and run those branches in parallel (the parallelisation pattern from Module 13), then combine their answers in respond; or have respond notice an unanswered question and append “I’ve passed your other question to a teammate.” Multi-intent messages are common on WhatsApp, so plan for them.

Where it breaks

• Routing errors are the new failure mode. A misrouted message gets a confident answer from the wrong specialist. Build a small test set (fifty real messages with the correct label) and measure the router’s accuracy whenever you change its prompt. Module 23 shows you how.
• Every router costs a call. About a second and a few paise per message. For high volume, a keyword pre-check (“contains an order number and the word ‘where’”) can skip the model for obvious cases.
• Subgraph keys must line up. Add a compiled graph as a node when its state shares keys with the parent, and the keys it does not share are simply invisible to it. If the orders subgraph needed customer_tier and the parent never set it, the subgraph would fail or behave oddly. Wrap it in a function node when in doubt.
• The respond node’s rule is crude. A keyword check on “refund” will catch “we cannot refund this” as well. That is acceptable for a safety net, but it is not a substitute for a well-scoped branch.
• Graphs get big. Ten branches with subgraphs inside subgraphs are hard to reason about. When a graph stops fitting on one screen, split it into subgraphs with clear owners.

Try it yourself

Add a fifth branch, store_info, for questions about opening hours. Make it deterministic: a dictionary of store hours and a node that finds the store name in the message with a regex or a word match. Update the router prompt and ROUTES. Then send “What time does the Aundh store close on Sunday?” and “Is Baner open on Diwali?” (assume your dictionary has no holiday data).

Show the expected result

The first message routes to store_info and gets the Sunday hours with no extra model call beyond the router. The second should not invent holiday hours. A good node replies with the regular hours and adds “Holiday hours vary; please call the store”, or routes to human. If your router prompt still lists only four options, the model will never say store_info, so the branch never runs: remember to update both the prompt and the ROUTES set.

Common mistakes

• One agent with every tool. More tools means more wrong turns. Give each branch only the tools its job needs.
• Making retrieval optional. If an answer must be grounded in documents, retrieve as a fixed step, not a tool the model may skip.
• Unvalidated router labels. Strip, lowercase, check the set, fall back to human.
• Using with_structured_output for routing on this model. Ask for a single word with StrOutputParser and validate in Python.
• Forgetting the subgraph’s END is local. It ends the branch, not the conversation. That is usually what you want, but know it.

Recap

• Design agent graphs like a small team: a router at the front, specialist branches with only the tools they need, and a shared respond exit.
• LCEL snaps prompt, model and parser together: prompt | llm | StrOutputParser().
• ToolNode and tools_condition from langgraph.prebuilt give you the agent loop in three lines.
• A compiled graph can be a node (subgraph) when it shares state keys; otherwise wrap it in a function node.
• Small typed state, one job per node, code for rules, and explicit loop limits.

At the end of the day Arjun walked Meera through the diagram. “Every message hits a front desk first. It sends allergy questions to a station that can only read the recipe archive. Order questions go to a station that can only look up orders and stock. Refunds go to the refund process we built yesterday, where the rupee limits are written in code. And everything leaves through one door, where we check nobody promised something they shouldn’t.”

“So the brownie mistake can’t happen again?”

“That exact one can’t. The allergy station can’t skip the recipe notes. Something new can still go wrong, and when it does I want to see exactly where.” He did not know yet that he would need that sooner than he thought: on the first evening of the Diwali rush, the allergy branch would get stuck in a loop, and Priya would show him how to rewind a graph to the moment it went wrong.

Go deeper

• LangGraph documentation: the how-to guides on subgraphs and on the prebuilt ToolNode.
• LangChain documentation: LCEL, prompt templates and the Anthropic chat model integration.
• Building effective agents (Anthropic): the routing pattern and when to use agents versus workflows.