What is an AI agent?
Explain the difference between a workflow and an agent, trace one request through the think-act-observe loop with its real cost, and decide when not to build an agent.
Words in this lessonAgent · Workflow (agentic workflow) · Agent loop · ReAct · Autonomy levels · Tool calling · Hallucination
- Agent
- A model running in a loop that decides its own next step, uses tools, checks results and keeps going until a task is done. Like a personal assistant you hand a goal, who makes the calls and checks back, rather than one you direct step by step.
- Workflow (agentic workflow)
- A system where a language model is called at fixed points along a path that your code decides in advance. Like a recipe card: every step is written down, so two trainees following it make the same cake.
- Agent loop
- The repeating cycle at the heart of every agent: decide what to do, do it with a tool, look at the result, and decide again. Like a head chef at the pass who calls an order, hears what the runner found, and calls the next order.
- ReAct
- A way of running a model that interleaves reasoning about what to do with actions that gather real information. Like a detective who thinks out loud, checks one clue, and lets that clue decide the next question.
- Autonomy levels
- A ladder describing how much control a system hands to the model, from a single call up to teams of agents. Like handing a new employee more responsibility one level at a time, from following a checklist to running a project.
- Tool calling
- When a model asks your code to run a named function with specific inputs, then uses the result. Like a waiter who cannot cook but can send a ticket to the kitchen and bring back what comes out.
- Hallucination
- When a model gives a fluent, confident answer that is false or made up. Like a nervous new employee who would rather invent an answer than say "I don't know".
On this page
On the first Monday of November, Crumb & Co.'s support inbox had 312 unread emails. Diwali hamper season had started, and with it came late deliveries, nut-allergy questions, and refund requests for squashed cakes. Two people handled support. Between them they spent about twelve hours a day answering email, and the queue still grew.
Meera stood by Arjun’s desk with the printout. “Every year this happens, and every year we hire temps who don’t know the menu. You keep telling me about AI. Can it do this?”
Arjun had spent the weekend watching demos of AI systems that booked flights and fixed code by themselves. He had a word for what he wanted to build, and the word was agent. “Give me two days,” he said. “I’ll build an autonomous agent that reads every email and handles it. End to end. No humans.”
Priya, sitting across the aisle, looked up from her screen. She did not say no. She said, “Show me the first version before lunch.”
Important
The idea in one sentence: an AI agent is a language model running in a loop, choosing its own next step and using tools until a goal is met; you reach for one only when a fixed, pre-planned sequence of steps cannot do the job, because the freedom that makes agents powerful also makes them slower, costlier and harder to trust.
Arjun’s first “agent”
Arjun’s first version took twenty minutes. He wrote a big system prompt (the standing instructions a model reads before every conversation) and sent each email to a large language model, or LLM (a program trained on huge amounts of text to predict the next word, and so to write).
SYSTEM = """You are Crumb & Co.'s autonomous support agent.
Handle every customer email completely. Resolve their problem."""
email = "Order CR-1042 hasn't arrived and the party is at 7 pm. Help!"
# ... one call to the model with SYSTEM and the email ...
The reply came back in two seconds, warm and confident:
“So sorry for the wait! Your order CR-1042 is out for delivery and will reach you by 6:30 pm. Enjoy the party!”
Arjun beamed. Priya opened the order database in another tab. CR-1042 was not out for delivery. It was stuck at the Baner store because the delivery partner had cancelled. The model had invented a status, a time, and a promise.
“It sounds so sure,” Arjun said.
“It always sounds sure,” Priya said. “That is a hallucination (when a model states something false as if it were fact). It wasn’t lying. It had no way to look anything up, so it wrote the most likely-sounding reply. You told it to resolve the problem, and it did, in words.”
Arjun tried again. This time he gave the model three functions it could ask to use: get_order, check_stock and issue_refund. He let it call them as many times as it liked. On a test email (“my cake was a bit late, not happy”), it looked up the order, saw it was delivered twenty minutes late, and issued a full ₹1,450 refund. Crumb’s policy for a twenty-minute delay is an apology and a discount code.
“Now it’s not making things up,” Arjun said slowly. “It’s doing things up.”
Priya laughed. “That’s a better problem. Let’s understand it properly.”
Recipe cards and head cooks
Priya picked up a marker and drew two boxes on the whiteboard.
“When a trainee in our Aundh kitchen makes the Christmas plum cake, what do they follow?”
“The recipe card.”
“Right. Soak the fruit. Cream the butter. Fold in the flour. Every step written down in advance. If the oven runs hot, does the card notice?”
“No.”
“But two trainees with the same card make the same cake. Predictable. Now, the head cook on a chaotic Saturday. Does anyone hand her a card?”
Arjun thought about the Baner kitchen during Diwali week. “No. She looks at the order screen, sees they’re out of almonds, phones Aundh, swaps in cashews, tastes, adjusts…”
“She decides the next step based on what just happened. That’s the difference I want you to hold onto. Not how smart the system is. Who decides what happens next.”
In AI engineering, these two shapes have names.
- A workflow is the recipe card. The model is called at fixed points along a path that you wrote in code. Classify the email, then draft a reply, then check the tone. The order of steps never changes.
- An agent is the head cook. The model looks at the situation, decides what to do, does it, looks at the result, and decides again. The model chooses the path.
This is the line Anthropic draws in its widely read essay Building effective agents: workflows are “systems where LLMs and tools are orchestrated through predefined code paths”, while agents are “systems where LLMs dynamically direct their own processes and tool usage”. The essay is linked at the end. It settles most arguments about what counts as an agent.
In plain English: in a workflow, your code holds the steering wheel. In an agent, the model holds the steering wheel, and your code holds the brakes.
Where the analogy breaks: a real head cook has twenty years of judgement and remembers last Diwali. A model has neither by default. It knows only what is in its context window (the text it can see during this one request), and it can be confidently wrong, as Arjun just saw. That is why agents need brakes a human cook never would.
Note
Pause and think. Arjun’s first attempt (one prompt, one reply) and his second (three tools, unlimited calls) failed in opposite ways. Which one was a workflow, which one was an agent, and what went wrong with each?
Show the answer
The first was neither, really: a single model call, the simplest possible system. It failed because it had no way to check facts, so it guessed. The second was an agent: the model chose which tools to call and when. It failed because nothing in the code limited what it could do, so it applied its own idea of fairness instead of Crumb’s refund policy. The first lacked information. The second lacked limits.
The loop inside every agent
“So what makes the head cook a head cook?” Priya asked. “Write down what she does, as steps.”
Arjun wrote: look at the situation, decide, do something, see what happened, repeat.
“That’s it. That’s every agent ever built, from a ten-line script to the coding assistants you watched all weekend.”
- Think. The model reads the goal and everything that has happened so far, and decides on the next step.
- Act. It asks to use a tool (a function your program runs on its behalf, like
get_order). This is called tool calling: the model never runs anything itself. It writes a structured request, and your code does the work. - Observe. Your code runs the tool and hands the result back to the model as new text in the conversation.
- Repeat until the model decides the goal is met, or until your code stops it.
- Think: a customer says the cake arrived squashed. I should check the order.
This think, act, observe pattern was named ReAct (short for Reason + Act) in a 2022 research paper by Yao and colleagues. Before ReAct, models either reasoned without acting, and made facts up, or acted without reasoning, and flailed. Weaving the two together, a thought, then an action, then a look at the result, made them far more reliable. Nearly every agent library you will meet in this module is a version of that loop.
Following one email through the loop
Priya made Arjun trace one email by hand, every step, before writing more code. Here is CR-1042 again, this time with an agent that has two read-only tools, get_order and check_stock, and a system prompt that says “check facts with tools; never promise refunds”.
Turn 1. The model thinks, then asks for a tool. It receives about 1,200 tokens (the small chunks of text models read and are billed by, roughly three-quarters of an English word each): the system prompt, the two tool descriptions, some built-in tool-use instructions, and the email. It replies with a request, not an answer:
{"type": "tool_use", "name": "get_order", "input": {"order_id": "CR-1042"}}
Your code acts. It runs get_order("CR-1042") against the database and gets back:
{"item": "Black Forest cake 1kg", "store": "Baner",
"status": "delayed", "reason": "delivery partner cancelled", "eta": "tomorrow 11:00"}
Turn 2. The model observes and thinks again. The whole conversation goes back, now about 1,400 tokens including the tool result. The model reads “delayed until tomorrow”, notes the party is tonight, and decides a swap might help. Nobody coded that idea. It asks for another tool:
{"type": "tool_use", "name": "check_stock", "input": {"store": "Baner"}}
Result: ["Red velvet cake 1kg", "Truffle cake 1kg"].
Turn 3. The model decides it is done. About 1,550 tokens in. No more tool requests. It writes:
“I’m so sorry: your Black Forest cake is delayed until tomorrow because our delivery partner cancelled. Our Baner store has a Red velvet or Truffle cake (1 kg) ready now. Would you like to swap, and shall I arrange a pickup or a new delivery?”
Every fact in that reply came from a tool. Now count the cost. Claude Opus 5.5 is priced at 20 per million output tokens (prices change, so always check the provider’s pricing page). The three turns read about 4,150 input tokens and wrote about 450 output tokens, including the model’s internal reasoning.
| Tokens | Cost | |
|---|---|---|
| Input (3 calls) | ~4,150 | ~$0.017 |
| Output (3 calls) | ~450 | ~$0.009 |
| Total | ~$0.026, about ₹2.30 |
Arjun’s original one-shot reply cost about $0.006, under 60 paise, and took two seconds. The agent took about seven seconds and cost four times as much. For 312 emails a day that is roughly ₹720 a day instead of ₹180. Cheap compared with a temp’s wages. But notice the shape: every extra loop re-reads the whole conversation, so cost and time grow with every step.
Note
Pause and think. In turn 2, the model decided by itself to check stock at Baner. Is that a good thing or a bad thing?
Show the answer
Both. It is good because it produced a genuinely helpful reply nobody had scripted. It is risky for exactly the same reason: if the model can choose helpful steps you did not plan, it can choose unhelpful ones too, like the ₹1,450 refund. The freedom and the risk are the same property. That is why you decide which tools an agent gets, and put limits inside those tools.
A ladder, not a switch
“So is a workflow bad and an agent good?” Arjun asked.
“Neither. It’s a ladder.” Priya drew rungs on the whiteboard. “Each rung gives the model a little more of the steering wheel.”
| Rung | What it looks like at Crumb | Who decides the next step? |
|---|---|---|
| 0. Single call | Summarise one email | Nobody: there is only one step |
| 1. Fixed chain | Classify, then draft, then tone-check | Your code, same path every time |
| 2. Router | Send allergy emails to a specialist prompt, refunds to another | Model picks a branch; your code defines the branches |
| 3. Tool-using agent | Look up orders and stock as needed, at most 8 steps | Model, inside limits you set |
| 4. Long-running agent | Work for an hour on the codebase, running tests | Model, with checkpoints and human approvals |
| 5. Multi-agent system | A planner hands work to specialist agents | Several models, coordinating |
The higher the rung, the more flexible the system and the harder it is to predict, test and pay for. Most products that work well in the real world live on rungs 1 to 3. You will build most of these rungs in this module: the loop itself in the next lesson, the workflow rungs in Workflow patterns, rung 5 in Multi-agent systems, and rung 4 in Module 14’s coding agent project.
Why agents break in ways workflows don’t
Three costs grow as you climb, and they compound.
Slower and pricier. You saw it in the trace: three calls instead of one, and each call re-reads everything before it. A twenty-step agent can easily cost fifty times a single call.
Errors multiply. Suppose each step is right 95% of the time. That sounds great. But for a ten-step task to succeed, every step must go right: 0.95 multiplied by itself ten times is about 0.60. One in four tasks fails somewhere. A two-step workflow with the same per-step accuracy succeeds about 90% of the time. Long autonomous chains are fragile unless something checks the work along the way.
Harder to test. A workflow’s path is fixed, so you can write a test for each step. An agent’s path changes from run to run. You need evals (repeatable tests that run many examples and score the results), which you will build in Module 23.
There is also a misconception to bury here. Many beginners think an agent is a smarter model. It is not. The same model sits inside a single call, a workflow and an agent. What changes is the system around it: who chooses the steps, which tools exist, and what limits apply. An agent is an architecture, not an IQ upgrade.
When NOT to build an agent
Priya wrote four questions on the whiteboard and underlined them. Ask them before you climb past rung 2.
- Can I write the steps down in advance? If yes, write a workflow. “Damaged cake, under ₹500, photo attached: refund automatically” is a policy, not a judgement. Code it.
- Is the task genuinely open-ended? “Find out why Koregaon Park’s online sales dropped in October” has no fixed path: you don’t know which data you’ll need until you look. That is agent territory.
- Can the result be checked? Agents shine when something can verify their work: tests pass, the order exists, the total adds up. Without a check, you are trusting compounding guesses.
- What does a mistake cost? A draft reply that a human reviews is cheap to get wrong. Money leaving the account, or an email going straight to a customer, is not. High-stakes actions need a human in the loop or a hard rule in code.
Warning
“Never refund more than ₹500” written in a prompt is a suggestion the model usually follows. Written in the issue_refund function as if amount > 500: return "Refused: needs manager approval", it is a rule. Put every limit that matters in code.
Try it yourself
Sort these six Crumb tasks onto the ladder (rung 0 to 5). No code, just judgement. Then check your answers.
- Translate the Diwali menu into Marathi.
- Tag each support email as delivery, allergy, refund or other, then send it to the right queue.
- Answer “Is the walnut brownie safe for my nut allergy?” using the recipe archive.
- Investigate why online orders failed between 2 and 4 pm yesterday, using logs, the order database and the deployment history.
- Plan, price and write copy for twelve new Christmas hampers, checking stock in all six stores.
- Reply to a customer’s thank-you note.
Show the answer
- Rung 0. One call, no decisions.
- Rung 2. A router: the model picks a label, your code sends it on. If you put this on rung 3, ask what an agent would decide here that a router cannot. Nothing.
- Rung 1, maybe 3. A fixed retrieve-then-answer chain usually works. If questions get tricky (“is it made on the same line as the almond cake?”), a small agent that can search several times helps. You will build that in Agentic RAG.
- Rung 3 or 4. Open-ended, many possible paths, and checkable against the logs. A good fit.
- Rung 5, or rung 3 with care. Several distinct skills (menu, stock, pricing, copy) could be specialists. One agent with good tools might do it just as well, more cheaply.
- Rung 0. Please do not build an agent for this.
Common mistakes
- Starting with the agent. Build the simplest thing that could work: one call, then a chain. Climb only when you hit a wall you can name.
- No step limit. A confused agent can loop for minutes, burning money each turn. Always cap the number of turns in code.
- Trusting the prompt to enforce rules. Prompts steer; code enforces. Spending limits, permissions and approvals go in code.
- Giving write tools on day one. Start an agent with read-only tools (look up, search, check). Add actions that change the world one at a time, each with its own limit.
- Thinking “agent” means “smarter”. Same model, different architecture. If a single call gets it wrong, an agent built on that model can get it wrong several times in a row.
Recap
- A workflow follows steps you wrote. An agent lets the model choose its next step. The question is always: who decides what happens next?
- Every agent runs a think, act, observe loop, named ReAct, where tools supply facts the model cannot know.
- Each loop re-reads the conversation, so agents cost more and run slower: about ₹2.30 and seven seconds for one email, against 60 paise and two seconds for a single call.
- Autonomy is a ladder. Errors compound with every step, so most real products live on rungs 1 to 3.
- Build an agent when the task is open-ended and checkable. Keep limits in code.
Before lunch, Arjun walked Meera through it. “Version one made up a delivery time. Version two gave away a refund. So here’s the plan: a router sorts the inbox into four queues, which is cheap and predictable. Only the messy ‘where is my order and what can I do’ emails go to a small agent. It can look things up, but it can’t spend money. Anything involving refunds goes to a human, with the agent’s draft attached.”
Meera tapped the printout. “So it’s not replacing the team. It’s doing the looking-up for them.”
“For now,” said Priya. “This afternoon, Arjun writes that small agent in plain Python. No frameworks. About forty lines.”
Go deeper
- Building effective agents (Anthropic), the essay this module leans on most.
- ReAct: Synergizing Reasoning and Acting in Language Models, the paper that named the loop.
- Anthropic documentation, especially the tool use overview.