Role-play bots, learning advisors and assistants

Crumb was hiring eight seasonal staff for the Christmas rush: plum cakes, stollen, the office-party orders. Every one of them would face an upset customer within their first week. Kavita normally trained new people by role-playing difficult customers with them for an afternoon. Eight people meant eight afternoons she did not have in December.

Meera’s idea was simple. “Can your AI pretend to be an angry customer? They practise on it, as many times as they like, and Kavita only checks in at the end.”

There was a second request waiting behind it. Sana and Imran, Crumb’s two apprentice bakers, were studying for a diploma alongside their shifts. The head baker had asked whether the AI could help them learn, “not do their homework for them”. And Kavita wanted something for the seasonal staff once they were on the floor: a support assistant they could ask “what’s our policy on cakes left in a hot car?” without hunting through the 200-page operations handbook.

Three bots, Arjun thought. A customer, a tutor and a helper. All of them are just a system prompt with a personality.

He was about to find out how much work “just” was doing in that sentence.

Arjun’s angry customer

Arjun’s first version:

SYSTEM = "Pretend to be an angry customer of Crumb & Co., a bakery in Pune."

He wired it into the chatbot loop from Module 02 and asked Riya, one of the new seasonal hires, to try it. Here is how it went:

Bot: I am absolutely furious! My order was a disaster!

Riya: I’m so sorry to hear that. Could you tell me what happened?

Bot: Well, thank you for asking so kindly. My cake was late. But honestly, these things happen, and I appreciate your help! Is there anything else I can assist you with?

Two turns, and the angry customer had turned into a cheerful support agent offering to assist her. Riya laughed. Arjun tried again with “Stay angry no matter what.” The next session went the other way: Riya apologised, offered a replacement, and arranged a refund, and the bot kept shouting for twelve turns. Nothing she did made any difference.

“Neither of those teaches anything,” Kavita said, reading over his shoulder. “Real customers calm down when you do it right, and they get worse when you do it wrong. That’s the whole skill.”

His first tutor attempt fared no better. With “You are a baking tutor”, Sana asked why her sourdough had not risen. The bot gave her a complete, excellent answer in six bullet points. She copied it into her notebook and could not explain it to the head baker an hour later. Then Imran asked what temperature cooked custard should be cooled to, and how quickly, and the bot gave confident numbers that Chef Nikhil said were not what Crumb’s food-safety plan used.

A flight simulator, not a mood

Priya drew an aeroplane cockpit on the whiteboard, badly. “Pilots train in flight simulators. What makes a simulator useful, rather than merely frightening?”

Arjun thought about it. “It reacts to what you do. If you pull up, the plane climbs.”

“What else?”

“There’s a scenario. Engine failure on take-off, or whatever.”

“Who decides the scenario?”

“An instructor.”

“And when the session ends?”

“There’s a debrief. Someone tells you what you did right and wrong.”

“So a simulator has four parts: a scenario, a model of the world that reacts to your actions, an instructor who sets things up and can stop them, and a debrief. Your bot has a mood. ‘Be angry’ is a mood. A mood can’t react, so it either collapses or never changes.”

“So the customer needs… a scenario and some kind of state? Like how annoyed he is right now?”

“And rules for what changes it. And somebody else does the debrief, because an actor who’s busy being furious is not the right person to grade you.”

The general technique is persona prompting: telling the model to speak as a particular character or expert. In Module 03 you used a light persona (a food-safety officer) to shift what the model paid attention to. A role-play bot needs a far richer persona: a character sheet, the kind an actor or a game designer would write, with a background, goals, a current emotional state, what moves that state, and hard limits.

Following one practice session from start to debrief

Priya and Arjun built the simulator in three parts: the actor (plays the customer), the state (a number the actor updates each turn), and the coach (a separate call that grades the session afterwards). Then they traced a full session with Riya.

The character sheet.

You are playing Mr. Rao, a Crumb customer, in a staff training simulation.

Scenario: You ordered a 1 kg Christmas plum cake (order 8812) for delivery
to your flat in Baner by 6 p.m. today. It is 6:40 p.m. Twelve guests arrive
at 7:30. The delivery app says "delivered", but nothing reached you.

Personality: an accountant, precise, polite by nature, very stressed.
You speak in short sentences. You are not abusive and never use slurs.

Calm meter: starts at 2 (out of 10). Update it every turn:
+2 if the staff member apologises specifically (not "sorry for any inconvenience")
+2 if they give a concrete next step with a time
+1 if they use your name
-2 if they blame you, the app or the delivery partner
-2 if they quote policy instead of helping
-3 if they tell you to calm down
The scene ends when the meter reaches 8 (you thank them) or after 10 turns.
Never break character, never act as the staff member, never give hints.

Turn by turn. Each turn, the actor returned its reply and its updated meter, as structured output. Riya saw only the reply. Here is what the system saw:

Turn	Riya said	Meter	Why it moved
Start	(bot opens) “My plum cake says delivered. It isn’t here. Guests at 7:30.”	2
1	“Sorry for any inconvenience. Our policy is that the app status is updated by the partner…”	0	generic apology (+0), quoting policy (-2)
2	“Mr. Rao, I’m really sorry, your cake should be with you and it isn’t, on a night with guests coming.”	3	specific apology (+2), name (+1)
3	“I’m calling the rider now and I’ll message you by 6:50 with exactly where it is.”	5	concrete next step with a time (+2)
4	“Found it: delivered to B-wing instead of C-wing. The rider is bringing it over, ETA 6:58.”	8	concrete next step with a time (+2), problem solved (+1, the actor’s judgement)

At 8, Mr. Rao said: “Thank you. That’s a relief. Please tell the rider C-wing is on the left.” Scene over.

The debrief. A separate call received the full transcript, the meter history and Kavita’s rubric, and wrote feedback. Part of it:

Strong recovery. Turn 1 lost ground: “sorry for any inconvenience” sounds scripted, and explaining how the app updates is policy, not help. From turn 2 you did exactly what works: you used his name, said what went wrong in his terms (“on a night with guests coming”), and gave a time you could keep (6:50). Next time, start with turn 2.

Every claim in the feedback pointed to a specific turn, because the rubric asked for quotes.

The cost. The session used five actor calls (each about 900 tokens in, including the growing transcript, and 80 out) and one coach call (about 1,500 in, 300 out). On a small model at illustrative prices of $0.25permillioninputtokensand$2 per million output, about $0.003, or ₹0.26 per session. Eight trainees doing ten sessions each: about ₹21 for the whole Christmas intake.

Opening the hood: actor, coach and advisor are different jobs

The simulator worked because each call had one clear job. That principle carries over to all three bots.

The actor stays in character and updates state. It must not help the trainee, and it must not grade. Combining those jobs is what made Arjun’s first bot collapse: chat models are trained to be helpful assistants, so when a persona is thin, the assistant leaks back in. A detailed character sheet, a goal for the character, and an explicit “never act as the staff member” keep the actor in role.

The coach reads the transcript with fresh eyes and a rubric. Asking the actor to grade at the end would mix two perspectives in one context: it has spent ten turns being Mr. Rao. A separate call, with a rubric written by Kavita, gives consistent, specific feedback, and you can improve the rubric without touching the character.

The learning advisor has the opposite goal from most chatbots: it should not hand over the answer. The established approach is Socratic tutoring: answering questions with guiding questions and hints that lead the learner to work it out, the way a good teacher does. In practice, a hint ladder works well: a gentle question first, then a more pointed hint, then a worked partial step, and the full answer only after the learner has tried or explicitly asks a second time.

For Sana’s sourdough, the first rung was: “What does the yeast in your starter need to produce gas? Which of those might have been missing in your kitchen this week?” She remembered the cold snap, and the answer was hers.

The support assistant has a different priority again: every fact must be right. The way to get there is grounding: giving the model the relevant source text (here, sections of the operations handbook) and instructing it to answer only from that text, cite the section, and say “I don’t know, please ask Kavita” when the text does not cover the question. A grounded assistant is only as good as the passages you give it. Finding the right passages in a 200-page handbook automatically is retrieval, the core of RAG, which you will build properly in Module 09. For now, with a small handbook, you can include the relevant chapter directly.

Building the three bots, step by step

Step 1: the character sheet as data. Store scenarios as files, so Kavita can write new ones without touching code: scenarios/wrong_address.txt holds the Mr. Rao sheet above. A second, scenarios/allergy_scare.txt, has a worried parent; a third, a wholesale client whose standing order was forgotten.

Step 2: the actor turn, with state. Install with uv add anthropic pydantic python-dotenv.

import anthropic
from dotenv import load_dotenv
from pydantic import BaseModel, Field

load_dotenv()
client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY

class ActorTurn(BaseModel):
    reply: str = Field(description="What the customer says next, in character")
    calm: int = Field(ge=0, le=10, description="Updated calm meter")
    why: str = Field(description="One line: what in the staff member's message moved the meter")
    scene_over: bool

def actor_turn(sheet: str, transcript: list[dict], calm: int) -> ActorTurn:
    response = client.messages.parse(
        model="claude-opus-5-5",
        max_tokens=2000,
        system=f"{sheet}\n\nCurrent calm meter: {calm}.",
        messages=transcript,              # alternating: assistant = customer, user = trainee
        output_format=ActorTurn,
    )
    return response.parsed_output

The trainee’s messages are the user turns and the customer’s lines are the assistant turns, so the model speaks as the customer. Your code passes the current meter into the system prompt every turn and keeps the history, as in the context engineering lesson.

Step 3: the session loop.

def run_session(sheet: str, opening: str, max_turns: int = 10) -> tuple[list[dict], list[int]]:
    transcript = [{"role": "user", "content": "(The customer contacts the store.)"},
                  {"role": "assistant", "content": opening}]
    calm, history = 2, [2]
    print(f"Customer: {opening}")
    for _ in range(max_turns):
        transcript.append({"role": "user", "content": input("You: ")})
        turn = actor_turn(sheet, transcript, calm)
        calm = turn.calm
        history.append(calm)
        transcript.append({"role": "assistant", "content": turn.reply})
        print(f"Customer: {turn.reply}")
        if turn.scene_over or calm >= 8:
            break
    return transcript, history

You should be able to play the Mr. Rao scene in the terminal. Try saying “please calm down” and watch the meter drop in history.

Step 4: the coach, as a separate call with a rubric.

RUBRIC = open("rubrics/kavita_v2.txt").read()   # specific apology, ownership, concrete times, no blame

def debrief(transcript: list[dict], history: list[int]) -> str:
    lines = "\n".join(f"{'Staff' if m['role'] == 'user' else 'Customer'}: {m['content']}"
                      for m in transcript[1:])
    response = client.messages.create(
        model="claude-opus-5-5",
        max_tokens=2000,
        system="You are Kavita's coaching assistant. Give specific, kind, honest feedback. "
               "Quote the staff member's exact words for every point. Max 150 words.",
        messages=[{"role": "user", "content":
                   f"Rubric:\n{RUBRIC}\n\nCalm meter by turn: {history}\n\nTranscript:\n{lines}"}],
    )
    return next(b.text for b in response.content if b.type == "text")

Step 5: the tutor and the assistant, as two more system prompts. The structure is the same chat loop; the difference is entirely in the briefing and the grounding.

TUTOR = """You are a baking tutor for Crumb's apprentices, Sana and Imran.
Teach by questions. For a "why" or "how" question, use this hint ladder:
1. Ask one guiding question. 2. If they are stuck, give a pointed hint.
3. Then a worked partial step. 4. The full answer only after they have tried
or ask twice. For reference facts (temperatures, ratios, times), answer directly
but ONLY from <course_notes>; if the notes do not cover it, say so.
Learner profile: {profile}
<course_notes>
{notes}
</course_notes>"""

ASSISTANT = """You help Crumb's store staff with policy questions.
Answer ONLY from <handbook>. Cite the section number, like (Handbook 7.3).
If the handbook does not answer it, say: "I'm not sure. Please ask Kavita."
Keep answers under 80 words; staff read them on the shop floor.
<handbook>
{sections}
</handbook>"""

The tutor’s {profile} is a small pinned record that persists between sessions: topics covered, what the learner found hard last time (“confuses proofing and fermentation”). It is the pinned-facts idea from the context engineering lesson, applied to learning. Module 15 covers memory for agents in depth.

Asked “A customer left a cake in a hot car for two hours and wants a replacement. What do we do?”, the assistant replied: “Offer a 20% voucher, not a free replacement; we can’t control storage after collection. Be kind and explain why. (Handbook 7.3)”. Asked about parking rules at the Aundh store, which the handbook does not cover, it said to ask Kavita.

Where it breaks

The simulator ran with all eight trainees in the first week of December. Here is what Arjun learned from the logs.

The magic words. By day three, two trainees had worked out that saying the customer’s name and a time always raised the meter. They started every reply with “Mr. Rao, by 6:50…” even when it made no sense. The fix: the character sheet now says points are only awarded when the step is genuinely relevant, and Kavita added scenarios where the “right” moves differ (a customer who wants to be heard before any solution).

Drift in long sessions. In sessions past twelve turns, Mr. Rao occasionally started offering solutions himself. Long transcripts dilute the character sheet. Keeping sessions short, re-sending the full sheet every turn and capping turns at ten fixed it.

Too realistic. One scenario designer wrote a customer who used insulting language, “to make it real”. It made one trainee cry. Realistic does not have to mean abusive; the character sheet’s “not abusive, never uses slurs” line is there on purpose, and scenarios are reviewed before use.

The flattering coach. Early debriefs praised everything (“Great job showing empathy!”). Models lean towards telling people what they want to hear, a tendency called sycophancy. A rubric with specific criteria, a requirement to quote the trainee’s words, and an instruction to name at least one improvement made the feedback honest.

The tutor that gave in. When Imran typed “just tell me” three times on a concept question, the tutor gave a full answer on the second ask, as designed, which was fine. But on a graded assignment question, copied from his coursework, it also helped. A tutor cannot know what is homework unless you tell it. The apprentices’ assignment questions are now listed in the tutor’s notes with “guide, never solve”.

Try it yourself

Write a character sheet for a second training scenario: a parent, Mrs. Fernandes, whose son had a mild allergic reaction after a Crumb birthday cake. She is frightened rather than angry and wants to know exactly what was in the cake.

1. Give her a meter called trust starting at 3, and at least four rules for what raises or lowers it.
2. Decide when the scene ends.
3. Play it twice: once responding the way you think is right, once by deflecting (“our kitchen follows all regulations”).
4. Run the coach on both transcripts.

Show the answer

A strong sheet might include:

• +2 for taking it seriously first (“Is your son okay right now?”) before anything else.
• +2 for a concrete step: “I’m getting the full ingredient list from Chef Nikhil and will send it to you within 30 minutes.”
• +1 for honesty about uncertainty (“I don’t want to guess; I’ll confirm with the kitchen”).
• -3 for defensiveness or quoting regulations.
• -2 for guessing about ingredients.
• Ends at trust 8, or if the trainee promises to send something specific and confirms the parent’s contact details.

The deflecting run should drop trust quickly, and the coach should quote “our kitchen follows all regulations” as the moment it went wrong. The best runs escalate to a real person (Chef Nikhil) rather than answering ingredient questions from memory: the same protocol the ticket router follows.

Common mistakes

• A mood instead of a character. “Be angry” collapses or never changes. Write a sheet: scenario, goals, hidden state, rules for change, limits.
• One call doing everything. Acting, coaching and fact-checking are separate jobs. Use separate calls.
• Tutors that hand over answers. Use a hint ladder for reasoning; give reference facts directly, from your materials.
• Ungrounded facts. Assistants and tutors must answer from text you supply and say “I don’t know” otherwise.
• Unchecked realism. Difficult is fine; abusive is not. Review scenarios.
• Hiding that it is an AI. Be honest whenever someone asks.

Recap

• Role-play bots, tutors and assistants are persona-driven chatbots; the persona is a full character sheet, not an adjective.
• Hidden state in structured output (a calm meter, a trust meter) makes a character react realistically and makes sessions measurable.
• Split the jobs: the actor stays in character, a separate coach grades with a rubric, and assistants answer only from grounded text.
• Learning advisors teach with a hint ladder and a learner profile that persists between sessions.
• Watch for gaming, drift, sycophancy and over-helpfulness, and always disclose that it is an AI.

By mid-December, all eight seasonal staff had done at least six simulator sessions. Kavita reviewed every debrief and sat with each person once, for twenty minutes instead of an afternoon. In the first week on the floor, Meera noticed something: nobody said “sorry for any inconvenience” any more.

“How does the fake Mr. Rao work?” she asked.

“He’s like a flight simulator for difficult customers,” Arjun said. “The AI plays a customer with a full backstory and a hidden calm meter. Good moves, like a specific apology or a real time, raise it; blaming or ‘please calm down’ drops it, so the trainee feels the effect of what they say. When it’s over, a different AI, using Kavita’s checklist, gives feedback with quotes. The apprentice tutor works the other way round: it asks questions instead of giving answers. And the staff helper only answers from the handbook, with the section number, or tells them to ask Kavita.”

Meera thought about that. “You’ve built a lot of little AI tools this month. Summaries, translations, routing, SQL, these bots. Are they all held together with tape?”

Arjun winced. They were, a bit: scripts copied between folders, each with its own way of reading the API key and its own idea of what to do when the API timed out. Priya, overhearing, said, “Then that’s your next job. Turn them into one proper toolkit.”

Go deeper

• Anthropic documentation, including system prompts and role prompting: https://docs.anthropic.com
• OpenAI documentation: https://platform.openai.com/docs
• Building effective agents: https://www.anthropic.com/research/building-effective-agents