Chain-of-thought, self-consistency, Auto-CoT and personas

Meera’s Diwali question sounded simple when she said it out loud. “Hampers over ₹2,000 get 15% off this year. Loyalty members already get 10% off everything. The discounts don’t stack: each item gets whichever discount is bigger. A loyalty member buys a ₹2,400 hamper and a ₹300 box of nankhatai. What do they pay?”

There was a reason she was asking. Arjun was building a pricing helper for the WhatsApp ordering line, so customers could ask “how much for this?” and get an instant answer. Crumb expected around 1,800 hamper orders over the festival.

Arjun typed the question into a small, fast model and asked for the answer only. It replied, in under a second: ₹2,295.

Meera had already done the maths on a napkin. “I get ₹2,310.”

Fifteen rupees. Arjun shrugged. Priya did not. “Fifteen rupees on 1,800 orders is ₹27,000 of discount nobody approved. And that’s if it’s always wrong by the same amount. Let’s see how often it’s wrong at all.”

Arjun asks the model to be more careful

Arjun ran the answer-only prompt ten times. It said ₹2,310 four times, ₹2,295 five times and ₹2,160 once. Four out of ten.

His instinct was to ask the model to try harder:

Hampers over ₹2,000 get 15% off. Loyalty members get 10% off everything.
Discounts don't stack: each item gets the bigger one. A loyalty member buys
a ₹2,400 hamper and a ₹300 nankhatai box. What do they pay?
Be very careful and double-check your answer. Reply with the amount only.

Ten more runs. Five out of ten. Barely different.

Long division in your head

Priya wrote 4,738 × 26 on the whiteboard. “Answer. Right now. No paper.”

Arjun stared. “About… a hundred and twenty thousand?”

“Now do it on paper.”

He wrote it out: 4,738 × 20 = 94,760. 4,738 × 6 = 28,428. Add them: 123,188.

“Same brain,” Priya said. “Very different accuracy. Why?”

“Because on paper I can write down the middle bits and come back to them. In my head I have to juggle everything at once.”

“The model has the same problem, but worse. Where does a language model keep its working while it solves a problem?”

Arjun thought about how generation works: one token at a time, each one chosen by looking back at everything already written. “It doesn’t have anywhere. Except… the text itself.”

“Exactly. Every token gets roughly the same small amount of computation. When you force the answer to come first, the model has to do all the arithmetic in the few steps it takes to produce ‘₹2,3’. When you let it write the steps, each step becomes text it can read back. The written steps are its scratch paper.”

This is called chain-of-thought prompting, often shortened to CoT: asking the model to produce its intermediate reasoning before its final answer. A 2022 Google paper showed that adding worked reasoning to examples made large models dramatically better at maths and logic word problems. A follow-up paper the same year found that adding only “Let’s think step by step” helped too, a version called zero-shot CoT because it needs no examples.

Following the discount question, one step at a time

Here is the chain-of-thought version Priya and Arjun wrote. It asks for steps, gives them a structure that matches the business rule, and separates the working from the answer with tags so code can read the answer cleanly:

Hampers over ₹2,000 get 15% off. Loyalty members get 10% off everything.
Discounts don't stack: each item gets the bigger one. A loyalty member buys
a ₹2,400 hamper and a ₹300 nankhatai box. What do they pay?

Work through it step by step inside <thinking> tags:
1. For each item, list every discount that could apply and what it saves.
2. Pick the larger saving for each item.
3. Work out each item's final price, then the total.
Then give only the final amount inside <answer> tags, like <answer>₹1,234</answer>.

And here is what came back, word for word:

<thinking>
Hamper, ₹2,400:
- Hamper discount applies (over ₹2,000): 15% of 2,400 = ₹360 off.
- Loyalty discount applies: 10% of 2,400 = ₹240 off.
- Bigger saving: ₹360. Hamper price = 2,400 - 360 = ₹2,040.
Nankhatai, ₹300:
- Hamper discount does not apply (not a hamper).
- Loyalty discount: 10% of 300 = ₹30 off.
- Nankhatai price = 300 - 30 = ₹270.
Total = 2,040 + 270 = ₹2,310.
</thinking>
<answer>₹2,310</answer>

Now look back at the wrong answers and you can reconstruct where they came from:

• ₹2,295 is 15% off everything: (2,400 + 300) × 0.85. The model applied the hamper discount to the nankhatai too.
• ₹2,160 is the hamper at loyalty price (2,400 × 0.9) with the nankhatai forgotten entirely.

Both are exactly the errors a person makes when juggling the problem in their head. The step-by-step structure forces each item and each discount to be considered on its own line.

The numbers. Arjun ran the CoT prompt ten times: nine correct. Here is what that cost, at illustrative small-model prices of $0.25permillioninputtokensand$2 per million output tokens:

Version	Output tokens	Time	Correct of 10	Cost per question
Answer only	about 8	0.6 s	4	about ₹0.003
Chain-of-thought	about 160	2.4 s	9	about ₹0.03

CoT is roughly ten times the cost and four times slower, and both are still tiny in rupees. For a customer waiting on WhatsApp, 2.4 seconds is fine. For tagging a million reviews overnight, ten times the output would matter more.

Self-consistency: ask five times, take a vote

Nine out of ten is good, not great. The one wrong run had taken a wrong turn halfway through. Priya suggested a technique called self-consistency: ask the same question several times, let each attempt reason independently, and take the most common final answer.

Why does voting help? Think of five people doing Meera’s sum independently on napkins. There are several correct ways to set it out, and they all land on ₹2,310. The mistakes scatter: one person gets ₹2,295, another ₹2,160. Correct reasoning paths converge; wrong ones rarely agree with each other. A majority vote filters out one-off slips.

Arjun’s five runs came back: ₹2,310, ₹2,310, ₹2,295, ₹2,310, ₹2,310. Majority: ₹2,310, four votes to one. Over fifty trials of five votes each, the majority was right every time.

The cost is five times the CoT cost, about ₹0.15 per question. That is worth it for a price quote. It is silly for “what time does the Baner store open?”. And it only works when answers are short and comparable, like a number, a label or yes/no. You cannot take a majority vote across five different email drafts.

Auto-CoT: let the model write its own worked examples

In the last lesson, examples taught the model Crumb’s categories. Examples that include reasoning, not only answers, are even more powerful for multi-step problems: few-shot CoT. But Crumb’s pricing questions come in many shapes (discounts, delivery fees, bulk orders, cancellation charges) and writing a careful worked example for each by hand is tedious.

Auto-CoT (automatic chain-of-thought), from a 2022 paper, automates it:

1. Collect a pile of real questions and group them by similarity, giving clusters like “discount questions”, “delivery-fee questions”, “bulk-order questions”.
2. Pick one representative question from each cluster.
3. Ask the model to solve each with “Let’s think step by step”, producing a worked example.
4. Keep the clean, correct ones and use them as few-shot CoT examples for new questions.

The clustering step is the clever part. If all your worked examples were discount questions, the model would try to reason about delivery fees like a discount calculator. Diversity across clusters avoids that. In practice, most teams do a lighter version: generate worked examples with a strong model, have a person check each one, and keep the best in an example bank.

Personas: who is answering?

While they were at it, Meera asked Arjun to check the label on the new walnut brownie. He asked the model to “review this label”, and got polite comments about wording. Priya changed one line at the top:

You are a food-safety officer who audits bakery labels against Indian
allergen labelling rules. You are thorough and blunt.

Review this label for our walnut brownie:
"Contains: flour, sugar, butter, cocoa."
List every problem, most serious first.

This time, the first line of the reply was: “Walnuts, a tree nut and major allergen, are not declared.” Then flour (gluten) and butter (dairy) not being marked as allergens.

That is persona prompting: asking the model to answer as a particular expert or character. A persona shifts what the model pays attention to, which vocabulary it uses and which problems it treats as important.

Be honest about what personas cannot do. They change focus and style, not knowledge. “You are a world-class chartered accountant” will not teach a model a GST rule it never learned. It may only make a wrong answer sound more authoritative. Use a persona to set the angle, and put the facts the persona needs in the context.

Opening the hood: reasoning models think before they speak

Since 2024, providers have released reasoning models: models trained, mostly through reinforcement learning on problems with checkable answers, to produce a long internal chain of thought before replying. OpenAI’s o-series and GPT-5 thinking modes, Claude’s extended thinking and Gemini’s thinking models all work this way. The reasoning happens automatically, in tokens you are billed for but often do not see in full.

When Arjun sent Meera’s original, answer-only question to a reasoning model, it answered ₹2,310 ten times out of ten, after a pause of about four seconds. It had done the chain-of-thought on its own.

That changes the playbook:

• “Think step by step” is mostly redundant with reasoning models. They already do. It rarely helps and occasionally makes things worse.
• Do not script their reasoning in tiny steps. Give them the problem, the rules and what a good answer looks like. They plan better than a rigid list you write for them.
• Control depth with an effort setting. Instead of asking for steps, you tell the API how hard to think. On Claude:

import anthropic
client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=4000,
    output_config={"effort": "high"},   # "low" for routine tasks, "high" for tricky multi-step ones
    messages=[{"role": "user", "content": "Hampers over ₹2,000 get 15% off... What do they pay?"}],
)
print(next(b.text for b in response.content if b.type == "text"))

• Thinking costs tokens and time. Higher effort means more hidden reasoning, which you pay for, and slower replies. Low effort for classification and short rewrites; high effort for maths, planning and code.
• Explicit CoT still matters for small and local models. The cheap open models you will run on your own machine in Module 20 often have no built-in reasoning. For them, “show your working in <thinking> tags, then answer in <answer> tags” is still one of the best tools you have.

The real code, step by step

Step 1: CoT with tags, and pull out only the answer.

import os
import re
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI()
MODEL = os.getenv("OPENAI_MODEL", "gpt-5-mini")  # model names change; see the models page

COT_PROMPT = """{question}

Work through it step by step inside <thinking> tags, then give only the
final amount inside <answer> tags, like <answer>₹1,234</answer>."""

def ask_with_cot(question: str) -> str | None:
    resp = client.chat.completions.create(
        model=MODEL, messages=[{"role": "user", "content": COT_PROMPT.format(question=question)}])
    text = resp.choices[0].message.content
    match = re.search(r"<answer>(.*?)</answer>", text, re.S)   # ignore the thinking
    return match.group(1).strip() if match else None

Your code throws the <thinking> part away (or logs it for debugging) and returns only what is inside <answer>. The customer never sees the working.

Step 2: self-consistency with a confidence check.

from collections import Counter

def vote(question: str, n: int = 5, min_agree: int = 4) -> str | None:
    answers = [a for a in (ask_with_cot(question) for _ in range(n)) if a]
    if not answers:
        return None
    best, count = Counter(answers).most_common(1)[0]
    print(f"votes: {answers}")
    return best if count >= min_agree else None   # split vote: refuse to guess

You should see something like votes: ['₹2,310', '₹2,310', '₹2,295', '₹2,310', '₹2,310'] and a returned ₹2,310.

Self-consistency needs variety between attempts. With older models you raised the temperature (the setting that controls how random each token choice is) to get it. Many newer models fix temperature, but separate calls still vary enough to vote.

Step 3: the fix Priya actually shipped. After all that, Priya made a suggestion that surprised Arjun. “For prices, the model shouldn’t do the maths at all.”

from pydantic import BaseModel

class Item(BaseModel):
    name: str
    price: int
    is_hamper: bool

def price_order(items: list[Item], loyalty: bool) -> int:
    """Crumb's discount rule, in code: each item gets its single best discount."""
    total = 0
    for item in items:
        discounts = [0.0]
        if item.is_hamper and item.price > 2000:
            discounts.append(0.15)
        if loyalty:
            discounts.append(0.10)
        total += round(item.price * (1 - max(discounts)))
    return total

print(price_order([Item(name="Diwali hamper", price=2400, is_hamper=True),
                   Item(name="Nankhatai box", price=300, is_hamper=False)], loyalty=True))

This prints 2310, every time, in microseconds, for free. The model’s job became reading the customer’s WhatsApp message and extracting the items as structured output (the previous lessons). Code did the arithmetic. In Module 06 you will learn tool calling, where the model can ask your code to run a function like price_order in the middle of a conversation.

Where it breaks

Reasoning that looks right but is not. A model’s written reasoning is not a guaranteed window into how it reached the answer. Researchers have found cases where the steps and the final answer disagree, or where the steps are a tidy story written after the fact. Fluent working is not proof. Check the answer, not the prose.

Leaking the working. In one early test, the WhatsApp helper forgot to strip the tags and sent a customer the full <thinking> block, complete with “the loyalty discount does not apply to…”. Always extract the answer in code.

CoT where it adds nothing. “What time does the Aundh store open?” does not need steps. Chain-of-thought on lookups and simple classification only adds tokens and latency.

Reasoning models overthinking. At high effort, a reasoning model once spent twelve seconds deliberating over whether “cake was gr8” was positive. Match the effort to the task.

Try it yourself

Ask a small, non-reasoning model this question twice: once with “answer only”, once with a CoT prompt and <answer> tags. Run each version five times.

“Crumb’s Baner store opens at 7 a.m. Bread arrives 90 minutes before opening. Shelving it takes one person 40 minutes, and two people working together take half as long. Sana starts shelving alone the moment the bread arrives. Ravi arrives 10 minutes after the bread and joins her. When is the shelving finished?”

Show the answer

The bread arrives at 5:30 a.m. In her first 10 minutes alone, Sana does 10/40 = 1/4 of the job. Together they work twice as fast, so the remaining 3/4 takes (3/4 × 40) / 2 = 15 minutes. Finished at 5:30 + 10 + 15 = 5:55 a.m.

Answer-only runs on small models often say 5:50 (treating it all as a 20-minute two-person job) or 6:10 (ignoring Ravi). The CoT version usually gets 5:55, and a five-way vote almost always does. If a reasoning model gets it right without any CoT prompt, that is the reasoning model doing CoT internally.

Common mistakes

• Parsing the whole reply as the answer. Once the model shows its working, extract only the final answer, with tags.
• CoT on everything. Use it for multi-step problems. Skip it for lookups and simple labels.
• Trusting confident reasoning. Well-written steps can hide a wrong step. Verify numbers in code.
• Treating personas as expertise. A persona sets the angle; the facts must come from you.
• Micromanaging reasoning models. Describe the goal and constraints, set an effort level, and evaluate the result.
• Voting on open-ended text. Self-consistency needs short, comparable answers.

Recap

• A model’s only working memory is the text it writes, so chain-of-thought gives it room to solve multi-step problems.
• Separate the reasoning and the answer with tags, and extract only the answer in code.
• Self-consistency samples several reasoning paths and takes a majority vote; disagreement doubles as a low-confidence signal.
• Auto-CoT builds diverse worked examples by clustering questions and solving one per cluster.
• Personas change focus and style, not knowledge.
• Reasoning models think internally: skip “think step by step”, set an effort level, and still check anything that matters in code.

Meera came back that evening. “So which number do customers see?”

“₹2,310, every time,” Arjun said. “Here’s what I learned. When I asked for the answer straight away, the AI was like someone doing long sums in their head: right about half the time. When I let it write out each step, it got nine in ten. The newer ‘thinking’ models do those steps by themselves. But for prices, Priya made me go one better: the AI reads the order, and plain code does the maths. Code doesn’t have off days.”

“So the AI is the one who listens, and the calculator is the one who counts.”

“Exactly that.”

Pleased, Arjun turned to his next experiment: running a free open model on the office’s spare GPU machine to tag reviews without paying per call. He sent it a simple prompt. It replied with what looked like a second review, then a third, then a line starting “Assistant:”, and kept going until it ran out of tokens.

Go deeper

• Chain-of-thought prompting paper: https://arxiv.org/abs/2201.11903
• Self-consistency paper: https://arxiv.org/abs/2203.11171
• Anthropic documentation, effort and extended thinking: https://docs.anthropic.com
• OpenAI documentation, reasoning models: https://platform.openai.com/docs