What is a large language model?

It is a Monday at Crumb & Co., and the support inbox has 340 unread emails. Priya has sorted a sample of 100: 41 ask “where is my order?”, 22 ask about allergens or eggless options, 17 want a refund for something squashed, melted or late, and the rest are a mix of compliments, complaints and one man asking whether the plum cake can be shipped to Toronto.

Two people answer all of this by hand. On a normal day each reply takes about four minutes. In Diwali week the backlog hits four days, and the one-star reviews start.

Meera calls a meeting. “Everyone keeps telling me AI can answer emails. Can it? And if it can, can I trust it?”

Arjun has been waiting for this moment. He opens a chatbot on his laptop, pastes in a real email, and turns the screen around.

Arjun’s demo goes well, then very badly

The email he pastes is from Mrs Kulkarni:

“Hi, I ordered a Diwali hamper (order CR-20931) on Friday. It still hasn’t arrived. When will it come? Also, are the kaju katli in it made in the same kitchen as the peanut chikki? My son is allergic.”

He types: Write a reply to this customer. Four seconds later:

“Dear Mrs Kulkarni, thank you for reaching out, and I’m so sorry your Diwali hamper hasn’t arrived yet. Your order CR-20931 was dispatched yesterday from our Baner store and should reach you by Thursday. Regarding your allergy question, our kaju katli is prepared in a separate nut-free area, so it is safe for your son. Wishing you a very happy Diwali!”

The room goes quiet in two different ways. Arjun is quiet because it is beautiful: warm, specific, perfectly punctuated. Meera is quiet because she is reading it.

“We don’t have a Baner store,” she says. “And kaju katli is made of cashews. It is not nut-free. Nothing in our sweets kitchen is nut-free.”

Arjun stares at the screen. “But it sounded so sure.”

Priya, who has said nothing so far, picks up a marker. “It always sounds sure. Before we decide whether it can touch the inbox, let us work out what it actually did in those four seconds. Then this answer will stop being a mystery.”

The game you already play every day

Priya writes half a sentence on the whiteboard:

“Our Christmas plum cake is soaked in ___”

“Rum,” says Arjun. “Maybe brandy.”

“Why not concrete?”

“Because nobody soaks cake in concrete.”

“How do you know? Has anyone ever told you the rule ‘cake is never soaked in concrete’?”

Arjun thinks. “No. I’ve just never seen it. I’ve seen ‘soaked in rum’ a hundred times.”

“Exactly. You have read so much text that you have a feel for what usually comes next.” She draws an arrow from the blank. “A large language model, or LLM, is a program that has played this one game, guess what comes next, on more text than any human could read in a thousand lifetimes. That is the whole trick. Everything else is a consequence.”

Your phone’s keyboard plays a tiny version of the same game when it suggests the next word. An LLM is that idea scaled up by a factor of millions: it has read a large slice of the public internet, books, articles and code, and its guesses have become good enough to write emails, explain tax rules or fix a bug.

“So when I asked for a reply,” says Arjun slowly, “it was guessing the reply?”

“One small piece at a time. Let me show you.”

Following one reply, piece by piece

An LLM does not produce a whole reply at once. It produces one small chunk, then the next, then the next. These chunks are called tokens (pieces of text, usually a word or part of a word; the next lesson is all about them). For now, think of each token as roughly one word.

Priya writes a short question at the top of the board, Is the plum cake eggless?, and then traces what the model does. These probabilities are illustrative, but the mechanism is exactly this.

Step 1. The model reads everything so far: the question, plus a marker meaning “the assistant’s reply starts here”. It produces a score for every token it knows, and turns the scores into probabilities:

Candidate next token	Probability
“Our”	34%
“Unfortunately”	21%
“No”	18%
“Yes”	9%
“The”	8%
(about 200,000 others)	10% between them

It picks one. Say “Our”.

Step 2. The text is now Is the plum cake eggless? → Our. The model reads all of that again and scores again. Now " plum" is at 61%, " Christmas" at 14%, " traditional" at 9%. It picks " plum".

Step 3 onwards. Same again, each time with one more token in view:

Step	Text so far (reply only)	Top candidates	Picked
3	Our plum	" cake" 88%, " cakes" 7%	" cake"
4	Our plum cake	" contains" 41%, " is" 33%, " uses" 12%	" contains"
5	Our plum cake contains	" eggs" 79%, " egg" 9%	" eggs"
6	Our plum cake contains eggs	“,” 52%, “.” 38%	“,”
…	…	…	…
31	…eggless chocolate cake instead.	[end of reply] 93%	[end]

The loop stops when the model picks a special end of reply token, or when it hits a length limit you set.

Press play and step through it. Watch two things. First, the probabilities change at every step, because the text in view changes. Second, once a token is chosen, it is never revisited. The model cannot go back and edit “contains” into “is”. It can only keep going forward.

So where did the Baner store come from?

Now Priya rewinds to Mrs Kulkarni’s email. Somewhere in the middle of that reply, the model had written “Your order CR-20931 was dispatched yesterday from our” and needed the next token.

What does text like that usually continue with? Online, sentences like “dispatched from our ___” are followed by a place: a warehouse, a store, a city area. The model knows Crumb is in Pune, because Arjun’s email mentioned Diwali and the address had “Pune” in it. So its candidates look something like:

Candidate	Probability
" warehouse"	19%
" Baner"	11%
" Kothrud"	10%
" Pune"	9%
" FC"	7%
…	…

Every one of those is a perfectly plausible continuation. Baner and Kothrud are real Pune neighbourhoods with lots of shops. The model rolled the die, landed on " Baner", and carried on writing, building every later token on top of that choice. Nowhere in this loop is there a step that asks, “Is there actually a Crumb store in Baner?” There is no database, no lookup, no check. There is only what text usually looks like.

The kaju katli answer is the same story. Bakery replies about allergies very often contain reassuring phrases like “prepared in a separate nut-free area”. That phrase is likely text. It is not checked text.

“So it didn’t lie,” says Arjun. “It doesn’t know what lying is. It wrote the most normal-sounding sentence.”

“Yes. When an LLM produces fluent, confident text that is not true, we call it a hallucination. And now you can see why it is not a rare glitch. It is the same mechanism that produces the good answers, landing somewhere that happens to be false.”

Build a tiny language model in ten minutes

Arjun is not satisfied with probabilities on a whiteboard. “Can I see one of these things work?”

“You can build a very small one. No API key, no GPU, twenty lines of Python. It will be terrible. That is the point: you will see the mechanism with nothing hidden.”

Create a file called tiny_lm.py. First, the “training text”: five sentences about Crumb’s cakes.

# tiny_lm.py: a toy "language model" that predicts the next word
import random
from collections import Counter, defaultdict

text = """
our plum cake is soaked in rum .
our plum cake is rich and dark .
our chocolate cake is eggless .
our chocolate cake is soaked in syrup .
the plum cake is soaked in brandy .
"""
words = text.split()  # ['our', 'plum', 'cake', 'is', 'soaked', ...]

Next, “training”. For every word, count which word came right after it.

followers = defaultdict(Counter)
for current, nxt in zip(words, words[1:]):
    followers[current][nxt] += 1

print(followers["is"])
print(followers["in"])

zip(words, words[1:]) pairs each word with the one after it: ('our', 'plum'), ('plum', 'cake'), and so on. Run it with uv run tiny_lm.py (or python tiny_lm.py):

Counter({'soaked': 3, 'rich': 1, 'eggless': 1})
Counter({'rum': 1, 'syrup': 1, 'brandy': 1})

That table is the model. Its “knowledge” is nothing more than these counts.

Now turn counts into probabilities, the same kind of table Priya drew:

def next_word_probs(word):
    counts = followers[word]
    total = sum(counts.values())
    return {w: c / total for w, c in counts.items()}

print(next_word_probs("is"))

{'soaked': 0.6, 'rich': 0.2, 'eggless': 0.2}

Finally, the generation loop: look at the last word, roll a weighted die, append, repeat until a full stop.

def generate(start, max_words=12):
    out = [start]
    for _ in range(max_words):
        probs = next_word_probs(out[-1])
        if not probs:          # never seen this word: nothing to predict
            break
        choices, weights = zip(*probs.items())
        word = random.choices(choices, weights=weights)[0]  # weighted die roll
        out.append(word)
        if word == ".":
            break
    return " ".join(out)

for _ in range(5):
    print(generate("our"))

Run it a few times. You will see output like this (yours will differ, because of the die):

our plum cake is soaked in rum .
our chocolate cake is soaked in brandy .
our plum cake is eggless .
our chocolate cake is rich and dark .
our plum cake is soaked in syrup .

Look closely at these. “Our chocolate cake is soaked in brandy” never appeared in the training text. The model generalised: it stitched together pieces it had seen. Sometimes that is useful.

And then: “our plum cake is eggless .” The training text never said that. In fact, the plum cake is not eggless. But every step was locally sensible: “plum” is followed by “cake”, “cake” by “is”, and “is” is followed by “eggless” one time in five. You can even work out how often it happens. After “our”, “plum” has a 50% chance; after “is”, “eggless” has 20%; every other step is certain. So 0.5 × 0.2 = 0.1: one run in ten, your toy model confidently claims the plum cake is eggless.

You have just built a hallucination machine from five sentences. A real LLM is vastly better, but the hallucination about kaju katli came from the same place.

Where the toy ends and the real thing begins

Arjun is delighted with his twenty lines. Priya lets him enjoy it, then draws a two-column table.

	Your toy	A real LLM
What it looks at	Only the last word	Everything in the request, up to hundreds of thousands of tokens
Vocabulary	17 words	Around 100,000 to 200,000 tokens
How it scores the next token	Counts from a table	A neural network (a big mathematical function built from layers of simple calculations) with billions of adjustable numbers
Training text	5 sentences	Trillions of tokens. Meta’s open Llama 3 models, for example, were trained on about 15 trillion
What it “knows”	Which word follows which	Grammar, facts, styles, reasoning patterns, code, many languages

The biggest difference is the first row. Your toy only sees “is” and must guess. A real model sees “Our plum cake contains eggs, butter and dried fruit soaked in rum, so it is” and can tell that “not eggless” is coming. Looking at the whole context is what makes it seem to understand.

Where the knowledge lives

Inside a real model are billions of numbers called parameters. Think of them as tiny dials. The open Llama 3 family came in sizes of 8 billion, 70 billion and 405 billion parameters; the big commercial models from OpenAI, Google and Anthropic do not publish their sizes, but they are in the same league or larger.

Training is the process that sets those dials. A computer shows the model a stretch of real text with the next token hidden, the model guesses, and the guess is compared with the real token. Then every dial is nudged a tiny amount in whichever direction would have made the right token a little more likely. Repeat that trillions of times. No one ever types in “éclairs are made of choux pastry”. That fact ends up spread across millions of dials because it appeared in the training text thousands of times.

Three consequences follow, and every one of them showed up in Mrs Kulkarni’s reply:

• The knowledge has a date. The model knows only what was in its training text, which was collected up to some knowledge cutoff. This year’s Diwali hamper contents are not in there.
• The knowledge is fuzzy. Facts seen thousands of times are reliable. Rare facts, like which Pune neighbourhoods have a Crumb store, are blurred or simply absent, and the model fills the gap with something plausible.
• It has never seen your data. Crumb’s order database, the recipe archive and the store handbook were never in its training text. If you want it to use them, you must hand them over inside the request. That is a big part of what this course teaches you to do.

From text predictor to helpful assistant

There is one more step. A model trained only to predict internet text is called a base model, and it is a strange thing to talk to. Ask it “How do I get a refund from Crumb?” and it might continue with “How do I cancel an order? How do I change my delivery address?”, because on the internet, a question is often followed by more questions, as in an FAQ page.

So model makers train it further. First the big run on internet text, called pretraining. Then extra rounds on many thousands of example conversations: a question, then a good answer. Then people compare pairs of answers and the model is nudged towards the ones people preferred. The result is a chat model: an LLM tuned to answer helpfully, follow instructions and decline harmful requests.

flowchart LR
  A["Trillions of tokens of text"] --> B["Pretraining: predict the next token"]
  B --> C["Base model: continues any text"]
  C --> D["Extra training on good conversations and human ratings"]
  D --> E["Chat model you call through an API"]

Every model you will call in this course, GPT from OpenAI, Gemini from Google and Claude from Anthropic, is a chat model built roughly this way. Underneath, it is still predicting the next token. The extra training changed which next tokens it finds likely: helpful ones.

What this means for the inbox

Back to Meera’s question. Priya lists what the mental model predicts, because a good mental model predicts behaviour.

• It will sound confident whether or not it is right. It chooses likely text, not checked text. Tone tells you nothing about accuracy.
• It cannot know Crumb’s facts unless you give them to it. Store list, order status, allergen sheet: none of these are in its dials. Give it the facts in the request, and it uses them very well.
• It can answer the same question differently twice. The die roll at each step means two runs can diverge.
• Every token costs time and money. One trip round the loop per token. Long replies are slower and pricier.
• It is genuinely good at language. Tone, empathy, summarising a rambling email, drafting in Marathi or English: this is where it shines.

“So,” says Meera, “it is not a database. It is a writer.”

“A very fast, very well-read writer who has never been inside our bakery,” says Priya. “If we hand it the facts and check the risky bits, it can draft most of those 340 replies. If we let it make up facts, it will tell a mother that cashews are nut-free.”

Try it yourself

Part 1: extend your toy model. Add these two lines to the training text in tiny_lm.py:

our kaju katli is made with cashews .
our peanut chikki is made with peanuts .

Run generate("our") twenty times. Can it now produce a false sentence about kaju katli? Work out how often.

Show the answer

Yes. After “is”, the toy has now also seen “made”. After “our”, it can pick “kaju” or “peanut”. So it can generate “our kaju katli is soaked in rum .” or “our peanut chikki is eggless .”, both stitched from real pieces, both false or absurd. The more varied the training text, the more fluent the nonsense can become. Real models are far better at keeping track of context, but the risk never drops to zero.

Part 2: poke a real chatbot. Open any free chatbot (ChatGPT, Gemini or Claude) and try:

1. Complete with one word: Our Christmas plum cake is soaked in. Regenerate the answer three times and note each one.
2. What is the phone number of the Crumb & Co. bakery on FC Road, Pune?
3. What day of the week is it today?

Show what you should expect

1. Mostly “rum” or “brandy”, with occasional variety. That is the weighted die roll.
2. A careful model says it does not know. A less careful one invents a plausible number. If you get a number, you have seen a hallucination: it has no way to know.
3. The model itself has no clock. Chat apps often slip today’s date into the request behind the scenes, so you may get the right answer. That is the app helping the model by giving it a fact, which is exactly the technique you will use with Crumb’s data.

Common mistakes

• “It looks things up.” A bare LLM does not search a database or the web. It generates likely text. Apps can add search or database lookups around it; you will build those yourself in Module 06 (tools) and Module 09 (retrieval).
• “It understands like a person does.” It is remarkably good with language and can reason through many problems, but it has no experience of your bakery and no built-in sense of what is true. Treat its output as a strong draft.
• “Confident means correct.” The tone comes from the same next-token process whether the content is right or wrong. Check prices, policies, dates and anything about health or allergies.
• “It learns from my chats.” The dials were set during training. Talking to it does not change them. If a chatbot seems to remember you, the app is resending earlier messages; you will see how in this module.
• “Hallucinations are rare bugs that will be patched.” They are a side effect of the core mechanism. Newer models hallucinate less, but you design around the risk rather than hoping it disappears.

Recap

• An LLM is a next-token predictor: it scores every possible next token, picks one, adds it, and repeats.
• It never plans the whole answer or goes back to edit; each token is built on the ones before.
• Its knowledge lives in billions of parameters set during training; it has a cutoff date and does not include your private data.
• Chat models are base models given extra training on good conversations and human ratings.
• Hallucinations come from the same mechanism as good answers: plausible text that nothing checked.

That evening Arjun explains it to Meera in the car park. “It’s autocomplete that has read most of the internet. It writes beautifully, but it doesn’t check anything. So we give it our real facts, and we never let it invent the risky ones.”

Meera nods. “Fine. Then tell me what it costs. Your demo said something about tokens. Is a token a word? Because I am not paying per word for that Baner store.”

Go deeper

• Hugging Face LLM course, a gentle hands-on tour of how these models work: https://huggingface.co/learn/llm-course
• How chat models are trained to follow instructions (the InstructGPT paper): https://arxiv.org/abs/2203.02155