The 30 words you will hear everywhere

On Wednesday, Arjun sits in on his first AI architecture meeting. Within five minutes he has written down “RAG”, “context window”, “embeddings”, “MCP”, “evals” and “temperature”, and understood none of them. Meera, next to him, has stopped writing altogether.

Then it gets worse. After the meeting, Arjun proudly demos the prototype chat widget he built last night for Crumb’s website. Meera types the kind of message that arrives every evening in festival season:

“Is the Diwali hamper safe for my son? He’s allergic to cashews. And can it reach Baner by Friday?”

The reply appears:

“Great choice! Our Diwali hamper is a festive favourite. Most of our sweets are nut-free, so it should be fine for your son. We deliver across Pune in 2 to 3 days, so Friday works!”

Meera goes pale. The hamper contains cashew barfi. It says so on the ingredient card in the recipe archive. “If a customer had read that,” she says quietly, “a child could have ended up in hospital.”

Arjun’s prototype sent the customer’s message straight to a language model and printed whatever came back. The model has never seen Crumb’s recipe archive. It wrote what a friendly bakery assistant usually says.

Priya closes the laptop. “That meeting was unfair to you, and this bug is the result. Everyone talks in shorthand. So let us do this properly. We will follow this exact question from Meera’s keyboard to a correct answer, and I will name every part as we pass it. By the end you will know about thirty words, and more importantly, where each one lives.”

A map before the trip

Priya starts with an analogy. “Imagine Crumb hires a brilliant new assistant who works from home, in another city. They have read every cookbook ever printed, but they have never been inside our bakery. You can only reach them by sending a package. If you send them just the customer’s question, what do they do?”

“Guess,” says Arjun. “Politely. Like the bot did.”

“So what goes in the package?”

Arjun thinks. “The hamper’s ingredient card. Our allergy policy. Some way to check the delivery schedule. And instructions: be careful about allergies.”

“Good. Every word in that meeting is a name for one of those things, or for how the package travels, or for what happens inside the assistant’s head. Here is the whole trip:”

sequenceDiagram
    participant C as Customer
    participant A as Crumb app
    participant V as Vector database
    participant M as LLM via API
    participant O as Order system
    C->>A: Is the hamper cashew-free? Baner by Friday?
    A->>V: Find passages about this hamper and allergies
    V-->>A: Ingredient card, allergy policy
    A->>M: System prompt + passages + question
    M-->>A: Please run check_delivery(Baner, Friday)
    A->>O: check_delivery(Baner, Friday)
    O-->>A: Thursday slot available
    A->>M: Here is the tool result
    M-->>A: Streamed answer, token by token
    A-->>C: Contains cashews. Arrives Thursday.

Do not try to read every arrow yet. Come back at the end and it will read like a story. Each stop below has the real values for Meera’s question, then the words that name what happened. For each word: one plain sentence, one everyday comparison, and the module that teaches it properly.

Stop 1: The message leaves the browser

Meera pressed Send. Crumb’s web server now holds her message. The model does not live on Crumb’s server; it runs in a provider’s data centre. So Crumb’s code must send a request across the internet, signed so the provider knows who is asking and who to bill.

• API. An API (application programming interface) is a published doorway that lets one program ask another to do something, in an agreed format. Like the counter at a bakery: you do not walk into the kitchen; you order at the counter in a set way and get your order back.
• API key. An API key is a long secret string that identifies your account when your code calls an API. Crumb’s starts with sk-ant- and is about a hundred characters long. Like the key to the shop’s back door: anyone holding it gets in, and whatever they take goes on your tab.
• SDK. An SDK (software development kit) is a ready-made library that wraps an API so you can call it in a few lines of your own language. Like ordering through a waiter who already speaks the kitchen’s language, instead of shouting into the kitchen yourself.

Crumb’s Python code uses Anthropic’s SDK, which sends the request to Anthropic’s API, signed with Crumb’s API key. Learn it properly in Module 01: LLM Mental Model and API Setup, and see how web requests work in the Developer Toolkit.

Stop 2: Finding the right facts

This is the stop Arjun’s prototype skipped, and it is why the bot lied. Before asking the model anything, the app must find the hamper’s ingredient card among Crumb’s thousands of pages of documents.

Keyword search will not do: the card says “kaju barfi” and “tree nuts”, while Meera typed “cashews”. The app needs to search by meaning. Here is what happens, with real values:

1. Months ago, Crumb’s documents were cut into about 4,000 passages of 150 to 300 words each.
2. Each passage was turned into a list of 1,536 numbers capturing its meaning, and stored.
3. Now Meera’s question is turned into 1,536 numbers too. The first few look something like [0.021, -0.043, 0.067, 0.009, ...].
4. The app asks: which stored passages have numbers closest to the question’s? It gets back three, with similarity scores (1.0 would be identical meaning):
   • 0.82: “Diwali Hamper 2026: kaju barfi (cashew), besan ladoo, … Contains tree nuts, milk, wheat.”
   • 0.79: “Allergy policy: never describe a product as safe for an allergy unless its card lists it as free from that allergen. Offer a human.”
   • 0.44: “Delivery areas: Baner, Aundh, Wakad …” (weaker match, kept anyway)

The words for what just happened:

• Chunking. Chunking means cutting long documents into small passages (step 1) so you can fetch only the relevant bit. Like slicing a cake so you can serve one piece without carrying the whole cake.
• Embedding. An embedding is a list of numbers that captures what a piece of text means, produced by a special embedding model; similar meanings get similar numbers. Like a GPS coordinate for meaning: “cashew allergy” and “contains tree nuts” land close together; “delivery van” lands far away.
• Vector. A vector is an ordered list of numbers, like [0.021, -0.043, 0.067, ...]. An embedding is a vector, here with 1,536 numbers. Like a recipe written only as quantities in a fixed order: flour, sugar, butter, eggs.
• Vector database. A vector database stores huge numbers of vectors and quickly finds the ones closest to a given vector (step 4). Like a librarian who shelves books by topic so well she can hand you the five most related books in a second.
• Semantic search. Semantic search means searching by meaning rather than exact words: it found “kaju barfi” though Meera typed “cashews”. Like asking a chemist for “something for a sore throat” and being handed honey lozenges, though you never said “honey”.
• RAG. RAG (retrieval-augmented generation) is the whole pattern: retrieve relevant passages first, then hand them to the model with the question, so it answers from real facts. Like an open-book exam: the student still writes the answer, but from the textbook in front of them.

Learn it properly in Module 07: Embeddings and Semantic Search, Module 08: Vector Databases and Module 09: RAG.

Show the answer

1. It may not fit. The model can only read a limited amount of text at once (the context window, coming up at Stop 3). Thousands of pages can exceed it.
2. It is slow and expensive. Providers charge per piece of text read. Sending 300,000 words with every question would cost roughly a hundred times more than sending three relevant passages, and take longer.
3. It can hurt accuracy. Models do better when the relevant facts are not buried in a mountain of irrelevant ones.

RAG exists because “send everything” breaks on size, cost and quality. Module 09 measures exactly this trade-off.

Stop 3: Packing the request

Now the app assembles the package. It contains three parts: standing instructions from Crumb, the three retrieved passages, and Meera’s question. Here is roughly what is sent, written with the SDK:

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=500,
    system=(
        "You are Crumb & Co.'s support assistant. Answer only from the passages "
        "provided. Never call a product safe for an allergy unless its card says "
        "it is free from that allergen. Use check_delivery for delivery questions."
    ),
    tools=[check_delivery_tool],  # described at Stop 5
    messages=[{"role": "user", "content": passages + "\n\nCustomer: " + question}],
)

The whole package comes to about 1,000 tokens.

• Prompt. The prompt is all the text you send the model as input: instructions, facts and the question. Like the brief you give a new baker before a shift: the clearer it is, the better the result.
• System prompt. The system prompt is the developer’s standing instructions (the system= text above), which shape the model’s behaviour for the whole conversation and which the customer never sees. Like the house rules pinned up in the kitchen.
• Token. A token is the small chunk of text a model reads and writes: often a whole short word, sometimes part of a longer one. “Diwali” might be split into “Div” and “ali”; “the” is one token. In English a token averages about three-quarters of a word, so 1,000 tokens is roughly 750 words. Providers bill per token, in and out. A tokenizer is the tool that does the splitting. Like Lego bricks: every sentence is built from a fixed set of brick shapes.
• Context window. The context window is the maximum number of tokens a model can consider at once, counting the prompt and its answer together. Modern models range from tens of thousands to over a million tokens. Like the size of a worktable: everything the baker needs must fit on it, and anything that falls off is forgotten.

Learn it properly in Module 01, Module 02: Conversations and Your First Chatbot and Module 03: Prompt Engineering.

Stop 4: Inside the model

The request reaches the provider’s data centre, where the assistant’s “head” is. You met some of these words in the last lesson; here they are in context.

• Model. A model is a function with a huge number of adjustable knobs, trained so its outputs are useful. Like an experienced baker: years of practice are in their hands, but they still need your order to know what to make.
• Parameters. Parameters (or weights) are the knob values inside a model, learned during training; large models have billions. Like the exact quantities in a recipe.
• Training. Training is the slow, expensive process of setting the parameters by showing the model enormous numbers of examples. Like years of apprenticeship. The provider did it months ago, not now.
• Inference. Inference is running the trained model to get an answer, with its knobs frozen. Every API call is inference. Like the experienced baker making today’s order: no learning, just doing.
• LLM. An LLM (large language model) is a model trained on vast amounts of text to predict the next token; that one skill lets it answer, summarise, translate and write code. Claude, GPT and Gemini are LLMs. Like an autocomplete that has read a library the size of a city.
• Transformer. The transformer is the neural network design almost every modern LLM uses, introduced in 2017. Like the standard floor plan most modern bakeries copy because it works so well.
• Attention. Attention is the mechanism inside a transformer that lets each token look at every other token and weigh which ones matter for understanding it. In Meera’s message it links “He’s” back to “my son”, and “it” to “the Diwali hamper”. Like a reader’s highlighter, marking which earlier words to keep in mind.
• GPU. A GPU (graphics processing unit) is a chip that does thousands of simple calculations at once, exactly what running a model needs. Like a kitchen with a thousand small ovens instead of one big one.
• Open weights. A model has open weights when its creator publishes the trained parameters so anyone can download and run it, such as Meta’s Llama, Google’s Gemma or Mistral’s models. Claude and GPT’s main models are closed: you reach them only through an API. Like a bakery that publishes its full recipe, versus one where you can only buy at the counter.

The LLM reads the ~1,000 tokens and starts predicting its answer, one token at a time. It can answer the allergy half from the ingredient card. But nothing in the package says whether Friday delivery to Baner is possible.

Learn it properly in Module 01 for how LLMs behave, Module 20: Local LLM Deployment for running open-weights models, and Module 30: Transformers from Scratch for transformers and attention.

Stop 5: Doing, not just saying

The model cannot see Crumb’s delivery schedule. But the package included a description of a function it is allowed to request. So instead of an answer, its first reply is a structured request:

{
  "type": "tool_use",
  "id": "toolu_01A9",
  "name": "check_delivery",
  "input": {"area": "Baner", "deadline": "Friday"}
}

Crumb’s code sees this, runs its own check_delivery function against the order system (about 0.1 seconds), and gets {"earliest_slot": "Thursday 4-6 pm"}. It sends that result back to the model in a second request, and now the model can finish.

• Tool calling. Tool calling (or function calling) is when the model, instead of answering, replies with a structured request for your code to run a named function with specific inputs, and then uses the result. The model never runs anything itself; your code decides whether to. Like a waiter who cannot cook but can send a ticket to the kitchen and bring back what comes out.
• Agent. An agent is a model running in a loop: decide the next step, call a tool, look at the result, repeat until the task is done. Like a personal assistant you hand a goal (“sort out this customer’s order”) rather than step-by-step instructions. One tool call is not much of an agent. If the model had also checked stock, reserved the slot and emailed a confirmation, choosing each step itself, it would be.
• MCP. MCP (Model Context Protocol) is an open standard, introduced by Anthropic in 2024, for connecting AI apps to tools and data sources in one common way. Build a tool once as an MCP server and many AI apps can use it. Like USB-C for AI tools: one plug shape, many devices.

Learn it properly in Module 06: Function Calling, Module 13: AI Agents and Module 18: MCP and Agent SDK.

Stop 6: The answer comes home

With the tool result, the model writes its reply, and Crumb’s app sends each piece to Meera’s screen the moment it arrives:

“I’m sorry, the Diwali hamper contains kaju barfi (cashew), so it isn’t safe for someone with a cashew allergy. Our Classic Mithai Box is made without tree nuts; would you like its full ingredient list? Either box can reach Baner on Thursday between 4 and 6 pm.”

Here is the timing for the whole trip: retrieval took 0.15 seconds, the first model call 1.2 seconds, the tool 0.1 seconds, and the second call began producing words after 0.8 seconds. So the first words appeared about 2.3 seconds after Send, and the reply finished at about 4 seconds. The two model calls together used about 2,150 input tokens and 150 output tokens. At a typical mid-range price ($3permillioninputtokens,$15 per million output; prices vary by model and change often), that is under one US cent, roughly 75 paise.

• Temperature. Temperature is a setting that controls how adventurous the model is when picking each next token: low means it nearly always takes the most likely one, high means it takes more chances. Some of the newest models manage this themselves and do not expose the setting. Like a baker who either follows the recipe to the gram or improvises with the spices.
• Streaming. Streaming means sending the answer to the user piece by piece as it is generated, rather than waiting for the whole thing. Like serving each dish as it leaves the kitchen instead of holding every plate until the whole order is ready.
• Latency. Latency is the waiting time between asking and getting a response. For LLM apps, people track two kinds: time to the first word (2.3 seconds here) and time to the complete answer (about 4 seconds). Like the time between ordering and the first plate landing on your table.
• Hallucination. A hallucination is a fluent, confident answer that is false or made up, like “most of our sweets are nut-free” in Arjun’s prototype. Like a nervous new employee who would rather invent an answer than say “I don’t know”. RAG and a good system prompt make it much rarer. Nothing makes it impossible.

Learn it properly in Module 01 for temperature, Module 19: Streaming, Retries, Fallbacks for streaming and latency, and Module 09 for keeping hallucinations down.

Show the answer

No. The total time is the same, about 4 seconds. Streaming reduces the time to the first word, which is what people feel most strongly, so the app seems much faster. That is why latency is usually measured twice. To make the whole answer genuinely faster, you would need other tools: a smaller model, shorter prompts, caching, or fewer round trips (Modules 19 and 21).

Stop 7: Making it better next week

The answer is right. Meera’s next question is the one every engineer must answer: “How do we know it gets this right every time, for every customer and every product?”

• Eval. An eval (evaluation) is a repeatable test that measures how good your AI system’s answers are, usually over dozens or hundreds of example questions with known correct answers. Like a tasting panel that samples every batch against the standard, instead of trusting that the oven “seemed fine today”. Priya writes an eval of 200 allergy questions covering every product, and a rule: any answer calling a nut-containing product “safe” is a failure. Arjun’s prototype failed 61 of the 200. The RAG version fails 2, and those two are now being fixed.
• Fine-tuning. Fine-tuning is training an existing model a bit more on your own examples, so it picks up a style, format or narrow skill. It changes the knobs. It is powerful but rarely the first fix: here, it could teach a house tone, but it would not teach the model next month’s new hamper. Better prompts and RAG solve most problems more cheaply. Like sending an experienced baker on a short course in your house style.

Learn it properly in Module 23: Evaluations and Module 33: Fine-Tuning.

Where each stop breaks

Every word is also a place where things go wrong. Experienced engineers hear a symptom and know which stop to check.

Symptom	Stop	Likely cause
Every request fails with “401 unauthorized”	1	Wrong, expired or missing API key
Bot says the hamper has no nuts	2	Chunking split the ingredient card so “kaju barfi” landed in a passage that was not retrieved, or semantic search fetched last year’s hamper
Bot forgets what the customer said ten messages ago	3	The conversation outgrew the context window and old messages were dropped
Monthly bill triples	3	Prompts grew: someone added the whole handbook to the system prompt
Bot invents a delivery slot	5	The tool was not called, or failed, and the model guessed instead
Customers say “it feels slow”	6	High latency to first word; streaming is off
A fix for one product breaks another	7	No eval to catch regressions

Try it yourself

Meera overhears eight sentences in the office. Name the word each one is really about.

1. “The bot told a customer we have a branch in Mumbai. We don’t.”
2. “The handbook is 200 pages, so we cut it into 600 small passages.”
3. “Customers think it is broken because nothing appears for eight seconds.”
4. “Someone pasted our secret into a public GitHub repo; revoke it now.”
5. “When a customer asks about an order, the model asks our code to look it up.”
6. “We want to run the model on our own server, so we need one whose parameters we can download.”
7. “Every Friday we rerun the same 300 test questions and compare the scores.”
8. “Our long chat sessions start forgetting what the customer said at the beginning.”

Show the answer

1. Hallucination.
2. Chunking.
3. Latency, and the usual fix is streaming so words appear sooner.
4. API key. A leaked key must be revoked and replaced immediately.
5. Tool calling.
6. Open weights.
7. Eval.
8. Context window. Long conversations overflow it, so the oldest messages get dropped or summarised.

Six or more right? You are ready for Module 01. Fewer? Reread the matching stops; most people need two passes.

Common mistakes

• Treating each word as a separate technology. They are parts of one pipeline. When a new term appears, ask “which stop on the journey is this?”
• Confusing training, fine-tuning and RAG. Training builds the model; fine-tuning adjusts its knobs a little; RAG does not change the model at all, only what you put in the prompt.
• Assuming the model knows your business. It knows the public internet up to its training. Your menu, prices and policies reach it only through the prompt or tools.
• Treating “agent” as a magic word. Many products sold as agents are a single prompt or a fixed sequence of steps. An agent chooses its own next step in a loop.

Recap

• One question travels through seven stops: app, retrieval, packing, model, tools, answer, evaluation.
• The app reaches the model through an API, signed with an API key, usually via an SDK.
• RAG finds your facts first, using chunking, embeddings (vectors), a vector database and semantic search.
• The LLM reads a prompt of tokens that must fit its context window, and may use tool calling or loop as an agent.
• Users feel latency and streaming; engineers fight hallucination and measure quality with evals.

Meera reads the new reply twice. Then she turns to Arjun. “Explain to me, in my words, why the first version said yes.”

Arjun takes a breath. “The first version sent your question to a very well-read stranger with no information about us. It guessed what a bakery usually says. The new version first searches our own documents by meaning, finds the hamper’s ingredient card and our allergy rule, packs them in with strict instructions, lets the model check the delivery schedule, and then we test it on 200 allergy questions. It said no because we gave it the facts.”

“Good,” says Meera. “Keep the 200 questions. Add more.”

Priya nods at Arjun. “You have the map now. Next, let me show you how this course works, so you can pace yourself. Six months is a long walk, and most people who quit do it in week three.”

Go deeper

• Course glossary: every word in this lesson, with longer explanations.
• Model Context Protocol: the official site for Stop 5’s newest word.
• Anthropic documentation: the request at Stop 3 and the tool call at Stop 5 are explained in its Messages and tool use guides.