Short-term, long-term, episodic and semantic memory

The email arrived on a Monday morning, and Meera forwarded it to Arjun with one line on top: “Please read this. Then tell me why.”

Dear Crumb, this is the third time this month I have told your WhatsApp assistant that I am allergic to peanuts. Third time. Last week it suggested I add peanut chikki to my Diwali hamper. Your staff at the Baner store have known about my allergy for six years. Why does your computer not know what your people know? — Sunita Kulkarni

Arjun scrolled through the logs. Sunita had mentioned the allergy on 3 March, 9 June and 2 October. Each time the bot had replied politely, “Thank you for letting us know!” And each time, the next conversation started from nothing, as if they had never met.

Meera walked over with her tea. “Lata at the Baner counter remembers Sunita’s allergy, her sister’s name and that she likes her bread sliced thin. Lata is not a computer. Why can’t the computer do what Lata does?”

It was a fair question, and Arjun did not have a good answer yet.

Arjun’s first attempt: send everything, every time

Arjun already knew one thing from Module 01: a large language model (LLM, the text-predicting engine behind Claude or ChatGPT) does not remember anything between calls. Each time you call the API, the model sees only what you send in that one request. Its context window (the fixed amount of text it can read in one go) is the whole world as far as the model is concerned.

So his first fix looked obvious. If the model only knows what is in the request, put everything in the request. He wrote a function that loaded every past conversation with a customer and pasted them all in front of the new message.

def build_messages(customer_id: str, new_message: str) -> list[dict]:
    messages = []
    for conversation in load_all_conversations(customer_id):  # every chat, ever
        messages.extend(conversation)
    messages.append({"role": "user", "content": new_message})
    return messages

He tested it with Sunita’s history. It worked, in the sense that the bot now said “I know you are allergic to peanuts.” Then he looked at the numbers.

Sunita had 47 conversations over eight months, averaging about 1,200 tokens each (a token is a chunk of text, roughly three quarters of an English word). That is about 56,400 tokens of history sent with every single message she types. Suppose input costs $3permilliontokens(priceschange;checkyourprovide{r}^{\prime }spricingpage).OnereplytoSunitanowcostsabout$0.17, or roughly ₹15. Crumb’s WhatsApp line handles about 2,000 messages a day. If every customer had Sunita’s history, that is around ₹30,000 a day just to re-read old chats, before the model writes a word.

Then he found the worse problem. He asked the bot, with full history attached, to suggest a Diwali hamper for Sunita. It suggested kaju katli and peanut chikki. The allergy was in the history, mentioned three times, but buried in conversation 3, 21 and 40, between delivery complaints and a long chat about her sister’s wedding cake. Models pay less reliable attention to details in the middle of very long inputs. Having a fact somewhere in the context is not the same as the model using it.

And a few loyal customers had over 200 conversations. Their history did not fit in the context window at all.

Priya’s question: what does Lata actually remember?

Priya listened to all of this and drew a counter on the whiteboard, with a stick figure behind it.

“Lata at Baner. When Sunita walks in, does Lata replay six years of conversations in her head before saying hello?”

“No,” said Arjun. “She just… knows things.”

“What kinds of things? Be specific.”

Arjun thought about it. “She knows what Sunita is saying right now. She knows Sunita is allergic to peanuts. She remembers that last Diwali the hamper was late and Sunita was upset. She knows chikki is made of peanuts and jaggery, which has nothing to do with Sunita, she just knows the menu. And she knows how to pack a hamper so the jars don’t break.”

Priya wrote five words on the board, one for each thing he had said.

“Congratulations. You have just described the five kinds of memory that people who build agents talk about. Psychologists named them decades ago, for humans. We borrow the names because the jobs are the same.”

What Lata knows	The name for it	What it holds
What Sunita is saying right now	Short-term (working) memory	The current conversation
“Sunita is allergic to peanuts”	Factual memory	Stable facts and preferences about a person
“Last Diwali her hamper was late”	Episodic memory	Specific past events, with a when and a what happened
“Chikki is made of peanuts”	Semantic memory	General knowledge about the world, not about any one person
How to pack a hamper safely	Procedural memory	Know-how: skills, rules, ways of doing things

“Notice,” Priya said, “Lata does not keep transcripts. She keeps the useful residue of conversations. That is the whole trick. Your job is not to remember everything. It is to remember the right things and bring them back at the right moment.”

Short-term memory: the desk

Priya drew a desk next to the counter.

“Short-term memory is the desk Lata is working at right now. The order slip, the customer in front of her, what was said a minute ago. It is fast and right there, but small. When the desk fills up, old papers fall off the edge.”

For an agent, short-term memory is the context window. The list of messages you send with each API call, the system prompt, any tool results from this conversation: that is the desk. You met this in Module 02, when you kept a messages list so the chatbot could follow a conversation. And in Module 14 you saw LangGraph save that list per thread with a checkpointer, so a conversation survives a server restart. That is still short-term memory. It belongs to one conversation.

Watch what the clip shows: as messages pile up, the oldest ones fall out of the window. Something has to decide what falls off. The simplest policy is “keep the last few turns”. A smarter one summarises older turns into a paragraph before dropping them. Either way, short-term memory is temporary by design.

Long-term memory: the filing cabinet

“Long-term memory,” Priya said, drawing a filing cabinet behind the desk, “is everything Lata knows that is not on the desk right now. She goes to the cabinet, pulls out the right card, and puts it on the desk when she needs it.”

For an agent, long-term memory is any store outside the model: a database table, a document, a vector database (a database that finds stored text by meaning rather than exact words, from Module 08). Three steps keep it useful:

1. Write. After (or during) a conversation, decide what is worth keeping and save it.
2. Retrieve. When a new message arrives, find the few memories relevant to it.
3. Inject. Put those memories into the context window, usually in the system prompt, so the model can use them.

flowchart LR
  U["New message"] --> R["Retrieve relevant memories"]
  LTM[("Long-term store")] --> R
  R --> C["Context window: system prompt + memories + recent turns"]
  C --> M["LLM"]
  M --> A["Reply"]
  A --> W["Decide what to write"]
  W --> LTM

That loop is the whole memory layer. Everything in the rest of this module is a better version of one of those boxes.

Now the five memory types fit into that cabinet like drawers:

• Factual memory holds stable truths about a person or account: allergies, delivery address, “prefers eggless”. Small, precise, high value. When a fact changes (Sunita moves house), the old one should be replaced, not kept alongside.
• Episodic memory holds events: “28 Oct 2025, hamper arrived two hours late, given a ₹200 voucher.” Each episode has a time. Episodes help the agent be consistent (“Last time we gave you a voucher”) and spot patterns (“third late delivery to this pincode”).
• Semantic memory holds general knowledge that does not belong to any one customer: the menu, ingredients, store hours, the refund policy. You already built this, in fact. A RAG system over the 200-page operations handbook (Module 09) is semantic memory.
• Procedural memory holds know-how: the system prompt, the rules for handling a refund, a learned instruction like “always confirm the pincode before promising same-day delivery”. Most agents keep this in their prompts and code. Some advanced setups let the agent update its own instructions, carefully.

Clearing up three misconceptions

Arjun had picked up some wrong ideas from blog posts, and Priya took them apart one at a time.

“Semantic memory means vector search.” No. This one trips up nearly everyone, because the word semantic shows up in both. Semantic search (Module 07) is a retrieval technique: finding text by meaning. Semantic memory is a category of knowledge: general facts about the world. You can store factual or episodic memories in a vector database and retrieve them with semantic search. You can store semantic memory in a plain SQL table. The technique and the category are independent.

“Just buy a bigger context window.” Some models now read a million tokens. That makes the desk bigger, not the filing cabinet smarter. You still pay for every token on every call, the model still attends unevenly to giant inputs, and a customer with five years of history still outgrows it eventually. A bigger desk is welcome. It is not memory.

“The model learns from our chats.” It does not. Remember training versus inference from Start Here: the model’s parameters (its internal numbers) are frozen when you use it. Nothing Sunita says changes the model. Every bit of “learning” in an agent lives in data you store and send back. That is good news, actually: it means you control it, can inspect it, and can delete it.

Following one message through all five memories

Priya picked a real message from the logs to make it concrete. A week after the complaint, Sunita wrote:

“Can you add something sweet to my usual hamper? It’s for my sister this year.”

“Walk it through,” Priya said. “Which memory answers which part?”

They went phrase by phrase.

• “my usual hamper” needs episodic memory. The order history shows two Diwali hampers, in October 2024 and October 2025, both the ₹1,499 Classic. “Usual” means the Classic.
• “something sweet” needs semantic memory. The menu says kaju katli contains cashews (a tree nut), chikki contains peanuts, and the millet ladoo contains neither.
• “for my sister” is a trap, and it is where factual memory gets subtle. The stored fact is “Sunita is allergic to peanuts.” The hamper is for her sister Kavita. Do we know anything about Kavita? A careful agent should not silently apply Sunita’s allergy to Kavita, and should not ignore it either. It should ask.
• How to reply comes from procedural memory: the system prompt’s rule “for any allergy-related question, never guess; confirm, and suggest the customer check the label.”
• The conversation so far is short-term memory: the last few messages, so “it” and “this year” make sense.

Then they built the context the model would actually receive, and counted tokens:

Piece	Kind of memory	Tokens (approx.)
System prompt with the allergy rule	Procedural	150
“Allergic to peanuts (confirmed 3 times)” and “Sister: Kavita”	Factual	40
Last two Diwali orders, one line each	Episodic	50
Three menu items with ingredients	Semantic	90
Last six messages of this chat	Short-term	400
Total		about 730

About 730 tokens instead of 56,400. At the same $3permillion,thatisabout$0.002, roughly ₹0.19 per reply, around 75 times cheaper. More important, the allergy now sits in a short, clearly labelled block near the top, where the model reliably notices it.

The reply the model produced with this context:

“Of course! Your usual Classic Diwali Hamper is ₹1,499. For something sweet, the millet ladoo box (₹220) is peanut-free and nut-free. Quick check, since it’s for Kavita this time: does she have any allergies we should know about?”

That is what Lata would have said.

The real code, built up in two layers

Here is a minimal version of both layers, so you can feel the difference. Install the SDK first with uv add anthropic python-dotenv, and put your ANTHROPIC_API_KEY in a .env file.

Layer 1: short-term memory with a sliding window. Keep a list of messages for this conversation, and send only the last few.

import anthropic
from dotenv import load_dotenv

load_dotenv()
client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

KEEP_EXCHANGES = 3  # remember the last 3 back-and-forths
history: list[dict] = []


def chat(user_text: str, system: str) -> str:
    history.append({"role": "user", "content": user_text})
    # 3 exchanges = 6 messages, plus the new user message = 7.
    # An odd number means the window always starts on a user message.
    recent = history[-(2 * KEEP_EXCHANGES + 1):]
    response = client.messages.create(
        model="claude-opus-5-5",
        max_tokens=500,
        system=system,
        messages=recent,
    )
    reply = next(b.text for b in response.content if b.type == "text")
    history.append({"role": "assistant", "content": reply})
    return reply

Two details are worth a second look. history keeps everything, but recent is what travels to the model: the desk, not the archive. And the window size is odd on purpose. The API expects a conversation to start with a user message, and slicing an even number of messages off a user-assistant-user list would start on an assistant turn.

Layer 2: a tiny long-term memory. For now, a dictionary of facts per customer, injected into the system prompt.

LONG_TERM = {
    "sunita": [
        "Allergic to peanuts (told us on 3 Mar, 9 Jun and 2 Oct).",
        "Usually orders the Classic Diwali Hamper (Rs 1,499).",
        "Has a sister, Kavita.",
    ],
}

BASE_RULES = (
    "You are Crumb's friendly support assistant. "
    "For any allergy question, never guess: check, and ask if unsure."
)


def system_prompt_for(customer_id: str) -> str:
    facts = LONG_TERM.get(customer_id, [])
    lines = "\n".join(f"- {fact}" for fact in facts) or "- Nothing yet."
    return f"{BASE_RULES}\n\nWhat you know about this customer:\n{lines}"


print(chat("Can you add something sweet to my usual hamper? It's for my sister.",
           system=system_prompt_for("sunita")))

Run it, and you should see a reply that mentions the Classic hamper, avoids peanuts, and asks about Kavita. Now run it with system_prompt_for("rohan"), a customer with no facts. The reply becomes generic and asks which hamper he means. Same model, same code. Only the memory changed.

Where this breaks

Arjun shipped the dictionary version to a test group of ten regular customers on Wednesday. By Friday it had already bitten him three times.

Nobody fills in the dictionary. He had typed Sunita’s facts by hand. That does not scale to 4,100 customers. Something has to extract memories from conversations automatically, and decide which are worth keeping. That is exactly what the next lesson’s tool, Mem0, does.

Facts go stale. Rohan told the bot he had moved from Baner to Kothrud. The old address was still in the dictionary, so the bot now had both and picked the wrong one. Factual memory needs updates, not just additions.

The wrong things get remembered. In one test, the bot itself said, incorrectly, “our brownies are gluten-free.” If you automatically save what the assistant says, you can turn a hallucination (a confident, made-up answer) into a permanent “fact”. What you let into memory needs rules. Lesson 3 is about exactly those rules: what to store, what to forget, and how to keep it private.

Retrieval gets harder as memory grows. Ten facts per customer fit in a system prompt. Four hundred do not. Then you need to pick the relevant few for each message, which means search, which means a vector database.

Try it yourself

Take this short conversation and label every piece of information with its memory type. Decide which pieces are worth saving long-term at all.

Customer: Hi, it’s Farah. Is the Koregaon Park store open on Sunday? Bot: Yes, 8 am to 9 pm. Customer: Great. I’m vegan now, by the way, so no more butter croissants for me. Last month’s croissants were stale, too. Bot: Sorry about that! I’ve noted it.

Show the answer

• “Koregaon Park opens 8 am to 9 pm on Sunday” is semantic (general store knowledge). Do not save it per customer; it belongs in the store-hours source.
• “Farah is vegan now” is factual, and important. Save it, and note it replaces any older “loves butter croissants” preference.
• “Last month’s croissants were stale” is episodic. Worth saving with a date: it explains her mood and may matter if it happens again.
• “Is it open on Sunday?” and the greetings are short-term only. Nothing to keep.
• “Sorry about that! I’ve noted it” is the assistant’s words. Do not save them as facts.

Common mistakes

• Treating the transcript as the memory. Raw chats are expensive to re-send and hard for the model to use. Keep the useful residue: short, self-contained statements.
• Confusing “semantic memory” with “semantic search”. One is a kind of knowledge, the other a way of finding things.
• Saving the assistant’s claims as facts. The model can be wrong. Only store what the user said or what a trusted system confirmed.
• Forgetting who a memory is about. “Likes millet ladoo” filed under the wrong person is worse than no memory at all.
• Assuming the model learns from use. It does not. If it is not stored and sent back, it is gone.

Recap

• An LLM forgets everything between calls. Memory is something you build around it.
• Short-term memory is the context window for this conversation: fast, small, temporary.
• Long-term memory lives outside the model and follows a loop: write, retrieve, inject.
• Long-term memory comes in kinds: factual (who you are), episodic (what happened), semantic (what is generally true) and procedural (how we do things).
• Good memory is about selection. 730 well-chosen tokens beat 56,400 tokens of transcript on cost and on accuracy.

On Friday, Arjun sat down with Meera and explained it the way Priya had explained it to him. “Lata doesn’t replay six years of chats. She has a desk for the conversation in front of her and a filing cabinet of the things that matter. The bot had a desk and no cabinet. I’ve built a cabinet. Now I need a clerk who files things in it automatically.”

Meera smiled for the first time that week. “Then hire the clerk.” In the next lesson, Arjun does: he sets up Mem0, a library that listens to conversations and files memories for you.

Go deeper

• Cognitive Architectures for Language Agents (CoALA), the paper that popularised working, episodic, semantic and procedural memory for LLM agents.
• MemGPT: Towards LLMs as Operating Systems, which treats the context window like RAM and external storage like disk.
• LangGraph documentation, see the memory concepts page for short-term versus long-term memory in practice.
• Building effective agents by Anthropic.