Case study: the DeepSeek architecture

In late January 2025, Meera forwarded Arjun a news article with three exclamation marks in the subject line. A Chinese AI lab called DeepSeek had released a reasoning model that matched the best American models on maths and coding tests. The article said it had been trained for about 6 million US dollars, a tiny fraction of what rivals were thought to spend. The same day, the chipmaker Nvidia lost several hundred billion dollars of stock-market value.

“Is this real?” Meera wrote. “Should we be using it? Is it cheaper? Is it safe?”

Arjun replied within the hour, excited. “Already on it. I ran ollama run deepseek-r1 on our server. It thinks out loud before answering, it’s amazing, and it’s free!”

Priya’s reply came a few minutes later. “Before you tell Meera we’re running DeepSeek, run ollama show deepseek-r1 and read the architecture line.”

Arjun’s first attempt: “we’re running DeepSeek”

Arjun ran the command. The architecture line said qwen2. The parameter count said 7.6 billion.

“That’s not DeepSeek’s model?”

“It’s a distilled DeepSeek model,” said Priya. “The real DeepSeek-R1 is a 671-billion-parameter mixture of experts. It would need hundreds of gigabytes of GPU memory even when quantised. What you downloaded is a 7B model from Alibaba’s Qwen family that DeepSeek fine-tuned on about 800,000 examples written by the big R1. Remember distillation, two lessons ago? This is it. A very good student. Not the teacher.”

That confusion was common in early 2025, and it is the right place to start, because it shows how to read this story properly. There are several different things called “DeepSeek”, released over about a year:

Release	When	What it is
DeepSeek-V2	May 2024	236B-parameter MoE (21B active) that introduced multi-head latent attention and DeepSeekMoE
DeepSeek-V3	December 2024	671B-parameter MoE (37B active), the general chat model and the base for R1
DeepSeek-R1-Zero	January 2025	V3’s base model trained for reasoning with reinforcement learning only
DeepSeek-R1	January 2025	The polished reasoning model, the one in the headlines
R1 distilled models	January 2025	Qwen and Llama models from 1.5B to 70B, fine-tuned on R1’s outputs

Reading the 6-million-dollar headline

DeepSeek’s V3 report said the final training run used about 2.788 million GPU-hours on Nvidia H800 chips. Assuming a rental price of 2 US dollars per GPU-hour, that is about 5.6 million dollars. The report also said, plainly, that this covers only the official training run, not the earlier research, experiments, failed runs, data work, staff, or the cost of owning the thousands of GPUs.

So the headline compared one line of DeepSeek’s budget with other companies’ total spending. The real achievement was still remarkable: a frontier-class model trained with far less compute than people expected. But the reason was not one magic trick. It was many efficiency ideas stacked together, most of which you have now met in this module. Let us take them one at a time.

Idea 1: a very sparse mixture of experts

You saw in the mixture-of-experts lesson that MoE models hold many expert networks but route each token through only a few. DeepSeek pushed this further, in a design it calls DeepSeekMoE.

Many small experts instead of a few big ones. Mixtral had 8 large experts per layer and picked 2. DeepSeek-V3 has 256 smaller routed experts per layer and picks 8. Priya’s kitchen analogy: instead of eight big stations (“cakes”, “bread”), there are 256 small ones (“chocolate ganache”, “laminated dough”, “sugar syrup at one-thread consistency”). Each order visits eight, and they can combine in far more ways, so each expert can specialise more finely.

A shared expert every token visits. Some knowledge is needed for almost every token: basic grammar, common words. Rather than have many routed experts each learn it, V3 adds one shared expert that every token always goes through. Think of the packing station every order passes.

The numbers. V3 has 671 billion total parameters but activates about 37 billion per token: about 5.5%. Compare Mixtral, which activated about 28% (13 of 47 billion). Each token costs roughly what a 37B dense model would, while the model holds 671B parameters of knowledge.

Balancing without the extra loss. The load-balancing loss you met last lesson nudges the router to share work, but it also pulls the model slightly away from what it is really trying to learn. V3 instead uses what its authors call auxiliary-loss-free balancing. Each expert gets a bias number added to its routing score, used only when choosing experts, not when weighting their outputs. After each training step, overloaded experts’ biases go down a little and underused experts’ biases go up. Ravi at the pass, noticing the mithai station is swamped, starts giving the idle stations a slight edge in close calls, without changing how any dish is made.

Idea 2: multi-head latent attention, a smaller notebook

Remember the KV cache, the notebook of keys and values a model keeps for every past token. It was the reason Crumb’s server ran out of memory on the Saturday before Diwali. You also saw one fix, grouped-query attention, where several query heads share one set of keys and values.

DeepSeek went further with multi-head latent attention (MLA), introduced in V2. The idea: do not store every head’s keys and values at all. Store one small compressed vector per token, called a latent vector, and rebuild each head’s keys and values from it when attention needs them.

“It’s shorthand on the order slip,” said Priya. “Shanti doesn’t write ‘one eggless chocolate truffle cake, 1 kg, with a happy birthday message in white icing’. She writes ‘ECT 1kg HB-W’. The kitchen can rebuild the full order from that because everyone learned the shorthand. MLA learns the shorthand: a compression matrix to write it, and expansion matrices to read it back.”

The arithmetic

In a toy layer with 8 heads of 128 numbers each, a standard KV cache stores $2\times 8\times 128=2,048$ numbers per token. Compress each token to a 64-number latent instead, and the cache stores 64: 32 times smaller.

import torch
import torch.nn as nn

torch.manual_seed(0)
d_model, n_heads, d_head, d_latent = 1024, 8, 128, 64
T = 500                                              # tokens in the conversation so far

# Standard attention caches full keys and values for every head
W_k = nn.Linear(d_model, n_heads * d_head, bias=False)
W_v = nn.Linear(d_model, n_heads * d_head, bias=False)

# Latent attention caches one small compressed vector per token...
W_down = nn.Linear(d_model, d_latent, bias=False)            # write the shorthand
# ...and rebuilds every head's keys and values from it when needed
W_up_k = nn.Linear(d_latent, n_heads * d_head, bias=False)   # read it back as keys
W_up_v = nn.Linear(d_latent, n_heads * d_head, bias=False)   # read it back as values

x = torch.randn(T, d_model)
standard_cache = (W_k(x), W_v(x))                    # what a normal layer stores
latent_cache = W_down(x)                             # what a latent layer stores
k = W_up_k(latent_cache).view(T, n_heads, d_head)    # rebuilt at attention time
v = W_up_v(latent_cache).view(T, n_heads, d_head)

std_numbers = sum(t.numel() for t in standard_cache)
print("standard cache numbers:", std_numbers)
print("latent cache numbers:  ", latent_cache.numel())
print(f"{std_numbers / latent_cache.numel():.0f}x smaller; rebuilt keys shape: {tuple(k.shape)}")

standard cache numbers: 1024000
latent cache numbers:   32000
32x smaller; rebuilt keys shape: (500, 8, 128)

Two details make the real thing work. First, rebuilding keys and values for every past token at every step would be slow, but the expansion matrices can be mathematically folded into the query and output matrices, so at inference time the model can attend using the compressed vectors more or less directly. Second, position information from RoPE (the rotating position scheme from Module 30) does not survive this folding, so MLA keeps a small separate, uncompressed key only for position, which DeepSeek calls decoupled RoPE.

In V3’s published configuration, each token in each layer caches 576 numbers (a 512-number latent plus a 64-number position key). With 61 layers in 16-bit, our arithmetic gives about 70 KB per token, smaller than the 128 KB per token of Crumb’s 8-billion-parameter model, for a model with 671 billion parameters. DeepSeek’s V2 paper reported that V2, using MLA, needed about 93% less KV cache than the company’s earlier 67B dense model, and that MLA matched or beat standard attention on quality in their tests.

Idea 3: training cheaply

Two more ideas cut V3’s training bill.

FP8 training. The quantisation lesson showed models can run with fewer bits. V3 was largely trained with 8-bit floating-point numbers (FP8) for most of the heavy matrix maths, keeping sensitive parts in higher precision and using fine-grained scaling (one scale per small block of numbers, exactly the grouping trick that rescued GPT-2’s 4-bit weights). Fewer bits mean faster arithmetic and less memory traffic. Training at this scale in FP8 had rarely been shown to work before.

Multi-token prediction (MTP). Normally the model learns to predict one next token. V3 added a small extra module trained to also predict the token after that. The extra target gives a denser training signal. And at inference time, the MTP module can act as a built-in draft model for speculative decoding: the V3 report says the second-token guess was accepted roughly 85 to 90% of the time, which speeds up generation.

Add engineering work on communication between thousands of GPUs, and you get 14.8 trillion training tokens processed for those reported 2.788 million GPU-hours.

Idea 4: R1 and learning to reason by trial and error

V3 was a strong general model. R1 was the one that made headlines, because of how it learned to reason.

Last lesson, you saw that reasoning models are trained to write long chains of thought. The usual assumption was that you needed huge numbers of human-written reasoning examples to teach this. DeepSeek tried something bolder with R1-Zero: take V3’s base model and use only reinforcement learning, with no human-written reasoning examples at all.

Rewards a program can check

RLHF needed a reward model trained on human comparisons. R1-Zero instead used rule-based rewards on problems where a program can check the answer:

• Accuracy reward: maths problems with known answers (the final answer must match) and coding problems (the code must pass tests).
• Format reward: the model must put its reasoning between <think> and </think> tags and its answer in a defined place.

No neural reward model means nothing for the policy to sweet-talk. This approach is now often called reinforcement learning with verifiable rewards.

GRPO: grading on a curve

The RL algorithm was GRPO, group relative policy optimisation, which DeepSeek had introduced in an earlier maths paper (DeepSeekMath, 2024). PPO, used in classic RLHF, needs an extra value model (a “critic”, as big as the policy itself) to estimate how good each situation is. GRPO drops the critic. Instead, for each prompt it samples a group of answers and judges each answer against the group’s average: grading on a curve.

Follow one prompt: Meera’s hamper question from last lesson, correct answer ₹3,435. The model samples a group of 4 answers. Using the accuracy reward only:

Answer	Final number	Reward $r_i$
1	3,435	1
2	3,810	0
3	3,435	1
4	3,435	1

The group mean is 0.75 and the standard deviation (spread) is 0.5. Each answer’s advantage, how much better it did than its siblings, is:

$$A_i=\frac{r_i-\text{mean}(r)}{\text{std}(r)}$$

• Correct answers: $(1-0.75)/0.5=+0.5$
• The wrong answer: $(0-0.75)/0.5=-1.5$

Training then makes every token of answers 1, 3 and 4 a little more likely and every token of answer 2 noticeably less likely, using a clipped update much like PPO’s, plus a KL penalty that keeps the model near a reference, the same leash as RLHF. Over thousands of prompts, reasoning patterns that tend to reach correct answers get reinforced.

import re
import torch

def reward(completion: str, correct_answer: str) -> float:
    """Rule-based reward: format plus correctness. No neural reward model."""
    r = 0.0
    if re.search(r"<think>.+</think>", completion, re.S):           # showed its working
        r += 0.2
    m = re.search(r"<answer>\s*₹?([\d,]+)\s*</answer>", completion)
    if m and m.group(1).replace(",", "") == correct_answer:          # right final answer
        r += 1.0
    return r

group = [   # four sampled completions for the hamper question
    "<think>3 x 1250 = 3750, over 3000 so 10% off = 3375, plus 60</think><answer>₹3,435</answer>",
    "<think>3 x 1250 = 3750, plus 60 = 3810</think><answer>₹3,810</answer>",
    "<think>3750 minus 375 is 3375, add delivery 60</think><answer>3435</answer>",
    "The total is ₹3,435.",
]
rewards = torch.tensor([reward(c, "3435") for c in group])
advantages = (rewards - rewards.mean()) / (rewards.std() + 1e-6)
print("rewards:   ", [round(r, 2) for r in rewards.tolist()])
print("advantages:", [round(a, 2) for a in advantages.tolist()])

rewards:    [1.2, 0.2, 1.2, 0.0]
advantages: [0.86, -0.7, 0.86, -1.02]

Look at answer 4. It has the right number, but no tags, so a strict rule gives it 0 and it is pushed down. Rule-based rewards are precise and unhackable in one sense, and blunt in another: they reward exactly what you wrote, nothing more. The 1e-6 avoids dividing by zero.

What emerged

DeepSeek reported that during R1-Zero’s training its answers grew steadily longer, from hundreds to thousands of tokens of reasoning, without anyone asking for that. The model began re-checking its work and trying alternative approaches. The paper highlights one transcript where the model writes, mid-solution, something like “Wait, wait. That’s an aha moment I can flag here”, and then re-evaluates its steps. Nobody programmed reflection; it was rewarded into existence because it led to correct answers.

R1-Zero also had problems: it mixed languages within a single answer and its reasoning was often hard to read. So the released R1 used a multi-stage recipe:

flowchart TD
    A["V3 base model"] --> B["Cold-start SFT on thousands of readable reasoning examples"]
    B --> C["Reasoning RL with GRPO (plus a language-consistency reward)"]
    C --> D["Sample many answers, keep good ones: about 800k examples"]
    D --> E["SFT on those examples (reasoning and general tasks)"]
    E --> F["Final RL for helpfulness and harmlessness"]
    F --> G["DeepSeek-R1"]
    D --> H["Distilled models: Qwen and Llama, 1.5B to 70B"]

The released R1 performed comparably to OpenAI’s o1 reasoning model on several maths and coding benchmarks, according to DeepSeek’s report, and its weights were released under the permissive MIT licence.

The distilled models

The last arrow is the model Arjun downloaded. DeepSeek took the roughly 800,000 examples generated in step D and fine-tuned smaller open models on them: ordinary supervised fine-tuning, no RL. These distilled models reasoned far better than their originals. DeepSeek also reported a telling comparison: distilling from R1 gave a small model better reasoning than running large-scale RL on that small model directly. Big models discover reasoning patterns; small models learn them faster by imitation.

Running a distilled R1 yourself

Through Ollama (Module 20):

# ollama pull deepseek-r1:7b   (about 5 GB, a 4-bit Qwen-based distilled model)
import ollama

reply = ollama.chat(model="deepseek-r1:7b", messages=[{"role": "user", "content":
    "A customer orders 3 Diwali hampers at ₹1,250 each. Orders above ₹3,000 get "
    "10% off. Delivery is ₹60. What is the total?"}])
print(reply["message"]["content"])

Expect a long passage of reasoning before the answer. Depending on your Ollama version, the reasoning appears inside <think> tags in the content or in a separate thinking field. Library APIs change quickly. If a name has moved, check the docs linked below.

Where it breaks: an honest assessment for Crumb

Thinking costs tokens. On “What time does the Camp store open?”, the 7B distilled model wrote over a thousand tokens of deliberation before answering. That is slower and, on a paid API, more expensive. Reasoning models are for problems that need reasoning.

Distilled is not the real thing. The 7B student is far better at maths than the original Qwen 7B, but far weaker than the 671B R1, especially on knowledge and on languages like Marathi. Judge it on Crumb’s own test set, not on headlines.

Where your data goes. Running open weights on your own server keeps data on your machines, which suited Meera’s privacy rule. Using DeepSeek’s hosted app or API sends data to servers operated in China, under Chinese law. Many companies and some governments restricted the hosted service for that reason. Researchers also reported that the hosted versions refuse or deflect some politically sensitive topics. Self-hosted open weights and a hosted service are different risk decisions.

Benchmarks are not your business. Matching another model on competition maths says little about writing a kind, accurate reply to an upset customer in Hinglish. Evaluate on your own tasks, as in Module 23.

The field moved on, and keeps moving. DeepSeek kept releasing updates through 2025, including an experimental sparse attention mechanism of the kind you met in the first lesson of this module, and other labs adopted many of the same ideas. The lasting lesson is not “use DeepSeek”. It is how these techniques combine.

Try it yourself

1. A group of 4 answers gets rewards [1, 1, 0, 0]. Compute each advantage using the sample standard deviation.
2. Using the KV-cache formula from the first lesson of this module, how many bytes per token would a standard attention layer need with 128 heads of 128 numbers each for both keys and values, in 16-bit? Compare with MLA’s 576 numbers per layer.

Show the answer

1. Mean 0.5. Sample standard deviation: each answer is 0.5 from the mean, so $\sqrt{(4\times 0.25)/3}\approx 0.577$. Advantages: $+0.5/0.577\approx +0.87$ for the correct answers, $-0.87$ for the wrong ones. A balanced group gives symmetric pushes.
2. $2\times 128\times 128=32,768$ numbers, or 65,536 bytes per layer per token, in 16-bit. MLA’s 576 numbers are 1,152 bytes: about 57 times smaller per layer. (V3’s real keys are slightly larger than this simplified version because of the position part, so the true saving is larger still.)

Common mistakes

• Confusing the distilled models with R1. deepseek-r1:7b is a Qwen model trained on R1’s outputs. Check the architecture and parameter count of anything you download.
• Reading “37B active” as “fits in 37B of memory”. MoE needs memory for all 671B parameters.
• Repeating the 6-million-dollar figure as the total cost. It was the reported compute for the final training run only.
• Assuming rule-based rewards fit every task. They work where a program can check answers (maths, code, formats). Tone and helpfulness still need human or AI preference signals.
• Treating open weights and hosted APIs as the same choice. They differ in privacy, data residency, content behaviour and cost.

Recap

• DeepSeek-V3 is a 671B-parameter mixture of experts with 256 fine-grained routed experts plus a shared expert, activating about 37B parameters per token and balancing load with adjustable routing biases instead of an extra loss.
• Multi-head latent attention caches one small compressed vector per token and rebuilds keys and values from it, shrinking the KV cache dramatically.
• FP8 training and multi-token prediction cut training cost; the MTP module doubles as a speculative-decoding draft at inference.
• R1-Zero learned long reasoning from reinforcement learning alone, using GRPO, which scores each sampled answer against its group’s average, and rule-based rewards for correct, well-formatted answers.
• R1 added a cold start, more SFT and RL stages, and its outputs were distilled into small Qwen and Llama models that anyone can run.

Arjun rewrote his reply to Meera.

“It’s real, with an asterisk. The 6 million dollars was the final training run, not the whole bill. What they actually did well was use every efficiency trick at once: a huge kitchen of specialists where each order only visits a few stations, order slips in shorthand so the notebook stays small, and a training method where the model practised maths problems and was rewarded only for right answers, grading each attempt against its own other attempts. It taught itself to check its work.”

“And what I’m running on our server?”

“A small student model that learned from the big one by copying its homework. Very good at sums, weaker at Marathi, and it overthinks simple questions. Great for hamper pricing. Not for ‘what time do you open’.”

Meera’s reply was one line. “So could we teach a small model our own homework?”

Priya forwarded it to Arjun with a smile. “That’s the next module. Fine-tuning.”

Go deeper

• DeepSeek-V3 on GitHub, with the technical report and model details.
• DeepSeek-R1 on GitHub, with the R1 paper and distilled models.
• Ollama, for running the distilled models locally.
• Hugging Face docs, see the TRL library, which includes a GRPO trainer.
• Training language models to follow instructions with human feedback, for the PPO-based RLHF that GRPO simplifies.