Module 32
LLM Internals
Look inside a language model while it runs and while it is trained. You follow the KV cache and the attention tricks that make serving affordable, shrink models with quantisation and distillation, speed them up with mixture of experts and speculative decoding, see how RLHF and reasoning methods shape behaviour, and finish with the DeepSeek architecture, which combines nearly all of it.
By the end of this module you can
- compute a model's KV cache size and explain how paged, flash and sparse attention each help
- quantise weights by hand, measure the quality cost, and write a distillation loss
- explain mixture-of-experts routing and why speculative decoding gives exactly the same output faster
- walk through RLHF, DPO, chain of thought and tree of thought with worked numbers
- explain DeepSeek's MoE, multi-head latent attention and GRPO training, and read its headline claims critically
Lessons
5 in this module- 01KV cache, paged, flash and sparse attentionExplain and size the KV cache, and choose between paged, flash and sparse attention to fix memory, speed and long-context problems when serving LLMs.
- 02Quantisation and distillationQuantise weights by hand and in code, measure the quality cost with perplexity, and train a small student model to imitate a large teacher.
- 03Mixture of experts and speculative decodingTrace a token through a mixture-of-experts layer, build a toy MoE in PyTorch, and implement speculative decoding that gives the big model's exact output faster.
- 04RLHF, chain of thought and tree of thoughtExplain how RLHF and DPO turn a base model into an assistant using preferences, and use chain of thought, self-consistency and tree of thought to improve reasoning.
- 05Case study: the DeepSeek architectureExplain how DeepSeek combined fine-grained MoE, multi-head latent attention, FP8 training, multi-token prediction and GRPO-based reinforcement learning, and read its headline claims critically.