How Mixture-of-Experts LLMs Work

A technically rigorous course on the mechanics of Mixture-of-Experts (MoE) language models — the Switch / GLaM / Mixtral / DeepSeek-MoE family. Each lesson is framed as a contrast: at each point in the stack we ask “the dense model does X here — what does the MoE do instead, and why?”

Audience: technically comfortable, already familiar with dense decoder-only LLMs (next-token prediction, attention, the feed-forward network, fine-tuning).
Prerequisites: linear algebra, multivariate calculus, basic probability.

Appendices