How LLMs work: inside the next token

This series opens a large language model along one running sentence — The cat sat on the — and shows what has to happen before the next token appears. Start with the interactive overview below, then read each guide in order.

The six guides

1. Neural networks: a function with knobs — neurons, forward passes, and backpropagation.
2. Tokenization: how text becomes numbers — tokens, vocabularies, and GPT-2’s tokenizer.
3. Embeddings: giving every token a vector — lookup tables and position vectors.
4. Attention: letting tokens look at each other — weighted context and the causal mask.
5. The transformer: blocks, stacks and a model that works — assembling the stack and counting parameters.
6. Next-token prediction: from probabilities to text — logits, softmax, temperature, top-k and top-p.

Read them in order: each guide assumes the ones before it.