Attention: the series
11 interactive diagrams
Attention, dog lovers!
One of the most important papers in AI history is a 2017 work by Vaswani et al., titled "Attention Is All You Need." It introduced the Transformer architecture, which now powers ChatGPT, Llama, DeepSeek, and almost every large language model built since. The mechanism at the heart of it is attention: a way for every token in a sequence to look at every other token and decide how much to weight it. If you want to understand modern AI, attention is the concept to start with. Turns out, attention really is all you need. Just like dogs are all you need, if you ask the right people. I will explain it using dogs.
Attention Series:
1. QKV Projection
2. Attention Computation
3. Self Attention
4. Cross Attention
5. Self Attention vs Cross Attention
6. Self Attention (Shared KV)
7. Multi-Head Attention
8. Fused QKV (Multi-Head)
9. Single vs Multi-Head Attention
10. Multi-Query Attention
11. Grouped-Query Attention
