Qwen 3.6 ~ Recording
AI by Hand ✍️ Seminars
Library › Seminar Series 2026
Thank you to everyone who came out for the Frontier AI Seminar on Qwen 3.6. There is something special about a room full of people willing to slow down and trace a single token by hand, and I am grateful you spent that hour with me.
In the Gemma 4 seminar, we looked at the transformer techniques that have settled, the parts of the modern recipe you can now expect almost everywhere. In this session we turned to the parts that are still moving, and we followed a single token through three of them.
We started with position. I worked RoPE by hand, rotating the query and key vectors by their position, and then YaRN, which stretches a trained context window toward 256K or even 1M without retraining.
We then looked at gated attention. I showed the learned gate that Qwen places on the attention path, and we compared the same attention output with the gate and without it, so you could see what it actually buys over a plain GQA or MLA setup.
We finished with decode. I traced speculative decoding, where a small drafter proposes tokens and the model verifies them in parallel, and then multi-token prediction, where the model emits several tokens in one step instead of one at a time.
Here is what I wanted you to walk away with. Once you have rotated the vector, gated the output, and drafted a token by hand, the acronyms stop being intimidating, and you can look at any model's spec sheet, its context length, its attention variant, its decode speed, and see what is actually doing the work.
Outline
PE, RoPE, YaRN
No Positional Encoding
Linear Positional Encoding
Sinusoidal Positional Encoding
Input vs Key Positional Encoding
RoPE: Rotary Positional Encoding
YaRN: Yet another RoPE extensioN
Gated Attention
Decoding: Fast and Furious
Auto-Regressive
Train
Speculative Decoding
Multi-Token Prediction
Recording & Workbook
Limited-time free: the full recording and the Excel workbook are open to everyone for now.