Calculating AI by Hand ✍️
Step-by-step walkthroughs
Most materials for learning AI are either full of equations or buried in code-heavy Jupyter notebooks. I made this series of 28 calculation walkthroughs to take a different approach, the first I created for AI by Hand, from late 2023 through 2024.
Each post shrinks one model down to an example that fits on one page, and works it end to end by hand, so you can follow every calculation with a pen. The recurring revelation is that, underneath the jargon, almost every model is just a series of matrix multiplications.
I have since revised the whole series and reordered it from the simplest building block to the most advanced. I hand-calculated every example myself; on a few occasions, sharp-eyed readers caught my mistakes, and their corrections are noted in the posts.
Start at the top, or jump to whichever model you have always wanted to understand. Grab a pen. ✍️
1. Matrix Multiplication by Hand ✍️
2. Multi Layer Perceptron (MLP) by Hand ✍️
3. Backpropagation by Hand ✍️
4. SVM by Hand ✍️
5. Batch Normalization by Hand ✍️
6. Dropout by Hand ✍️
7. Recurrent Neural Network (RNN) by Hand ✍️
8. LSTM by Hand ✍️
9. Deep RNN by Hand ✍️
10. Self Attention by Hand ✍️
11. Transformer by Hand ✍️
12. Autoencoder by Hand ✍️
13. Variational Auto Encoder (VAE) by Hand ✍️
14. Sparse Auto Encoder (SAE) by Hand ✍️
15. Generative Adversarial Network (GAN) by Hand ✍️
16. Sampling a Sentence by Hand ✍️
17. Residual Network by Hand ✍️
18. U-Net by Hand ✍️
19. Discrete Fourier Transform by Hand ✍️
20. Graph Convolutional Network (GCN) by Hand ✍️
21. CLIP by Hand ✍️
22. Vector Database by Hand ✍️
23. Mixture of Experts (MoE) by Hand ✍️
24. Switch Transformer by Hand ✍️
25. Mamba's S6 by Hand ✍️
26. Sora's Diffusion Transformer (DiT) by Hand ✍️
27. BitNet by Hand ✍️
28. Reinforcement Learning with Human Feedback (RLHF) by Hand ✍️
