Library › Calculating AI by Hand ✍️
Matrix Multiplication by Hand ✍️
Reinforcement Learning with Human Feedback (RLHF) by Hand ✍️
One of the key ingredients of the deep learning revolution is the ability to perform matrix multiplications at incredible speed and scale.
Despite linear algebra being a prerequisite of my courses, I found many students lack confidence in multiplying matrices. So I teach them this unique system for multiplying matrices by hand.
How does it work?
Setup
Step 1 of 9: Given
Multiply A (green, 2×2) by B (yellow, 2×3).
Lay them out compactly: B across the top, A down the left, and an empty grid for the result C where they meet.
Lay Out the Matrices
Step 2 of 9: Place B
Copy B's values across the top.
Step 3 of 9: Place A
Copy A's values down the left.
Compute C
Step 4 of 9: Row 1 × Column 1
Each result cell is the dot product of the green row and the yellow column that meet there.
Step 5 of 9: Row 1 × Column 2
Step 6 of 9: Row 1 × Column 3
Step 7 of 9: Row 2 × Column 1
Step 8 of 9: Row 2 × Column 2
Step 9 of 9: Row 2 × Column 3
Key Properties
This system has several desirable properties. Say we are calculating A × B = C.
💡 Dimensions: We can easily see C's dimensions must match A's height (vertical) and B's width (horizontal).
💡 Scalable: If we scale any of the matrices A, B, C to an arbitrary size, we can easily see how the sizes of the other matrices must change accordingly to match.
💡 Row vs. Column Vectors: For each value in C, we can easily see the corresponding row vector (green) and column vector (yellow) that are combined using dot product to obtain the value.
💡 Stackable: Because of their compactness, we can stack multiple grids like these to represent the underlying sequence of matrix multiplications of a deep neural network (e.g., a multi-layer perceptron).
Download
⬇ Matrix Multiplication (9 pages)