Matrix Multiplication by Hand ✍️
One of the key ingredients of the deep learning revolution is the ability to perform matrix multiplications at incredible speed and scale.
Despite linear algebra listed as a prerequisite of my courses, I found many students lack confidence in multiplying matrices. Thus, I teach my students this unique system to multiply matrices by hand.
Let's say we are calculating A x B = C. This system offers several desirable properties:
💡 𝗗𝗶𝗺𝗲𝗻𝘀𝗶𝗼𝗻𝘀: We can easily see C's dimensions must match A's height (vertical) and B's width (horizontal).
💡 𝗦𝗰𝗮𝗹𝗮𝗯𝗹𝗲: If we scale any of the matrices A, B, C to an arbitrary size, we can easily see how the sizes of other matrices must change accordingly to match the dimensions.
💡 𝗥𝗼𝘄 𝘃𝘀. 𝗖𝗼𝗹𝘂𝗺𝗻 𝗩𝗲𝗰𝘁𝗼𝗿𝘀: For each value in C, we can easily see the corresponding row vector (green) and column vector (yellow) that are combined using dot-product to obtain the value.
💡 𝗦𝘁𝗮𝗰𝗸𝗮𝗯𝗹𝗲: Because of their compactness, we can stack multiple grids like these to represent the underlying sequence of matrix multiplications of a deep neural network (e.g., multi-layer perceptron), like the one I shared a while ago.
One month ago, I decided to share my hands-on exercises for deep learning and generative AI. I was surprised by how well those exercises are received by the community. Thank you!
In 2024, I will continue to share more exercises I developed. In my first post of the year, I want to go back to the most basic and the absolutely important---matrix multiplication! Enjoy!
