Multi Layer Perceptron (MLP) by Hand ✍️
Calculating AI by Hand: 2 of 28
Library › Calculating AI by Hand ✍️
Multi Layer Perceptron (MLP) by Hand ✍️
Reinforcement Learning with Human Feedback (RLHF) by Hand ✍️
One of the struggles to teach a large course on deep learning is to figure out how to set up a unified coding environment for all the students. Should students use our department's own cloud infrastructure? Should students use the free Google Colab, Hugging Face, AWS? Should students install the environment in their own laptops (i.e., BYOD)?
Questions like these are important. Without a unified environment, it would be a nightmare for me and my TAs to support the variety of environments students may be using. But, these questions often distract us from the main goal of learning the key programming concepts.
Is it possible to go old-school? Can students practice coding a deep learning framework using pen and paper, but still connecting theories to practices in a meaningful way? It would certainly lower the barrier of entry to learning AI, if we don't even need computers.
Here's a hands-on coding exercise I created for this purpose.
Setup
Step 1 of 7: Given
Given a code template (left), implement the multi layer perception as depicted (right).
First Layer
Step 2 of 7: Linear Layer
The size of input features is 3. The size of output features is 4. We can see the size of the weight matrix is 4 by 3. Also, there is an extra column for the biases (bias = T).
Step 3 of 7: ReLU
The activation function is ReLU. We can see the effect of ReLU on the first feature (-1 -> 0).