The perceptron is the oldest idea in this series and the one the whole thing is named after. Frank Rosenblatt introduced it in 1958: weight each input, add the weighted inputs up, and fire if the total clears a threshold. One perceptron is one row of W, and it produces one number. A layer is a stack of them side by side, which is why W is a matrix rather than a list.
A single layer has two steps. First, a linear transformation: F = W × X. Then, an activation function: H = ReLU(F). The linear part is a matrix multiplication, W multiplied by X producing F. The activation applies element-wise, so every negative value in F becomes zero and everything else stays as it was, which means H has the same shape as F.
Rosenblatt's perceptron fired all or nothing, a hard threshold at zero. Modern networks use ReLU instead, and the difference matters more than it looks: a hard step has no useful slope, so there is nothing for training to follow, while ReLU is flat on one side and a straight line on the other. What we stack in a Multi-Layer Perceptron is therefore not quite Rosenblatt's unit. The name stuck, the step function did not.
The activation is also what makes the stacking worth doing. Stack matmuls without one and they collapse into a single matmul, however many we use. This is the layer I described in the introduction, and the reason deep networks can learn nonlinear patterns.
Next:
2. Wide