Activation: the series
4 interactive lessons
The Activation series
Activation functions are the small nonlinear hinges that let neural networks learn non-trivial patterns. Without them, no matter how deep the network, the whole thing collapses to a single linear map.
This series introduces four of them through one running scene — five boba tea shops on the same block.
1. ReLU
2. Leaky ReLU
3. Softmax
4. Sigmoid
