Sparse Auto Encoder (SAE) by Hand ✍️
Calculating AI by Hand: 14 of 28
Library › Calculating AI by Hand ✍️
Sparse Auto Encoder (SAE) by Hand ✍️
Reinforcement Learning with Human Feedback (RLHF) by Hand ✍️
A Sparse Autoencoder (SAE) maps a model's dense activations into a higher-dimensional, sparse space where individual features become interpretable. Anthropic's "Scaling Monosemanticity" showed that sparse autoencoders produce interpretable features for large models like Claude 3 Sonnet. How does an SAE achieve this?
Setup
Step 1 of 11: Given
Model activations for five tokens (X)
They work but not interpretable.
Can we map each activation (3D) to a higher dimensional space (6D) that we can interpret?
Encoder
Step 2 of 11: Linear Layer
Multiply X with encoder weights and add biases
Step 3 of 11: ReLU
Apply ReLU to add non-linearity
ReLU suppresses negative activations (set to 0).
Output: Sparse and interpretable features 𝘧
"Sparsity" means we want many zeros (21/30 here). I hand picked weight and bias values to purposely let ReLU zero out many features.
"Interpretability" is achieved when only one or two features are positive. Here, 𝘟4 and 𝘟5 both have ones only at 𝘧5. By examining the input data, we can guess what 𝘧5 may mean by checking what 𝘟4 an 𝘟5 have in common, for example, both showing a "park."