GELU (Gaussian Error Linear Unit) is SiLU's more decisive sibling: same x · gate structure, but the gate now uses the Gaussian CDF Φ(x) instead of sigmoid σ(x). That swap is what made GELU the activation across BERT, GPT-2/3, T5, and ViT.
Φ has a clean approximation, σ(1.702x), so we can compare the two activations through a single shared lens: both run the same sigmoid gate, just with different inputs feeding it.
Members read on: the worked example and the full walkthrough.