Adapter Layers

Fine-Tuning series: 7 of 8

Fine-Tuning Series:

1. Weight Update

2. Pretrain vs Fine-Tune

3. Full Fine-Tuning

4. Freezing Layers

5. Linear Probe

6. Feature Extraction + Head

7. Adapter Layers

8. LoRA

Houlsby et al., 2019 proposed a different strategy: instead of choosing which layers to freeze, insert small trainable modules — adapters — between every frozen layer.

Paid members: open the interactive diagram below ↓