Pretrain vs Fine-Tune

Fine-Tuning series: 2 of 8

Library › Fine-Tuning

1. Weight Update

2. Pretrain vs Fine-Tune

3. Full Fine-Tuning

4. Freezing Layers

5. Linear Probe

6. Feature Extraction + Head

7. Adapter Layers

8. LoRA

The previous lesson showed W + ΔW = W₁ as a single abstract step. That same step shows up in two very different settings, and the setting is what separates pretraining from fine-tuning.

Paid members: the full breakdown continues below ↓