This is the other half of the pair I started in the encoder. A decoder is an MLP that gets wider. It starts from the 8 numbers an encoder produced and works back out: 8 to 16, 16 to 32, 32 to 64.
Nothing is retrieved here. The original 64 features are gone, and the decoder never sees them. Everything in the output was rebuilt from the code, using weights that learned what a plausible reconstruction looks like.
That is the whole trick behind generation. If we give a decoder a code it has never seen, it will still produce something full-width and plausible, which is what the generator does in a GAN.
Next:
7. Autoencoder