Self Attention - Spreadsheet

Make your own AI by hand ✍️ exercises

🔗 Google Sheet: https://by-hand.ai/s/self

I’d like to share with you a customized spreadsheet version of the self-attention by hand exercise I posted a few months ago. Now you can finally use our own numbers and see the calculations.

In this version, I introduced two new features:

1. I added the famous QKV diagram in the right to help you see the correspondences between my matrix representation and the graphical representation of the self attention mechanism. You may notice that I turned the diagram upside down to better match the natural flow of matrix multiplications.

2. I added the equations as in the paper, with slight modifications for simplification, next to each key calculation step.

Hope this resource helps you understand the self attention mechanism better!