Sharding Blueprint

Frontier AI by hand ✍️

Matrix Multiplication Sharding: Out-Dimension x In-Dimension (2 x 2)

Last week, you may have heard the news that Google is negotiating with Meta to supply TPUs for its new data centers—an investment …