Mistral 3 Live Seminar!
Frontier AI by hand ✍️
Last week, Mistral 3 was a big release. The very first AI by Hand walkthrough—back in 2023—was on MoE, inspired by the original Mistral model release. Can’t believe it has been two years!
So for this week’s Frontier issue, I ran a live session with Frontier members to walk through Mistral 3.
It’s the perfect opportunity to do something special: showing one method I use to study frontier model architectures myself, and connecting this new model to concepts we’ve already explored in the Frontier series.
When a new frontier model comes out—especially when there are no papers or blog posts available yet—one method I use is to start with the Hugging Face repository itself. I treat the files the model ships with as the ground truth and try to reconstruct the architecture directly from them.
By reading configuration files and inspecting weight names and shapes, I map numbers back onto an architecture blueprint by hand. The goal isn’t to reverse-engineer everything perfectly, but to build intuition—learning how much you can understand about a model just from the artifacts it leaves behind.
In this live session, we started with going through safetensor.index.json and see to what extend we can reverse-engineer the architecture just from the weight names and shapes in this file alone.
Then we dove into params.json, unpacking each parameter and matching the numbers back to the underlying architecture blueprint.
Basically, I’ve been handing out fish—but this time, I showed you one way to catch your own fish by hand. ✍️ 🎣
⬇️ Watch the full recording and download the Excel Blueprint (Frontier subscribers only).
(P.S. This issue is written for advanced AI engineers and researchers. It is part of the premium Frontier subscription. If you are a beginner, please check out my free Foundation series, lectures, walkthroughs, and Excel exercises.)