9 AI Eval Formulas You Must Know (Jan 24, 2026)

AI by Hand ✍️ Seminars

Library › Seminar Series 2026

1. Manifold-Constrained Hyper Connections (mHC) from DeepSeek (Jan 9, 2026)

2. How Small Models Learn Tool Use (Jan 11, 2026)

3. Generative AI (Jan 11, 2026)

4. Attention (Jan 15, 2026)

5. Google Ironwood TPU: From Bits to HBM (Jan 19, 2026)

6. 9 AI Eval Formulas You Must Know (Jan 24, 2026)

7. Meta Superintelligence Labs vs Facebook AI Research (Jan 30, 2026)

8. Transformer: Six Levels of Understanding (Feb 13, 2026)

9. OpenClaw Seminar (Feb 23, 2026)

10. PPO → DPO → GRPO → Rubrics (Mar 2, 2026)

11. Gemma 4 (May 7, 2026)

12. Qwen 3.6 (May 21, 2026)

In this seminar, I took on a mission: in 50 minutes, I walked through 9 core math formulas for AI evaluation, by hand, and implemented each one live in Excel.

Thank you for attending live!

Everyone agrees that AI evaluation is important. But far fewer people have actually spent time learning the math underneath it, beyond making library calls.

Most of us encounter evaluation math like this:

Others uses it. It must work, right? We move on.

But what is cross-entropy really measuring? Why does it involve a log? What number is this actually producing, and why should we trust it?

Too often, the math behind AI evaluation is explained abstractly, buried in symbols and notation that feel disconnected from intuition.

Normally, I teach these evaluation metrics separately across different courses, lectures, and seminars. This was the first time I tried to put them all together, side by side, in one simple, coherent session. I wasn’t sure it would work, but I’m glad I tried.

I’m glad to be joined by Ofer Mendelevitch from Vectara. Ofer and I have collaborated before on a seminar about agent hallucination, and I’m excited to continue that conversation. Once again, Ofer was generous in sharing his industry insight, fielding questions in the chat and connecting the ideas to real production systems.

Below is a list of the 9 formulas I covered in this seminar. If they feel overwhelming at first glance, that’s exactly why I designed this seminar.

Outline

1. Perplexity (PPL)

2. Negative Log-Likelihood

3. Cross Entropy Loss

4. ​KL Divergence

5. Accuracy

6. Precision / Recall / F1

7. ROUGE

8. BLEU

9. BERTScore

Feedback

Recording & Workbook

Become a paid subscriber to access the full recording and the associated Excel workbook.