Library › Counting AI by Hand ✍️
Agent Tokens (Jul 14, 2026)
Thank you for attending the second lecture of Counting AI by Hand ✍️, and for those who could not join us live, here is the recording.
The first lecture counted the operations inside the model. This one counts the tokens moving through an application, which is where the question that started the series finally gets a direct answer.
Here are some of the counting questions from this lecture.
[Input vs Output] Why does the text the model writes cost more than the text you typed?
[Conversation History] What makes the same short question cost more late in a chat than at the start?
[Tool Calls] Which costs more, three turns of plain chat or two turns that call a tool?
[RAG] How many times do you pay for a chunk your retriever keeps finding?
[Multi-Agent] What does it cost when every agent in a system keeps its own conversation?
Mohsena works through every one of them in Excel, counting by hand. The cells hold tokens this time, and the total is your bill.
Outline
Introduction (00:00)
Tokens basics, and why output costs more (01:51)
FLOPs vs tokens
What a user sees
What actually happens: system prompt + history + tool results = input
Chat history token counting (08:01)
Basic chat, priced per turn
Tool calls and token billing (18:07)
Tool definitions, the call, the output
Why tool calls cost more (30:24)
RAG workflow and token math (32:15)
The same chunk, retrieved twice and paid for twice
Naive vs optimized: what has to be in the next request
RAG variations and other inputs (47:15)
Image + text
Memory and multi-agent costs (48:27)
Two sessions, and what the application stores between them
Planner and specialists, each with their own conversation
About the instructor
Mohsena Ashraf is a PhD student in computer science at CU Boulder, with a research focus in HCI. Her work is on explainable AI, and specifically on making AI's cost legible to people who are not engineers.
Is it just me having issues opening the Excel workbooks?