AI by Hand ✍️

Frontier AI Drawings: the series

Prof. Tom Yeh — Sun, 31 May 2026 13:39:53 GMT

I’d like to share a new edition of a series of hand-drawn illustrations I originally shared with members from August through October last year. For this edition, I revised the writing, refined the ex…

Token problems: the workbook

Prof. Tom Yeh — Thu, 28 May 2026 13:17:58 GMT

I designed this workbook to get your hands busy with AI agents. This is the first in a series I plan to develop and share with you over the summer.

This is not the first time I have tried to make you pick up a pen and write on paper by hand. ✍️ In my previous Agentic AI online seminars, I asked you to sketch diagrams, draw lines, and many of you did.

This time I want to bring it up a notch: can I get you to do some math? Don't worry, simple math: adding numbers, multiplying them, working things in reverse. The goal is to build real intuition about how agents work by solving problems by hand, rather than passively reading and watching or mindlessly vide-coding. 😂

This first workbook is about tokens. The next one will cover the context window.

Now let's see if you can solve all 20 problems.

1. Count the Tokens

Read Count the Tokens →

2. Subword Splitting

Read Subword Splitting →

3. Punctuation Counts

Read Punctuation Counts →

4. Tokens per Word

Read Tokens per Word →

5. Will It Fit?

Read Will It Fit? →

6. The Price of a Call

Read The Price of a Call →

7. Two Kinds of Token

Read Two Kinds of Token →

8. Input or Output?

Read Input or Output? →

9. The Two-Part Bill

Read The Two-Part Bill →

10. End-to-End Call Cost

Read End-to-End Call Cost →

11. Comparing Three Calls

Read Comparing Three Calls →

12. The Average Call

Read The Average Call →

13. The System Prompt

Read The System Prompt →

14. The System Prompt Tax

Read The System Prompt Tax →

15. Calls in a Budget

Read Calls in a Budget →

16. Tokens on a Budget

Read Tokens on a Budget →

17. How Long Will It Take?

Read How Long Will It Take? →

18. The Split Matters

Read The Split Matters →

19. Pricing per Million Tokens

Read Pricing per Million Tokens →

20. Tokens from the Bill

Read Tokens from the Bill →

Tokens From the Bill

Prof. Tom Yeh — Thu, 28 May 2026 12:43:59 GMT

Given a bill, how do you work out how many tokens were used? A real model bill lists input and output on separate lines, each priced per million tokens. Divide each line's cost by that side's price per million, then scale up to tokens. Agent operators use this to audit usage after the fact, confirming that the bill matches what the logs show.

Problem

An agent task's bill shows $12 for input and $30 for output. Input is priced at $3 per million tokens, output at $15 per million. How many tokens did the task process in total?

Input tokens from the input charge

Output tokens from the output charge

Add for the total tokens

Practice 1

An agent task's bill shows $35 for input and $45 for output. Input is priced at $5 per million tokens, output at $15 per million. How many tokens did the task process in total?

Input tokens from the input charge

Output tokens from the output charge

Add for the total tokens

Practice 2

An agent task's bill shows $6 for input and $50 for output. Input is priced at $2 per million tokens, output at $10 per million. How many tokens did the task process in total?

Input tokens from the input charge

Output tokens from the output charge

Add for the total tokens

Pricing Per Million Tokens

Prof. Tom Yeh — Thu, 28 May 2026 12:43:39 GMT

How does real model pricing work? Every major API quotes its price per million tokens, with input and output on separate lines. Output costs several times more than input. To find the total cost, price each side on its own, then add. When comparing models for an agent, these two numbers together determine the real cost per call.

Problem

A model charges $3 per million input tokens and $15 per million output tokens. A task processes 2,000,000 input tokens and 1,000,000 output tokens. What does the task cost?

Cost of the input tokens

Cost of the output tokens

Add for the total cost

Practice 1

A model charges $4 per million input tokens and $12 per million output tokens. A task processes 3,000,000 input tokens and 2,000,000 output tokens. What does the task cost?

Cost of the input tokens

Cost of the output tokens

Add for the total cost

Practice 2

A model charges $2 per million input tokens and $10 per million output tokens. A task processes 4,000,000 input tokens and 1,000,000 output tokens. What does the task cost?

Cost of the input tokens

Cost of the output tokens

Add for the total cost

Next:
20. Tokens from the Bill

The Split Matters

Prof. Tom Yeh — Thu, 28 May 2026 12:43:35 GMT

Can two calls with the same total tokens finish at very different times? Yes. Input is read in one parallel pass; output is written one token at a time. A task that reads a lot and writes a little will almost always finish faster than one that reads a little and writes a lot. Agents designed to summarize or classify can be significantly faster than agents designed to generate long replies, even at the same token budget.

Problem

A model reads input at 500 tokens per second and writes output at 50 tokens per second. Task A summarizes a long document: 2,000 input tokens, 500 output tokens. Task B expands a short outline into a full article: 500 input tokens, 2,000 output tokens. Both tasks process 2,500 tokens in total. Which finishes first?

Time for task A

Time for task B

Compare

Practice 1

A model reads input at 500 tokens per second and writes output at 50 tokens per second. Task A answers a question about a long report: 2,500 input tokens, 500 output tokens. Task B drafts a detailed report from a short brief: 500 input tokens, 2,500 output tokens. Both tasks process 3,000 tokens in total. Which finishes first?

Time for task A

Time for task B

Compare

Practice 2

A model reads input at 500 tokens per second and writes output at 50 tokens per second. Task A classifies a batch of messages: 1,500 input tokens, 500 output tokens. Task B writes replies from short descriptions: 500 input tokens, 1,500 output tokens. Both tasks process 2,000 tokens in total. Which finishes first?

Time for task A

Time for task B

Compare

Next:
19. Pricing per Million Tokens

How Long Will It Take?

Prof. Tom Yeh — Thu, 28 May 2026 12:43:15 GMT

Why does a short reply sometimes take longer than reading a large document? A call's time has two parts. Input is processed in parallel, every token in one pass, so even a large input clears quickly. Output is generated one token at a time, each waiting on the one before. For agents, this means latency is driven almost entirely by how much the agent writes, not by how much it reads.

Problem

A call sends 2,000 input tokens and gets back 200 output tokens. The model reads input at 1,000 tokens per second and writes output at 40 tokens per second. How long does the call take?

Time to read the input

Time to write the output

Add the two times

Practice 1

A call sends 3,000 input tokens and gets back 300 output tokens. The model reads input at 1,000 tokens per second and writes output at 50 tokens per second. How long does the call take?

Time to read the input

Time to write the output

Add the two times

Practice 2

A call sends 4,000 input tokens and gets back 120 output tokens. The model reads input at 2,000 tokens per second and writes output at 30 tokens per second. How long does the call take?

Time to read the input

Time to write the output

Add the two times

Next:
18. The Split Matters

Tokens on a Budget

Prof. Tom Yeh — Thu, 28 May 2026 12:43:12 GMT

Given a budget, how many tokens can one call afford? The earlier pricing problems ran forward: tokens in, cost out. This one runs backward: cost in, tokens out. Divide the budget by the price per block, then scale up to tokens. Agents with per-call cost limits use this calculation to set a maximum context size before each turn.

Problem

You have a budget of 50 cents for one call. The model is priced at 2 cents per 100 tokens. How many tokens can the call use?

How many 100-token blocks the budget buys

Convert the blocks to tokens

Practice 1

You have a budget of 90 cents for one call. The model is priced at 3 cents per 100 tokens. How many tokens can the call use?

How many 100-token blocks the budget buys

Convert the blocks to tokens

Practice 2

You have a budget of 60 cents for one call. The model is priced at 4 cents per 100 tokens. How many tokens can the call use?

How many 100-token blocks the budget buys

Convert the blocks to tokens

Next:
17. How Long Will It Take?

Calls in a Budget

Prof. Tom Yeh — Thu, 28 May 2026 12:42:53 GMT

Given a fixed budget, how many calls can you make? Find the cost of one call first, then divide the budget by that cost. The result is the hard ceiling on how much work can be done in a session. For agents running autonomously, this ceiling determines how many steps the agent can take before it must stop.

Problem

An AI agent has a daily budget of 240 cents. Each call has 400 input tokens and 100 output tokens. Input is priced at 2 cents per 100 tokens; output at 8 cents per 100 tokens. How many calls can it make in a day?

Cost per call

Calls in the budget

Practice 1

An AI agent has a daily budget of 440 cents. Each call has 300 input tokens and 200 output tokens. Input is priced at 2 cents per 100 tokens; output at 8 cents per 100 tokens. How many calls can it make in a day?

Cost per call

Calls in the budget

Practice 2

An AI agent has a daily budget of 280 cents. Each call has 200 input tokens and 300 output tokens. Input is priced at 2 cents per 100 tokens; output at 8 cents per 100 tokens. How many calls can it make in a day?

Cost per call

Calls in the budget

Next:
16. Tokens on a Budget

The System Prompt Tax

Prof. Tom Yeh — Thu, 28 May 2026 12:42:50 GMT

What does it cost to repeat the same instructions on every call? The model has no memory between turns, so the system prompt is re-sent in full each time, adding a fixed cost regardless of what the user asks. A longer system prompt means a larger per-call tax. For agents with detailed instructions, this fixed cost can add up to a significant share of the total bill.

Problem

An AI agent's system prompt is 250 tokens. Over 10 calls, at 2 cents per 100 tokens, what do those system-prompt tokens cost?

Total system-prompt tokens over all calls

Price those tokens

Practice 1

An AI agent's system prompt is 300 tokens. Over 20 calls, at 2 cents per 100 tokens, what do those system-prompt tokens cost?

Total system-prompt tokens over all calls

Price those tokens

Practice 2

An AI agent's system prompt is 400 tokens. Over 5 calls, at 3 cents per 100 tokens, what do those system-prompt tokens cost?

Total system-prompt tokens over all calls

Price those tokens

Next:
15. Calls in a Budget

The System Prompt

Prof. Tom Yeh — Thu, 28 May 2026 12:42:46 GMT

Why does every call start with the same block of tokens? The model has no memory between turns, so the agent re-sends its instructions at the front of every call. That block is the system prompt: it defines how the agent behaves, and it costs tokens every single time. Labeling where it ends and the user message begins makes the per-call overhead visible.

Problem

The model replied bonjour to the user's message translate hello into french. The tokens before the user message are the system prompt. Label each token S for system prompt, U for user message, or O for output.

Label each token S, U, or O

Count the tokens of each kind

Practice 1

The model replied here is the fix to the user's message fix the login bug. The tokens before the user message are the system prompt. Label each token S for system prompt, U for user message, or O for output.

Label each token S, U, or O

Count the tokens of each kind

Practice 2

The model replied it is three pm now to the user's message what is the time. The tokens before the user message are the system prompt. Label each token S for system prompt, U for user message, or O for output.

Label each token S, U, or O

Count the tokens of each kind

Next:
14. The System Prompt Tax

The Average Call

Prof. Tom Yeh — Thu, 28 May 2026 12:42:42 GMT

How do you estimate total usage when every call is different? Average a sample of recent calls to get a representative size, then multiply by the expected volume. Plans and budgets scale from the average, not from any single call. Agents running at production scale need this estimate to set daily budgets and catch cost spikes early.

Problem

An AI agent handled three sample calls of 400, 500, and 600 tokens. It will run 200 calls today. Using the average call size, estimate the total tokens for the day.

Average the sample calls

Multiply the average by the day's calls

Practice 1

An AI agent handled three sample calls of 300, 600, and 900 tokens. It will run 100 calls today. Using the average call size, estimate the total tokens for the day.

Average the sample calls

Multiply the average by the day's calls

Practice 2

An AI agent handled four sample calls of 200, 400, 400, and 600 tokens. It will run 500 calls today. Using the average call size, estimate the total tokens for the day.

Average the sample calls

Multiply the average by the day's calls

Next:
13. The System Prompt

Comparing Three Calls

Prof. Tom Yeh — Thu, 28 May 2026 12:42:38 GMT

Which call costs more when total tokens are the same? You have to cost all three to find out. The split between input and output matters more than the total: a call heavy on input can cost far less than one heavy on output, even with identical token counts. Agents face this trade-off every time they choose how much context to include and how long a reply to request.

Problem

Three calls each process 1,000 tokens, but split them differently. Input is priced at 2 cents per 100 tokens; output at 8 cents per 100 tokens. Fill in the cost for each call, then rank them cheapest to most expensive.

Fill the cost column

Rank cheapest to most expensive

Practice 1

Three calls each process 1,200 tokens, but split them differently. Input is priced at 2 cents per 100 tokens; output at 8 cents per 100 tokens. Fill in the cost for each call, then rank them cheapest to most expensive.

Fill the cost column

Rank cheapest to most expensive

Practice 2

Three calls each process 1,500 tokens, but split them differently. Input is priced at 2 cents per 100 tokens; output at 8 cents per 100 tokens. Fill in the cost for each call, then rank them cheapest to most expensive.

Fill the cost column

Rank cheapest to most expensive

Next:
12. The Average Call

End-to-End Call Cost

Prof. Tom Yeh — Thu, 28 May 2026 12:29:54 GMT

How do you price a call when you only know the word count? Estimate input tokens from the words, then price input and output separately. Get the token counts right and the cost follows. In practice, agent prompts arrive as natural language, so word-to-token estimation is the first step in any cost calculation.

Problem

An AI agent receives a 200-word prompt and returns 200 output tokens. Input tokens are estimated at 1.5 per word. Input is priced at 2 cents per 100 tokens and output at 8 cents per 100 tokens. What does the call cost in total?

Estimate the input tokens

Cost of the input tokens

Cost of the output tokens

Add the two costs

Practice 1

An AI agent receives a 100-word prompt and returns 600 output tokens. Input tokens are estimated at 2.0 per word. Input is priced at 2 cents per 100 tokens and output at 8 cents per 100 tokens. What does the call cost in total?

Estimate the input tokens

Cost of the input tokens

Cost of the output tokens

Add the two costs

Practice 2

An AI agent receives a 200-word prompt and returns 100 output tokens. Input tokens are estimated at 2.5 per word. Input is priced at 2 cents per 100 tokens and output at 8 cents per 100 tokens. What does the call cost in total?

Estimate the input tokens

Cost of the input tokens

Cost of the output tokens

Add the two costs

Next:
11. Comparing Three Calls

The Two-Part Bill

Prof. Tom Yeh — Thu, 28 May 2026 12:29:50 GMT

Why does the same token count sometimes cost more? Because input and output are billed separately, at different rates. Input is read in a single pass; output is generated one token at a time, which is the expensive part. An agent that writes long replies will always cost more than one that reads the same amount and answers briefly.

Problem

An AI agent sends 600 input tokens and receives 200 output tokens. Input is priced at 2 cents per 100 tokens; output is priced at 8 cents per 100 tokens. What does the call cost in total?

Cost of the input tokens

Cost of the output tokens

Add the two costs

Practice 1

An AI agent sends 1,000 input tokens and receives 500 output tokens. Input is priced at 2 cents per 100 tokens; output is priced at 8 cents per 100 tokens. What does the call cost in total?

Cost of the input tokens

Cost of the output tokens

Add the two costs

Practice 2

An AI agent sends 400 input tokens and receives 300 output tokens. Input is priced at 2 cents per 100 tokens; output is priced at 8 cents per 100 tokens. What does the call cost in total?

Cost of the input tokens

Cost of the output tokens

Add the two costs

Next:
10. End-to-End Call Cost

Input or Output?

Prof. Tom Yeh — Thu, 28 May 2026 12:29:31 GMT

Are all the tokens in a call billed the same way? No. Every token is either input (what you send) or output (what the model generates), each billed at its own rate. The boundary between them is the dividing line between two different prices. Labeling each token is the first step to seeing where an agent's cost comes from.

Problem

A model receives the question what is the plan and replies first gather data then analyze results. Label each token I for input or O for output.

Label each token I or O

Count the tokens of each kind

Practice 1

A model receives the task summarize the meeting notes and replies the team agreed to ship by Friday. Label each token I for input or O for output.

Label each token I or O

Count the tokens of each kind

Practice 2

A model receives the request list the tasks assigned to Alice and replies Alice has three open tasks. Label each token I for input or O for output.

Label each token I or O

Count the tokens of each kind

Next:
9. The Two-Part Bill

Two Kinds of Token

Prof. Tom Yeh — Thu, 28 May 2026 12:29:27 GMT

Are all tokens the same? No. There are two kinds: input tokens (what you send) and output tokens (what the model generates, one at a time). Both are counted the same way, but output tokens cost more. Every agent call produces both: the prompt is input, the reply is output.

Problem

An AI agent is sent the input why do cats purr? and returns the output a cat purrs when it feels happy!. Tokenize the input and the output, then count the tokens of each kind.

Tokenize the input

Tokenize the output

Count the tokens of each kind

Practice 1

An AI agent is sent the input how many moons does the planet Jupiter have? and returns the output the planet Jupiter has ninety five moons!. Tokenize the input and the output, then count the tokens of each kind.

Tokenize the input

Tokenize the output

Count the tokens of each kind

Practice 2

An AI agent is sent the input is the sun a star? and returns the output yes, the sun is a star.. Tokenize the input and the output, then count the tokens of each kind.

Tokenize the input

Tokenize the output

Count the tokens of each kind

Next:
8. Input or Output?

The Price of a Call

Prof. Tom Yeh — Thu, 28 May 2026 12:29:08 GMT

What does a model API call cost? Every call is billed by the token: count the 100-token blocks in the call, then multiply by the price per block. Real APIs quote per million tokens, but these problems use cents per 100 to keep the arithmetic easy to follow by hand. An agent that calls the model on every user turn pays this cost each time.

Problem

An AI agent processes a call of 600 tokens. The model is priced at 2 cents per 100 tokens. What does this call cost?

Count the 100-token blocks

Multiply by the price per block

Practice 1

An AI agent processes a call of 2,000 tokens. The model is priced at 3 cents per 100 tokens. What does this call cost?

Count the 100-token blocks

Multiply by the price per block

Practice 2

An AI agent processes a call of 800 tokens. The model is priced at 5 cents per 100 tokens. What does this call cost?

Count the 100-token blocks

Multiply by the price per block

Next:
7. Two Kinds of Token

Will It Fit?

Prof. Tom Yeh — Thu, 28 May 2026 12:28:49 GMT

Can an agent always read a document it is given? Not if the document exceeds the space that remains in the context window. The window is shared: the system prompt and conversation history already occupy part of it. Estimate the document size first, then compare it to what is left. If the estimate exceeds the remaining space, the agent cannot read it in one call.

Problem

An agent has a 4,000-token context window. The system prompt and conversation history already use 1,200 tokens. The document to read is 55 paragraphs, each about 50 tokens. Can the agent read the document in this call?

Space available for the document

Estimate the document

Does it fit?

Practice 1

An agent has a 4,000-token context window. The system prompt and conversation history already use 600 tokens. The document to read is 8 pages, each about 500 tokens. Can the agent read the document in this call?

Space available for the document

Estimate the document

Does it fit?

Practice 2

An agent has a 4,000-token context window. The system prompt and conversation history already use 1,000 tokens. The document to read is 18 slides, each about 150 tokens. Can the agent read the document in this call?

Space available for the document

Estimate the document

Does it fit?

Next:
6. The Price of a Call

Tokens per Word

Prof. Tom Yeh — Thu, 28 May 2026 12:28:42 GMT

Does every language tokenize the same way? No. The ratio of tokens to words varies because tokenizers are trained mostly on English. Agents serving multilingual users will pay different costs for the same message depending on the language.

Problem

An AI agent processes a 20-word passage of English text and counts 26 tokens. What is the token-to-word ratio?

Divide tokens by words

Practice 1

An AI agent processes a 24-word passage of Spanish text and counts 36 tokens. What is the token-to-word ratio?

Divide tokens by words

Practice 2

An AI agent processes a 15-word passage of Hindi text and counts 45 tokens. What is the token-to-word ratio?

Divide tokens by words

Next:
5. Will It Fit?

Punctuation Counts

Prof. Tom Yeh — Thu, 28 May 2026 10:48:52 GMT

Does punctuation count as a token? Yes. Most tokenizers treat each mark as its own token, carrying structure the model needs: sentence boundaries, pauses, questions. Agents that read or write code, JSON, or structured data pay this cost on every bracket and comma.

Problem

An AI agent counts each punctuation mark as its own token. The agent receives Hello, world!. How many tokens does it count?

Write each token in a box

Count the filled boxes

Practice 1

An AI agent counts each punctuation mark as its own token. The agent receives Wait, what?. How many tokens does it count?

Write each token in a box

Count the filled boxes

Practice 2

An AI agent counts each punctuation mark as its own token. The agent receives Stop, look, and listen!. How many tokens does it count?

Write each token in a box

Count the filled boxes

Next:
4. Tokens per Word