How Long Will It Take?

Token problems: 17 of 20

Token problems: the workbook

1. Count the Tokens

2. Subword Splitting

3. Punctuation Counts

4. Tokens per Word

5. Will It Fit?

6. The Price of a Call

7. Two Kinds of Token

8. Input or Output?

9. The Two-Part Bill

10. End-to-End Call Cost

11. Comparing Three Calls

12. The Average Call

13. The System Prompt

14. The System Prompt Tax

15. Calls in a Budget

16. Tokens on a Budget

17. How Long Will It Take?

18. The Split Matters

19. Pricing per Million Tokens

20. Tokens from the Bill

Why does a short reply sometimes take longer than reading a large document? A call's time has two parts. Input is processed in parallel, every token in one pass, so even a large input clears quickly. Output is generated one token at a time, each waiting on the one before. For agents, this means latency is driven almost entirely by how much the agent writes, not by how much it reads.

Problem

A call sends 2,000 input tokens and gets back 200 output tokens. The model reads input at 1,000 tokens per second and writes output at 40 tokens per second. How long does the call take?

Time to read the input

Time to write the output

Add the two times

Practice 1

A call sends 3,000 input tokens and gets back 300 output tokens. The model reads input at 1,000 tokens per second and writes output at 50 tokens per second. How long does the call take?

Time to read the input

Time to write the output

Add the two times

Practice 2

A call sends 4,000 input tokens and gets back 120 output tokens. The model reads input at 2,000 tokens per second and writes output at 30 tokens per second. How long does the call take?

Time to read the input

Time to write the output

Add the two times

---

Next:
18. The Split Matters