AI by Hand ✍️

AI by Hand ✍️

Cost Forecast

Budgeting a Long Session

Prof. Tom Yeh's avatar
Prof. Tom Yeh
Aug 19, 2026
∙ Paid

Library › Context Problems

  1. The Context Window

  2. Window Occupancy

  3. Growth per Turn

  4. Remaining Space

  5. Turn Budget

  6. Retrieval Footprint

  7. Retrieval per Turn

  8. Search Rounds

  9. Window Sizing

  10. Truncation

  11. Compaction

  12. The Compaction Bill

  13. Compacted Retrieval

  14. The Compaction Threshold

  15. Verbatim Tail

  16. Stateless API

  17. Retrieval Cost

  18. Context Tax

  19. Quadratic Cost

  20. Cost Forecast

One turn's input grows by a fixed step, so the tenth turn sends ten times what the first did. That much is a straight line. But the bill is not the last turn, it is every turn added together, and adding up a straight line gives a curve. That is why doubling the length of a conversation does not double what it costs to run, it roughly quadruples it. The output never joins that curve. It is the same size every turn, so adding it up stays a straight line, and the input half of the bill runs away from it however dearly the output is priced.

Paid members: the worksheet and its printable PDF are below ↓

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2026 Tom Yeh · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture