The System Prompt Tax
Token problems: 14 of 20
Token problems: the workbook
The System Prompt Tax
What does it cost to repeat the same instructions on every call? The model has no memory between turns, so the system prompt is re-sent in full each time, adding a fixed cost regardless of what the user asks. A longer system prompt means a larger per-call tax. For agents with detailed instructions, this fixed cost can add up to a significant share of the total bill.
Problem
An AI agent's system prompt is 250 tokens. Over 10 calls, at 2 cents per 100 tokens, what do those system-prompt tokens cost?
Total system-prompt tokens over all calls
Price those tokens
Practice 1
An AI agent's system prompt is 300 tokens. Over 20 calls, at 2 cents per 100 tokens, what do those system-prompt tokens cost?
Total system-prompt tokens over all calls
Price those tokens
Practice 2
An AI agent's system prompt is 400 tokens. Over 5 calls, at 3 cents per 100 tokens, what do those system-prompt tokens cost?
Total system-prompt tokens over all calls
Price those tokens
