Input vs output tokens
Input tokens are what you send to the model; output tokens are what it sends back. Providers charge different rates for each — output usually costs more.
When you send a message to an AI model, the model reads it (input tokens) and generates a response (output tokens). Both count against usage, metered separately because they cost the provider different amounts.
Output tokens are typically 3-5x more expensive than input. The reason: generating each token requires running the full model forward once, while input tokens can be processed in parallel. A 5K-input, 500-output exchange often has output dominating the cost even though it's 10x smaller.
Practical implications: - **Long documents are cheap to read; long responses are expensive to generate.** Pasting a 200-page PDF for a short answer costs less than asking for a 40-page report from scratch. - **Verbose responses waste money.** If you don't need the model to think out loud for 800 words, ask for a shorter format explicitly. - **Streaming doesn't reduce cost.** Whether the response arrives instantly or streams over 5 seconds, the output-token count is the same.
For usage-metered subscriptions, both count toward your cap. For raw API pricing (Anthropic, OpenAI), the split is explicit — input rate ($/1M) and output rate ($/1M) are listed separately.
If you're building against an API and want to optimize cost: (a) reduce output length via specific-shape prompts, (b) use prompt caching for repeated identical input across many requests.