If you’ve used any AI product, you’ve encountered tokens — either in pricing pages, API docs, or error messages about “context limits.” But what actually is a token, and why should you care?
What is a token?
A token is roughly a word fragment — about 3-4 characters on average in English. The sentence “Hello, how are you?” is about 5 tokens. A long PDF might be 50,000 tokens.
AI models process text as tokens because it’s computationally efficient. The model reads a sequence of tokens (your prompt + conversation history) and predicts the next token, one at a time, until it’s done.
This is why token count matters:
- Context window: How many tokens the model can “remember” at once
- Input cost: How much you pay for the tokens you send
- Output cost: How much you pay for the tokens the model generates (usually more expensive)
Token pricing by model
Different models have different token prices. Here’s a rough guide:
| Model | Relative Cost | Best For |
|---|---|---|
| GPT-4o | Medium | General-purpose, multimodal |
| Claude Sonnet | Medium | Reasoning, long docs |
| Gemini Flash | Low | Fast, cheap responses |
| o3 | High | Complex reasoning tasks |
Prices change frequently as providers compete. In general, costs have dropped ~10x every 18 months.
How to minimize token usage
1. Keep system prompts tight
Every message includes your system prompt. A 5,000-token system prompt costs 5x as much as a 1,000-token one — and gets sent on every single message.
2. Trim conversation history
Most apps send the full conversation history with each message. On a 50-message thread, that’s a lot of tokens. Consider summarizing older messages or truncating after a certain depth.
3. Choose the right model
Don’t use a 0.001/1K model would do fine. Use powerful models for complex tasks, cheap models for routing, classification, and simple Q&A.
4. Cache repeated context
Many providers support “prompt caching” — if the beginning of your prompt is identical across requests, you only pay for it once. Structure your system prompt so the static parts come first.
Context windows: why they matter
Every model has a maximum context window — the maximum number of tokens it can process at once. Current maximums:
- GPT-4o: 128K tokens (~100,000 words)
- Claude Sonnet: 200K tokens (~150,000 words)
- Gemini 2.5 Pro: 1M tokens (~750,000 words)
For most chatbots, 128K is plenty. For apps processing entire books, legal contracts, or codebases, you’ll want a larger context.
Tokens and AI agents
When AI agents use tools (web search, code execution, file reading), each tool call adds tokens. A single agent run might consume:
- 2K tokens: your prompt
- 15K tokens: search results
- 5K tokens: reasoning about those results
- 1K tokens: final response
= ~23K tokens per “smart” response vs. ~3K for a direct answer
This is worth knowing when you’re designing agent-powered apps. Sometimes the simple answer is cheaper and better.
Practical advice for Chipp builders
On Chipp, your token consumption is logged per app and per session. You can:
- See which apps drive the most cost
- Identify unusually expensive sessions (often from large file uploads)
- Set spending caps per app to prevent runaway costs
If you’re building a consumer-facing app, token costs translate directly to per-message economics. A message that costs 0.05/message, is a real business.
Understanding tokens isn’t optional for AI builders — it’s fundamental. The good news: once you get it, pricing becomes predictable and controllable.