The idea
Models don't see letters or words. They see tokens: reusable chunks of text, usually a common word or part of a word. Rule of thumb in English: 1 token ≈ ¾ of a word, so 100 words ≈ 130 tokens. Other languages, code and unusual strings often take more tokens per word.
Mental model
Tokens are the component library of language: not raw pixels, not a bespoke component for every screen, but a set of reusable pieces.
Why it matters when you build
| Measured in tokens | What it means for you |
|---|---|
| Cost | APIs charge per input token and per output token (output is usually pricier). |
| Speed | Generation time grows roughly with the number of output tokens. Shorter answers are faster. |
| Limits | The context window and the maximum output length are token budgets. |
- Estimate a feature's cost as (input tokens + output tokens) × price × calls per user.
- Asking for concise output is a real speed and cost lever, not just a style preference.
Gotchas
- Models can be oddly bad at letter-level tasks (counting letters, spelling backwards) because they never see individual letters, only tokens.