AI Foundations 3 - Tokens and Pricing

Models don’t read words the way you do. They break text into chunks called tokens, and a token isn’t the same thing as a word. “Cat” might be one token. “Unbelievable” might split into two or three. Code is chunked the same way — brackets, keywords, and indentation all count.

Why tokens matter

Every token costs something, and every token takes time to produce. A short prompt on a small file runs fast and cheap. Paste in a ten-thousand-line log file and ask for a summary, and you’ll feel both the wait and the bill.

Input and output tokens

Input tokens are everything you send in: your prompt, any files, the conversation so far. Output tokens are what the model generates back. They’re counted and priced separately, and they behave differently — one you control directly, the other you only shape indirectly by asking for shorter or longer answers.

Pricing

Providers charge per token, usually in price-per-million units, and output tokens almost always cost more per token than input tokens. Makes sense — generating text is the harder, slower half of the job.

Streaming

Rather than making you wait for the entire response, most models can stream tokens out one at a time as they’re generated. That’s why chat interfaces show text appearing word by word instead of all at once. It doesn’t make the model faster overall, but it makes the wait feel shorter.

Token optimization

A few habits keep both cost and latency down:

• Don’t paste in more context than the task actually needs.
• Reuse a stable prompt prefix where the provider supports caching it — repeated setup shouldn’t be repriced every call.
• Ask explicitly for a short answer when a short answer is all you need. Models default to being thorough, which is usually not what you’re paying for.

Previous: AI Foundations 2 - Hallucinations and Limitations Next: AI Foundations 4 - Context