AI Foundations 4 - Context

Everything a model can see for the current task lives in its context — think of it as short-term memory that gets rebuilt fresh each time you ask something. Nothing outside that window exists to the model, no matter how obvious it seems to you.

System and user prompts

Most setups split instructions into two layers. A system prompt sets the general rules — tone, role, boundaries — and usually stays fixed across a whole session. A user prompt is the specific ask for this particular turn. Same system prompt, different user prompts, very different outcomes each time.

Additional context

Beyond the two prompts, almost anything can get folded in: the contents of a file, a screenshot, a stack trace, terminal output, earlier messages in the conversation. All of it becomes part of what the model is reasoning over, whether or not it’s actually relevant to the question at hand.

Conversation history

A multi-turn conversation keeps stacking. Each new message and each new reply gets added to what the model sees on the next turn. That’s how a model can refer back to something you said five messages ago — but it also means a long conversation is carrying a lot of extra weight by the end.

Context management

More isn’t automatically better. Stuff a model’s context with unrelated files and old, irrelevant messages, and it gets harder — not easier — for the model to find the part that actually matters. A tight, relevant context usually beats a bloated one.

Context limits

Every model has a hard ceiling on how much it can hold at once — a context window. Long-running conversations eventually hit that ceiling, and something has to give: older messages get summarized, trimmed, or dropped so the conversation can keep going.

Next step

Manually deciding what to paste into context every time doesn’t scale. The next piece of the puzzle is letting the model pull in what it needs on its own, on demand, instead of you handing it everything up front.

Previous: AI Foundations 3 - Tokens and Pricing Next: AI Foundations 5 - Tool Calling