The idea
The context window is the full stream of tokens the model sees when it generates: system instructions, the user's message, the conversation so far, any documents or tool results. All of it, every time.
The model is stateless. Each request sends the whole conversation again. "Memory" in a chat app means the product resends past messages or saves facts and pastes them back in later.
Modern windows are large (hundreds of thousands of tokens, some around a million), but bigger isn't free: more cost, more latency, and a model can miss a detail buried in the middle of a huge context.
Mental model
The context window is the artboard. Whatever is on the canvas exists, and nothing else does.
Why it matters when you build
- This is your new information architecture. Decide what the model sees, in what order, at what moment.
- Quality comes from context. Same model plus better context gives a dramatically better result. Most prompt problems are really context problems.
- Curate, don't stuff. Give it the relevant five documents, not all five hundred.
- Memory is a product decision. What to save, when to reload it, and whether the user can see and edit it are all yours to design.
Gotchas
- Long conversations eventually overflow the window. Plan for trimming or summarizing old turns.
- Everything in the context can steer the model, including text pasted from untrusted sources (see prompt injection, module 7).