What is a context window?
AI foundations, models and capabilities
A context window is the total token budget a model can use in one request, including the prompt, retrieved material, prior turns, and the model's reply. It acts like working memory rather than long-term knowledge. Larger windows let systems process more material, but longer context can still reduce recall and accuracy if information is poorly structured.
Reviewed by Jackie, Head of Learning & Development, Levellers - Last reviewed 8 June 2026
What this means
A context window is the amount of text a model can actively use when producing the next response. In provider documentation, that budget includes input and output tokens, and for some models reasoning tokens as well. Anthropic also describes it as working memory, which helps explain why it is different from the broader data a model was trained on.
In practice, the context window is where your live task sits: system instructions, the user's latest prompt, prior conversation, retrieved passages, tool results, and the reply the model is about to generate. If the total goes too high, the system must truncate, compact, reject the request, or reduce what it sends.
Why it matters
For business work, the context window determines how much evidence the model can see at once. That affects common tasks such as answering policy questions, reviewing long documents, or carrying context across a support or operations conversation. The practical limit is not just whether text fits, but whether the model can still use it well.
That last point matters because more context is not automatically better. Anthropic notes that accuracy and recall can degrade as token count grows, and Lost in the Middle found that models often perform worse when the key information sits in the middle of long inputs. So a bigger window reduces one constraint, but it does not remove the need for good retrieval, chunking, and prompt design.
How it works
When a request is sent, the application assembles a prompt package. That can include earlier chat turns, current instructions, source extracts, and tool outputs. OpenAI states that the context window is the maximum number of tokens available in a single request, and Anthropic shows the same budget logic for user messages, tool configuration, tool results, and model output.
As conversations get longer, systems need context management. OpenAI provides compaction so a full context window can be transformed into a smaller canonical next window, and Anthropic recommends server-side compaction when conversations regularly approach limits. In other words, production systems usually rely on retrieval, summarisation or compaction, rather than trying to carry the full raw history forever.
Context size also varies by model. OpenAI's model pages list different context windows by model family, and Anthropic documents both 200k and 1M token windows across Claude models. That is why workflow design should start with the actual model budget, not a generic assumption about what LLMs can handle.
Examples
Contract review. A legal or commercial team may want the model to compare a draft against playbook clauses and prior comments. The useful question is not only whether the draft fits, but whether the right sections, definitions and exceptions are in context at the same time.
Support handover. A service assistant may need the original issue, troubleshooting steps, CRM notes and the latest customer reply. Session memory can preserve history across turns, but the next run still needs an effective in-context package.
Internal policy assistant. If a user asks how leave, procurement or security policy applies to a case, retrieval can bring in the relevant excerpts instead of stuffing the full handbook into every prompt. That usually gives a cleaner prompt and a better signal-to-noise ratio.
Common misunderstandings
A context window is not the same as memory. It is the live token budget for one request. External session stores can keep longer histories, but only the material sent into the next request is actually in context.
A context window is not the same as training data or knowledge cutoff. Anthropic explicitly distinguishes the context window from the larger corpus the model was trained on.
A larger window does not guarantee better answers. Provider guidance and research both warn that long-context performance still depends on structure, relevance and information placement.
Risks and boundaries
The first boundary is mechanical: if you exceed the model's context limit, outputs can be truncated or the request can fail validation, depending on the platform and model.
The second boundary is quality: long prompts can hide the signal inside too much text. Lost in the Middle found that relevant information placed in the middle of long contexts is often used less reliably. This makes context design a workflow problem, not just a model specification problem.
The third boundary is operational: carrying full histories, tool traces and large documents raises cost and complexity. That is why compaction, token counting and selective retrieval matter in production.
What to do next
Take one real workflow and map its token budget. List what must always stay in the prompt, what can be retrieved on demand, and what can be compacted or summarised between turns. Then estimate the token load before you build. OpenAI and Anthropic both provide token counting or context management guidance for this stage.
Have a question or a suggestion, or want to understand how we research and review these guides? Read about our editorial standards and how to reach us.
FAQs
Is a context window the same as memory?
No. It is the live budget for a single request. External session or memory layers can store more history, but only the portion sent into the next model call is in context.
What happens if you exceed the context window?
OpenAI says tokens beyond the limit can be truncated, while Anthropic documents validation errors on newer Claude models when prompt and output tokens exceed the limit.
Do longer context windows remove the need for RAG?
Usually no. Long windows help, but research and provider guidance still point to degradation in long contexts, which keeps retrieval, chunking and compaction useful.
Is a context window measured in words?
No. Providers document these limits in tokens, not words, and both input and output token use count toward the total.