What is a Context Window in Language Models?
A context window is the total amount of text, measured in tokens, that a large language model can consider at one time, including your prompt, any documents you provide, the conversation history, and the model's own response. A model with a 128,000-token context window can handle roughly 100,000 words in a single interaction. Once you exceed the limit, the model cannot see everything anymore and typically drops or ignores the earliest content. Larger context windows allow bigger documents and longer conversations but generally cost more per request and can be slower.
A context window is the maximum amount of text an AI language model can process at once, measured in tokens, and it determines how much information you can feed the model in a single conversation.
What counts toward the context window
The context window is not just your question to the model. It includes the system instructions that set up the model’s behaviour, any documents or reference material you paste in, the full back-and-forth history of the conversation so far, and the response the model is about to generate. All of these compete for the same fixed budget of tokens. A long conversation with an AI assistant can quietly eat through a large portion of the context window before you even ask your actual question, especially if earlier messages included large pasted documents.
This is why the same context window limit that sounds generous for a single question can feel tight in an extended back-and-forth, particularly when working with code, contracts, or reports that you keep referring back to across many messages.
Tokens, not words or characters
Language models process text as tokens rather than raw words or characters. A token is a chunk of text the model’s tokenizer produces, and it typically covers common words as a single unit while splitting rarer words, numbers, and symbols into several tokens. As a working estimate, one token equals roughly 0.75 words in English, so a 128,000-token context window corresponds to about 100,000 words, enough to hold a full-length book. Dutch and other languages sometimes require more tokens per word depending on how the tokenizer was trained, which is worth remembering when estimating whether a Dutch-language document will fit.
Why larger context windows matter for business use
A bigger context window opens up tasks that were previously impractical: analysing an entire legal contract in one pass instead of splitting it into fragments, reviewing a full codebase for a specific bug, or carrying a long customer support conversation without losing earlier context. It also simplifies application design, since developers spend less effort chunking documents or managing conversation memory manually.
The trade-off is cost and speed. Providers generally charge per token, so filling a large context window with unnecessary information increases the price of every request. Processing time can also increase with input length. In practice, the most cost-effective approach is to include only what is genuinely relevant to the task rather than maximising the size of every prompt.
Working within the limit
For inputs that exceed the context window, or that would be wasteful to include in full, retrieval-augmented generation is the standard approach: rather than pasting an entire knowledge base into the prompt, the system retrieves only the passages relevant to the current question and inserts those. This keeps requests smaller, cheaper, and often more accurate, since the model is not distracted by irrelevant material.
For applications built around an AI workflow, managing the context window deliberately, summarising history, trimming irrelevant steps, retrieving only what is needed, is often what separates a reliable automation from one that becomes slow or inconsistent as conversations or documents grow. Understanding tokens and context limits also helps explain some cases of AI hallucination: a model asked to reason over content that got truncated or pushed out of its window may fill gaps with plausible-sounding but incorrect information.
Our tip: Before building a feature around a large document or long conversation history, check the exact token limit of the model you plan to use and estimate your typical input size against it. It is far cheaper to discover a context window problem during design than after users start hitting it in production.
Frequently asked questions
What happens when you exceed a model's context window?
The exact behaviour depends on the provider, but generally the model either rejects the request outright or silently truncates the oldest part of the input, meaning the beginning of a long document or the earliest messages in a conversation get dropped. This can produce answers that ignore instructions given at the start, or that miss information from earlier in a document, so it is worth checking your input length against the model's stated limit before assuming it will just work.
Does a bigger context window always give better answers?
Not automatically. A larger context window lets you provide more information, but models do not weigh every part of a long input equally well; relevant details buried in the middle of a very long document are sometimes missed or under-used, an effect often called 'lost in the middle'. For best results, keep prompts focused on what is actually relevant rather than pasting an entire document just because the context window allows it.
How is a token different from a word?
A token is the unit a language model actually processes, and it does not map one-to-one to a word. Common English words are often a single token, while longer or less common words, numbers, and punctuation can be split into multiple tokens. As a rough rule of thumb, one token is about 0.75 words in English, so 1,000 tokens is roughly 750 words. Other languages, including Dutch, sometimes use more tokens per word due to different tokenisation.
How do you work with documents larger than the context window?
Common approaches include summarising the document first and feeding the model the summary plus the relevant sections, splitting the document into chunks and processing each separately, or using retrieval-augmented generation, where only the most relevant passages are retrieved and inserted into the prompt rather than the entire source. For ongoing conversations, some applications automatically summarise older messages once the conversation approaches the context limit, keeping recent detail intact while compressing history.