Home / Glossary / Context window
What is Context Window?
The context window is the maximum number of tokens an LLM can process in one request (input + output combined). Modern models range from 4 K to 2 M tokens. osFoundry’s catalog lists the context window for every model.
Detail
Tokens are sub-word units: roughly 3-4 characters of English per token on average, while languages like Japanese often consume a token or more per character — the same document can cost 2x the tokens in Japanese. A 128 K window holds roughly 100,000 English words.
The window is shared: system prompt, conversation history, retrieved context, and the model’s generated reply all count against the same limit. When a long chat suddenly “forgets” earlier turns, older history has been truncated or summarized to fit.
Advertised size is not effective size. Long inputs are prone to lost-in-the-middle recall gaps, where content buried mid-context is recalled worse than the start or end — the classic symptom is a whole-document summary that gets sloppy in the middle sections.
Locally, context length is a memory budget: the KV cache grows roughly linearly with context, on top of the model weights, and long settings also slow generation. As of 2026, flagship cloud models advertise 1-2 M tokens, standard tiers 128-400 K, while local setups typically run 8-128 K. Past the limit, split the input, summarize in stages, or retrieve only the relevant chunks (RAG).
How osFoundry approaches Context Window
osFoundry’s knowledge bases + RAG pipeline retrieve only the relevant chunks for each query, keeping the context window focused. You can also pick a model with a bigger window from the catalog if you need it.
Frequently asked questions
How many characters fit in a 128 K context window?
Roughly 100,000 English words, or about 100,000 Japanese characters — tokenizer-dependent. Languages with lower tokenizer efficiency consume the window faster for the same content.
What happens when I exceed the context window?
The overflow is truncated, or the tool silently summarizes/drops older history. Whether you get an error or silent loss depends on the implementation — missing middle topics in long-meeting summaries is the classic symptom.
Is a bigger context window always better?
A larger cap is more flexible, but long inputs cost more, respond slower, and are prone to lost-in-the-middle recall gaps. Retrieving only the relevant chunks (chunking/RAG) is often more reliable than stuffing everything in.
Related terms
Related features