A context window is the maximum amount of text (measured in tokens) that an LLM can process in a single request. Everything the model "sees" — system prompt, conversation history, retrieved documents, tool results — must fit within this limit.
Why context window matters for agents
Agents accumulate context rapidly. Each loop iteration adds:
- The user's original request
- Previous reasoning traces
- Tool call arguments and results
- Retrieved documents (via RAG)
A long-running agent task can exceed the context window, forcing you to truncate, summarize, or split the work.
Managing context in agents
Common strategies:
- Summarization — Compress older conversation turns into a summary
- Selective retrieval — Only fetch the most relevant documents (RAG)
- External memory — Store state outside the prompt, retrieve on demand
- Task decomposition — Break large jobs into smaller agent runs
- Context budgeting — Reserve tokens for tools vs. history vs. instructions
Context window sizes (2026)
Windows vary widely by model — from ~128K tokens on many production models to 1M+ on specialized variants. Bigger is not always better: larger contexts increase cost, latency, and the risk of the model losing focus on important details ("lost in the middle" problem).
Related reading
See Agent Memory and RAG for how agents work within context limits.