An LLM (large language model) is a neural network trained on vast amounts of text to understand, generate, and reason about language. It is the reasoning engine at the core of most AI agents.
Popular LLMs include GPT-4, Claude, Gemini, and open-weight models like Llama. Agents use LLMs not just to chat, but to plan, decide which tools to call, and synthesize results.
LLM as the agent's brain
In agent architecture, the LLM handles:
- Understanding ambiguous natural-language input
- Planning multi-step approaches to a goal
- Selecting which tool to call and with what parameters
- Synthesizing tool results into coherent output
- Deciding when the task is complete or needs escalation
The LLM does not execute actions itself. It decides what should happen; your runtime executes tools and returns results.
Choosing an LLM for agents
Considerations for agent workloads differ from simple chat:
- Reasoning quality — Can it plan reliably across multiple steps?
- Tool calling accuracy — Does it produce valid structured outputs?
- Context window — How much history and retrieved content can it hold?
- Latency and cost — Agent loops multiply token usage per task
- Reliability — Consistency matters more than creativity for ops workflows
Related reading
See Context Window for how LLM memory limits affect agent design.