AutoGen's hidden token tax: why a 3-agent chat costs 15 what you expect

AutoGen’s hidden token tax: why a 3-agent chat costs 15× what you expect

Cost-audit series, episode 2. This series began with an AI agent that burned 136M tokens overnight →.

AutoGen is Microsoft’s multi-agent framework. It’s genuinely good at orchestrating agents that hand off work to each other. But its default memory model has a cost shape that surprises almost every team that hits it in production.

This audit shows you exactly where the tokens go, with line numbers.

The setup: a 3-agent RoundRobin chat

The canonical AutoGen pattern is a RoundRobinGroupChat with N agents taking turns on a task. Here’s the minimal version from the docs:

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination

planner  = AssistantAgent("planner",  model_client=client, system_message="You plan.")
coder    = AssistantAgent("coder",    model_client=client, system_message="You code.")
reviewer = AssistantAgent("reviewer", model_client=client, system_message="You review.")

team = RoundRobinGroupChat(
    [planner, coder, reviewer],
    termination_condition=MaxMessageTermination(max_messages=10),
)
await team.run(task="Build a web scraper for Hacker News.")

Enter fullscreen mode Exit fullscreen mode

Three agents, 10 turns total (~3–4 turns each). Seems cheap. It isn’t.

The default context: unbounded, per-agent

Every AssistantAgent gets its own UnboundedChatCompletionContext by default:

# autogen-agentchat/src/autogen_agentchat/agents/_assistant_agent.py, __init__ (L708)
if model_context is not None:
    self._model_context = model_context
else:
    self._model_context = UnboundedChatCompletionContext()

Enter fullscreen mode Exit fullscreen mode

source

UnboundedChatCompletionContext.get_messages() returns self._messages — the full list, no cap, no truncation:

# autogen-core/.../model_context/_unbounded_chat_completion_context.py (a ~20-line file)
async def get_messages(self) -> List[LLMMessage]:
    """Get at most `buffer_size` recent messages."""
    return self._messages

Enter fullscreen mode Exit fullscreen mode

source

(The docstring says “at most buffer_size” — that’s a copy-paste artifact from BufferedChatCompletionContext. There is no buffer. It returns everything.)

The handoff tax: every agent sees every message

When an agent’s turn arrives, on_messages_stream adds all incoming messages to its own context before calling the LLM:

# _assistant_agent.py, in on_messages_stream (STEP 1: "Add new user/handoff messages
# to the model context")
await self._add_messages_to_context(
    model_context=model_context,
    messages=messages,          # ← the full message_thread from the group manager
    ...
)

Enter fullscreen mode Exit fullscreen mode

And _add_messages_to_context appends each one:

# _assistant_agent.py, static method _add_messages_to_context
await model_context.add_message(llm_msg)
...
await model_context.add_message(msg.to_model_message())

Enter fullscreen mode Exit fullscreen mode

source

The group manager (BaseGroupChatManager) maintains a single _message_thread and appends every response to it:

# _base_group_chat_manager.py
self._message_thread: List[BaseAgentEvent | BaseChatMessage] = []
...
await self.update_message_thread(delta)   # called after every agent response

Enter fullscreen mode Exit fullscreen mode

source

So at turn T, the agent receiving the baton gets T-1 messages added to its already-growing context. Its context now contains everything it has ever seen.

The math: O(N × T²) total tokens

Let’s be precise. Define:

• T = total turns in the conversation
• N = number of agents
• m = average tokens per message (system prompt + response, ~300 tokens is realistic for a coding task)

Each agent speaks every N turns. When agent i speaks on turn t, its context contains all t-1 prior messages (because it has been accumulating them since turn 1).

Tokens consumed by agent i on turn t:

context_tokens(t) = (t - 1) × m

Enter fullscreen mode Exit fullscreen mode

Total tokens for agent i across all its turns (it speaks at turns N, 2N, 3N, … up to T):

Σ (kN - 1) × m  for k = 1 to T/N
≈ m × N × (T/N)² / 2
= m × T² / (2N)

Enter fullscreen mode Exit fullscreen mode

Total tokens across all N agents:

N × m × T² / (2N) = m × T² / 2

Enter fullscreen mode Exit fullscreen mode

The N cancels. Total cost scales as T² regardless of how many agents you add.

Worked example: 10 turns, 3 agents, 300 tokens/message

Turn Agent Context size (messages) Tokens in this call 1 planner 0 prior + system ~300 2 coder 1 prior + system ~600 3 reviewer 2 prior + system ~900 4 planner 3 prior + system ~1,200 5 coder 4 prior + system ~1,500 6 reviewer 5 prior + system ~1,800 7 planner 6 prior + system ~2,100 8 coder 7 prior + system ~2,400 9 reviewer 8 prior + system ~2,700 10 planner 9 prior + system ~3,000 Total ~16,500 tokens

Naïve expectation (10 calls × 300 tokens each): 3,000 tokens

Actual: ~16,500 tokens — 5.5× more.

At 20 turns it’s ~63,000 tokens vs 6,000 expected — 10.5× more.

At 30 turns: ~139,500 tokens vs 9,000 — 15.5× more.

The multiplier grows linearly with T. This is the same O(T²) shape as ConversationBufferMemory in LangChain — but AutoGen’s version is per-agent, so it’s easy to miss in per-call logs.

Why per-call logs hide this

If you’re watching your LLM provider’s per-call token counts, you see something like:

call 1:  300 tokens  ✓ cheap
call 2:  600 tokens  ✓ fine
call 3:  900 tokens  ✓ ok
...
call 10: 3,000 tokens  ← this one looks expensive

Enter fullscreen mode Exit fullscreen mode

Each call looks like a modest increase. The cumulative total — 16,500 — only shows up when you sum across the run. Most observability dashboards show per-call costs, not per-run totals. The runaway is invisible until the bill arrives.

The fix: cap the context

AutoGen ships two bounded alternatives, named in the AssistantAgent.__init__ docstring (around L1034 of _assistant_agent.py): BufferedChatCompletionContext (limits message count) and TokenLimitedChatCompletionContext (limits tokens):

Option 1: BufferedChatCompletionContext (sliding window)

from autogen_core.model_context import BufferedChatCompletionContext

coder = AssistantAgent(
    "coder",
    model_client=client,
    model_context=BufferedChatCompletionContext(buffer_size=5),  # last 5 messages
)

Enter fullscreen mode Exit fullscreen mode

Cost shape becomes O(T × buffer_size) — linear. For buffer_size=5 and 30 turns: ~42,000 tokens vs 139,500 unbounded. 3.3× cheaper.

Option 2: TokenLimitedChatCompletionContext (token budget)

from autogen_core.model_context import TokenLimitedChatCompletionContext

coder = AssistantAgent(
    "coder",
    model_client=client,
    model_context=TokenLimitedChatCompletionContext(token_limit=2000),
)

Enter fullscreen mode Exit fullscreen mode

Caps the context at a fixed token budget. More predictable than a message count because message sizes vary.

Which to use?

Scenario Recommendation Short tasks (≤10 turns) Default is fine; monitor cumulative cost Long tasks (>10 turns) BufferedChatCompletionContext(buffer_size=8–12) Strict cost budget TokenLimitedChatCompletionContext(token_limit=N) Need full history Default + add per-run cost alerting (see below)

Detecting this in CI before it hits production

The pattern is detectable statically: any file that instantiates AssistantAgent without a model_context= argument is using the unbounded default.

# Flag unbounded AssistantAgent instantiations
grep -rn "AssistantAgent(" src/ | grep -v "model_context="

Enter fullscreen mode Exit fullscreen mode

For dynamic detection — measuring actual token growth across a run — this is exactly what tokenscope does: it instruments LLM calls, tracks per-run cumulative cost, and can block a CI build when a PR’s token delta exceeds a threshold.

The wartzar-bee/ci-guardrail GitHub Action wraps tokenscope into a one-line workflow addition:

- uses: wartzar-bee/ci-guardrail@v1
  with:
    token_threshold: 50000   # block if PR adds >50k tokens/run
    github_token: ${{ secrets.GITHUB_TOKEN }}

Enter fullscreen mode Exit fullscreen mode

Summary

Naïve expectation Actual (unbounded) With BufferedContext(5) 10 turns, 3 agents 3,000 tokens ~16,500 tokens ~12,000 tokens 20 turns, 3 agents 6,000 tokens ~63,000 tokens ~27,000 tokens 30 turns, 3 agents 9,000 tokens ~139,500 tokens ~42,000 tokens

The default UnboundedChatCompletionContext is correct for short tasks and full-history use cases. It becomes a cost trap in long multi-agent conversations. The fix is one constructor argument — but you have to know to add it.

The broader pattern: every major agent framework defaults to unbounded context because it’s the safest correctness choice. Cost is a second-class citizen in the default config. That’s the gap this series documents.

Next in the series: CrewAI — the delegation overhead. How hierarchical agent trees multiply your token bill.

tokenscope on npm · wartzar-bee/ci-guardrail · @wartzarbee on dev.to