If you’ve built even a simple AI agent, you’ve probably noticed that the “agent loop” itself is deceptively simple: the model gets a message, decides whether to call a tool, gets the result back, and repeats until it has an answer. But real-world agents need a lot more than that bare loop to actually work well.
What happens when a conversation gets so long it blows past the model’s context window? What if a tool call gets interrupted halfway through and leaves your message history in a broken state? What if you want the agent to keep a running todo list of what it’s working on, or delegate parts of a task to a specialized sub-agent, or read and write files as part of its job?
You could bolt all of this onto your agent manually. Or, if you’re using Deep Agents, you get most of it for free through something called middleware.
This post walks through what middleware actually is, why Deep Agents ships with a default stack of it, and how each piece behaves, with runnable code for each one so you can see it working instead of just reading about it.
So What Is Middleware, Really?
If you’ve done any web development, the term “middleware” probably already rings a bell. It’s the same idea here.
Middleware is code that sits around the core agent loop and gets a chance to run before or after certain things happen, like before a tool call executes, after the model responds, or right before messages are sent to the model. Instead of writing all of this logic directly inside your agent, you attach separate, independent pieces of middleware that each handle one specific concern.
This matters for two reasons:
- You don’t have to build common behaviors from scratch. Things like managing a todo list, summarizing long conversations, or handling file access are problems almost every non-trivial agent runs into. Deep Agents ships default middleware for these so you don’t reinvent them every time.
- You can customize behavior without touching the agent’s core logic. Need a custom summarization strategy? Want to pause and ask for human approval before certain tool calls run? You add or swap out middleware, and the rest of the agent stays untouched.
Think of it like a pipeline. A request comes in, passes through a stack of middleware (each one doing its own small job), reaches the model, and the response passes back out through that same stack on the way out.
Why Deep Agents Comes With Middleware Already Built In
When you call create_deep_agent, you’re not getting a bare-bones agent loop. You’re getting an agent that already knows how to track tasks, manage files, spin up sub-agents, keep conversations within context limits, and recover gracefully from interrupted tool calls, all because of a default middleware stack that’s assembled for you automatically.
Here’s the order that stack runs in, from first to last:
- TodoListMiddleware — tracks and manages todo lists for organizing the agent’s work
-
SkillsMiddleware — (only if you pass
skills) injects skill metadata before file tools run - FilesystemMiddleware — handles reading, writing, and navigating files, plus permission enforcement
- SubAgentMiddleware — spawns and coordinates sub-agents for delegated tasks
- SummarizationMiddleware — condenses message history when conversations get long
- PatchToolCallsMiddleware — repairs broken tool calls after an interrupted run resumes
- AsyncSubAgentMiddleware — (only if you configure async sub-agents)
After that, there’s room for your own custom middleware, followed by a “tail” of provider-specific extras: things like prompt caching for Anthropic or Bedrock models, memory injection, excluded-tool filtering, and human-in-the-loop approval steps.
The important thing to understand up front is that order matters. Each middleware runs at a specific point for a reason, for example, PatchToolCallsMiddleware needs to run before prompt caching so the cached message prefix actually matches what gets sent to the model. We’ll come back to details like this as we go through each piece.
What We’ll Cover
Rather than explain all of this in the abstract, the rest of this post goes through each middleware one at a time with a runnable example, so you can actually see:
- What it does in practice
- When it kicks in (always, or only under certain conditions)
- What breaks or gets harder without it
By the end, you should have a clear mental model of what’s happening under the hood every time you call create_deep_agent, and enough understanding to start adding your own custom middleware or overriding the defaults when you need something different.
Let’s start with the first one in the stack: TodoListMiddleware.
1. TodoListMiddleware
"""
Example: TodoListMiddleware
"""
import os
import sys
sys.stdout.reconfigure(encoding="utf-8")
sys.stderr.reconfigure(encoding="utf-8")
from dotenv import load_dotenv
load_dotenv()
from langchain.tools import tool
from langchain_nvidia_ai_endpoints import ChatNVIDIA
from deepagents import create_deep_agent
@tool
def get_weather(city: str) -> str:
"""Get the current weather in a city."""
return f"The weather in {city} is sunny, 24C."
@tool
def get_time(city: str) -> str:
"""Get the current time in a city."""
return f"The current time in {city} is 12:00 PM."
@tool
def get_population(city: str) -> str:
"""Get the approximate population of a city."""
return f"The population of {city} is approximately 14 million."
# Build the model directly so we can control the timeout
model = ChatNVIDIA(
model="nvidia/nemotron-3-super-120b-a12b",
api_key=os.environ["NVIDIA_API_KEY"],
)
agent = create_deep_agent(
model=model,
tools=[get_weather, get_time, get_population],
system_prompt="You are a helpful assistant. For multi-step requests, "
"plan your steps using the todo list before executing them.",
)
result = agent.invoke(
{
"messages": [
(
"user",
"I need a full briefing on Tokyo: check the weather, the "
"current time, and the population. Plan this out first, "
"then give me a summary.",
)
]
},
config={"configurable": {"thread_id": "todo-example-1"}},
)
for m in result["messages"]:
if m.type == "ai":
if m.tool_calls:
for call in m.tool_calls:
print(f"[Tool Call] {call['name']} -> {call['args']}")
if m.content:
print(f"AI: {m.content}")
elif m.type == "tool":
print(f"[Tool Result - {m.name}]: {m.content}")
if "todos" in result:
print("\nFinal todo list state:")
for todo in result["todos"]:
print(f" - {todo}")
Enter fullscreen mode Exit fullscreen mode
2. SkillsMiddleware
"""
Example: SkillsMiddleware
The SkillsMiddleware is added to the default stack automatically when you pass
`skills=` to `create_deep_agent`. It loads each skill's `name` and `description`
from the YAML frontmatter of `SKILL.md` into the system prompt at startup, and
gives the agent a `read_file` tool so it can pull the full instructions in
progressive disclosure when a task matches a skill.
This example uses `FilesystemBackend` so skills are loaded from disk relative
to a project root. Two sample skills live under `middleware-examples/skills/`:
- code-review: reviews code for bugs, security, performance, and style.
- git-commit: writes conventional commit messages from a diff or summary.
Run from the project root with:
uv run middleware-examples/02_skills_middleware.py
"""
import os
import sys
from pathlib import Path
sys.stdout.reconfigure(encoding="utf-8")
sys.stderr.reconfigure(encoding="utf-8")
from dotenv import load_dotenv
load_dotenv()
from langchain_nvidia_ai_endpoints import ChatNVIDIA
from deepagents import create_deep_agent
from deepagents.backends.filesystem import FilesystemBackend
ROOT = Path(__file__).resolve().parent.parent
SKILLS_DIR = ROOT / "middleware-examples" / "skills"
backend = FilesystemBackend(root_dir=str(ROOT))
llm = ChatNVIDIA(
model="nvidia/nemotron-3-super-120b-a12b",
timeout=120,
)
agent = create_deep_agent(
model=llm,
backend=backend,
skills=[str(SKILLS_DIR)],
system_prompt=(
"You are a helpful assistant. When a user request matches one of your "
"available skills, read the skill's SKILL.md and follow its instructions."
),
)
result = agent.invoke(
{
"messages": [
(
"user",
"Please turn this diff into a commit message:\n"
"+ added 5 req/sec rate limiting to /login\n"
"+ added unit tests for the limiter\n"
"- removed the in-memory counter fallback",
),
(
"user",
"review the below code:\n"
"def add(a, b):\n"
" return a.tO_string() + b",
),
]
},
config={"configurable": {"thread_id": "skills-example-1"}},
)
print("\n=== Skill state ===")
for m in result["messages"]:
if m.type == "ai" and m.content:
print(f"\nAI: {m.content}")
Enter fullscreen mode Exit fullscreen mode
3. FilesystemMiddleware
"""
Example: FilesystemMiddleware
The FilesystemMiddleware handles file system operations such as reading,
writing, and navigating directories. When you pass permissions, filesystem
permissions enforcement is included.
This example shows basic filesystem operations without custom permissions.
Run with: python middleware-examples/03_filesystem_middleware.py
"""
import os
import sys
from pathlib import Path
sys.stdout.reconfigure(encoding="utf-8")
sys.stderr.reconfigure(encoding="utf-8")
from dotenv import load_dotenv
load_dotenv()
from langchain_nvidia_ai_endpoints import ChatNVIDIA
from deepagents import create_deep_agent
from deepagents.backends.filesystem import FilesystemBackend
# FIX: this was `.parent.parent`, which resolves to one directory *above*
# your project (this file lives at <ROOT>/middleware-examples/<this file>,
# so a single .parent already gets you back to <ROOT>). The extra .parent
# silently wrote poem.txt outside the folder you were checking.
ROOT = Path(__file__).resolve().parent
# FIX: virtual_mode=True still resolves relative paths under root_dir (same
# as before) but also blocks path traversal ('..', '~') and absolute paths
# escaping root_dir. virtual_mode=False gives the agent no real guardrails.
backend = FilesystemBackend(root_dir=str(ROOT), virtual_mode=True)
llm = ChatNVIDIA(
model="nvidia/nemotron-3-super-120b-a12b",
timeout=120,
)
agent = create_deep_agent(
model=llm,
backend=backend,
system_prompt=(
"You are a helpful assistant. Use the filesystem to save and "
"read files as needed."
),
)
result = agent.invoke(
{
"messages": [
(
"user",
"Write a short poem to a file called 'poem.txt'.",
)
]
},
config={"configurable": {"thread_id": "fs-example-1"}},
)
# DEBUG: log every message, including tool calls and tool results, not just
# AI text. This is what would have made the original bug obvious immediately
# — you'd have seen a write_file call and a WriteResult path, just not in
# the folder you expected.
for m in result["messages"]:
if m.type == "ai":
if getattr(m, "tool_calls", None):
for tc in m.tool_calls:
print(f"TOOL CALL: {tc['name']}({tc['args']})")
if m.content:
print(f"AI: {m.content}")
elif m.type == "tool":
print(f"TOOL RESULT [{m.name}]: {m.content}")
# Sanity check: prove where the file actually landed.
expected_path = ROOT / "poem.txt"
if expected_path.exists():
print(f"\n✅ poem.txt created at: {expected_path}")
else:
print(f"\n⚠️ poem.txt not found at expected path: {expected_path}")
Enter fullscreen mode Exit fullscreen mode
4. SubAgentMiddleware
"""
Example: SubAgentMiddleware
The SubAgentMiddleware spawns and coordinates subagents for delegating
tasks to specialized agents. Only the parent agent exposes the task tool
that creates subagents.
This example creates a subagent for research. The parent agent delegates
research to the subagent via the task tool.
Run with: python middleware-examples/04_subagent_middleware.py
"""
import os
import sys
sys.stdout.reconfigure(encoding="utf-8")
sys.stderr.reconfigure(encoding="utf-8")
from dotenv import load_dotenv
load_dotenv()
from langchain_nvidia_ai_endpoints import ChatNVIDIA
from deepagents import create_deep_agent, SubAgent
llm = ChatNVIDIA(
model="nvidia/nemotron-3-super-120b-a12b",
timeout=120,
model_kwargs={"chat_template_kwargs": {"enable_thinking": False}},
)
research_subagent = SubAgent(
name="research",
description="Researches a topic and returns findings.",
model=llm,
system_prompt="You are a research assistant. Find answers concisely.",
)
writer_subagent = SubAgent(
name="writer",
description="Writes content, reports, or summaries based on research findings.",
model=llm,
system_prompt="You are a skilled writer. Produce clear, well-structured content.",
)
reviewer_subagent = SubAgent(
name="reviewer",
description="Reviews content for quality, accuracy, and completeness.",
model=llm,
system_prompt="You are a meticulous reviewer. Check for errors and suggest improvements.",
)
agent = create_deep_agent(
model=llm,
subagents=[research_subagent, writer_subagent, reviewer_subagent],
system_prompt=(
"You are a helpful assistant. Delegate tasks to subagents using the "
"task tool. Use 'research' for finding information, 'writer' for "
"creating content, and 'reviewer' for quality checks."
),
)
result = agent.invoke(
{
"messages": [
(
"user",
"Research the history of Python, write a short summary, then review it.",
)
]
},
config={"configurable": {"thread_id": "subagent-example-1"}},
)
for m in result["messages"]:
if m.type == "ai" and m.content:
print(f"AI: {m.content}")
Enter fullscreen mode Exit fullscreen mode
답글 남기기