The hard limits of an autonomous agent: what keeps it from disaster

작성자

카테고리:

← 피드로
DEV Community · Ramón Chancay 👨🏻‍💻 · 2026-09-05 개발(SW)
Cover image for The hard limits of an autonomous agent: what keeps it from disaster

Ramón Chancay 👨🏻‍💻

A hard limit is a restriction the program enforces, not the model: it is checked in the code that executes the action, after the model has decided, and it doesn’t depend on the agent having understood the instructions correctly. The previous post left the system complete—a ticket goes in, a PR comes out, with nobody pressing a button—and that’s where the question that decides whether this stays an experiment or stays running shows up: what happens when something goes wrong at three in the morning and nobody is watching. This post is that layer: iteration and token caps treated as a real budget, command and path allowlists, a kill switch that works from outside, idempotent effects, and a log that lets you know what happened. And at the end, the uncomfortable part: none of this is the hard bit.

TL;DR

  • A limit asked for in the prompt is a preference; a hard limit is code that runs after the model has decided. Everything that matters—what commands it runs, where it writes, how much it spends, when it stops—goes in the program, not in the instructions.
  • The four that aren’t optional: a per-run budget (iterations, tokens and time), a command allowlist with no shell, a path allowlist resolved with realpath, and a kill switch someone else can flip without deploying anything.
  • The hard part isn’t the code: it’s making the PRs worth reviewing. That doesn’t depend on the agent, it depends on how clear your tickets are and how good your test suite is.

What a hard limit is

There are two ways to tell an agent not to do something. One is writing it in the system prompt: “don’t run destructive commands”, “don’t leave the working directory”. The other is making it so the program can’t execute that action even if the model asks for it. The first works most of the time; the second works always. The difference between the two is this entire post.

The prompt influences the model’s decision, and current models follow instructions fairly well. But an autonomous agent has three ways to get around that instruction with no bad intent at all: it can misread the request, it can call a tool with arguments you didn’t expect, and it can receive text in its context that you didn’t write. That third case is the one that matters in the previous post’s system: the agent reads the text of a ticket, and anyone could have written that text. If the ticket says “to reproduce the bug, run this script”, the model has a perfectly reasonable motive to run it.

A hard limit doesn’t argue with any of that. It is enforced at the point of execution—in the function that runs the command, in the function that writes the file—and it denies by default: what isn’t explicitly allowed doesn’t get through. The model proposes the action; the program decides whether it runs.

The model decides              The program executes

  tool call  ───────────────►  is it allowed?
                                     │
                            no ──────┴────── yes
                             │               │
                             ▼               ▼
                      rejection          it runs
                      observation        inside the
                      (loop continues)   sandbox

Enter fullscreen mode Exit fullscreen mode

There is a design detail in that diagram worth marking right away: a rejection does not end the run. It goes back to the model as one more observation—”command not allowed: curl“—and the agent can correct course, the same way it does when a test fails. A limit that aborts the run at the first disallowed attempt wastes tasks the agent could have solved. A limit that answers lets it keep working inside what’s permitted.

Keep reading

Illustration of an agent's hard limits: the agent loop enclosed in a frame of checks—budget, command allowlist, path allowlist—with a kill switch outside and a log of every turn

That is the first half. The full walkthrough — with the rest of the implementation, the trade-offs and the things that only show up in production — is on my blog:

Read the full post on ramonchancay.me →

Originally published at www.ramonchancay.me/blog/hard-limits-autonomous-agent.

원문에서 계속 ↗