If you’ve built autonomous agents with CrewAI, LangGraph, or Microsoft AutoGen, you know that giving an LLM access to bash tools or database queries is genuinely terrifying.
A single jailbreak, prompt injection, or weird hallucination can run:
bash
rm -rf /
DROP TABLE production_users;
().__class__.__base__.__subclasses__() # Sandbox breakout
Enter fullscreen mode Exit fullscreen mode
Top comments (8)
Subscribe
Collapse Expand
Founder of Bookmarx and Vordi. Building tools that turn saved content into organized, useful knowledge.
-
Location
Mumbai, India
-
Work
Founder of Bookmarx and Vordi
-
Joined
Sep 1, 2026
• Sep 2
The local policy layer is sensible, but sub-50µs only describes decision latency. The harder problem is keeping AST or policy evaluation aligned with runtime context and preventing encoded or indirect commands from bypassing static checks. Do you fail closed when the parser cannot classify an action?
Collapse Expand
Great point Raunak—that’s the exact reason why pure static analysis without execution-level state management eventually falls apart.
To answer directly: yes, Bartholomew strictly fails closed. If the parser cannot deterministically classify an AST node, encounters malformed syntax, or detects unresolvable dynamic sinks (like eval, exec, import, or dunder introspection), the decision engine defaults to a hard DENY.
On encoded and indirect execution paths:
- Normalization & Dynamic Sink Traversal: The parser expands common shell obfuscation layers (like base64 pipes or hex escapes) and flags non-literal code evaluation as an immediate invariant breach rather than trying to guess what an opaque string might resolve to at runtime.
- The Runtime Safety Net (Micro-Rollback): As you rightly noted, static analysis has physical limits once runtime indirection kicks in. That’s why in v2.4 we added in-memory Copy-on-Write micro-rollbacks. Target paths are snapshotted in memory before any tool executes. Even if an indirect command runs, the moment it attempts to touch files outside the workspace root (os.path.commonpath), the filesystem is rolled back in 2.30 microseconds and orphaned artifacts are unlinked.
The strategy is: fail-closed on ambiguous ASTs upfront, backed by an atomic undo button at the OS boundary for speculative runtime side-effects.
If you want to test how the engine handles obfuscated strings or dynamic escapes, try running the zero-install CLI in your terminal:
npx btp-guard
If you’re building harness guards for autonomous agents, I’d suggest pairing AST pre-filtering with in-memory transactional rollbacks rather than relying solely on static inspection—it gives you a deterministic safety net when models attempt indirect runtime execution.
Collapse Expand
Founder of Bookmarx and Vordi. Building tools that turn saved content into organized, useful knowledge.
-
Location
Mumbai, India
-
Work
Founder of Bookmarx and Vordi
-
Joined
Sep 1, 2026
• Sep 3
That fail-closed behavior is the key detail. I would still distinguish rollback from prevention in the product language, since external effects such as network calls cannot be atomically undone. Do you block network and subprocess boundaries separately from filesystem rollback?
Spot-on distinction, and this is an essential architectural detail. You cannot “rollback” a dispatched TCP packet or an external API POST request once it crosses the wire.
In Bartholomew, we decouple this through a strict two-phase boundary model:
1. Pre-Execution Prevention (Non-Idempotent / Irreversible Boundaries)
For network sockets, subprocess spawns, and external API tools, Bartholomew operates strictly as a Pre-Execution Invariant Gate (<1µs AST & Payload Inspection) before any OS syscall or socket creation occurs:
-
Subprocess Boundary: Commands proposed by the agent are parsed at the AST level before invocation. Destructive system calls, privilege escalation attempts, or unauthorized binary executions are dropped locally (
DENY), meaning the subprocess is never spawned in the first place. -
Network Boundary: Outbound network connections undergo domain/CIDR allowlisting and egress inspection. If an unapproved endpoint or high-entropy exfiltration vector is detected, the request is vetoed before the socket handshake initiates. In strict container modes, execution defaults to
--network none.
2. Transactional Rollback (Local Stateful Boundaries)
Rollbacks are reserved specifically for local mutable state (the filesystem and workspace memory map):
- Before any mutating tool (
write_file,apply_patch, file edits) touches disk, Bartholomew captures an in-memory Copy-on-Write (CoW) byte-level checkpoint. - If a downstream tool crashes, triggers an invariant assertion, or attempts a path escape outside the authorized workspace root mid-turn, the filesystem tree reverts atomically with zero residual or orphaned state, accompanied by structured JSON-RPC recovery hints.
Your point on product language is well taken—we will make the distinction between Pre-Execution Boundary Prevention and Transactional State Rollback even sharper in the documentation.
Really appreciate the sharp feedback!
Founder of Bookmarx and Vordi. Building tools that turn saved content into organized, useful knowledge.
-
Location
Mumbai, India
-
Work
Founder of Bookmarx and Vordi
-
Joined
Sep 1, 2026
• Sep 5
That separation is much clearer, and it makes the guarantees testable per boundary. A compact capability matrix for filesystem, subprocess, network, and external APIs would make the product claims especially easy to evaluate.
Collapse Expand
Founder of UnitBuilds CC
-
Location
Swakopmund, Namibia
-
Pronouns
He/Him
-
Work
Senior software Engineer (day-job), Owner of UnitBuilds (sadly second).
-
Joined
May 24, 2026
• Sep 2
Remember, if you block a model from executing a task through the standard way, it will find a way around it… As Anthropic learned the hard way, when they model cheated at a benchmark. So unfortunately the only foolproof solution is to hard-gate the actions at execution in the harness, so it can think of acting, but the tool call is blocked from running.
Collapse Expand
Man, you hit the nail right on the head with that Anthropic benchmark example. Once an agent realizes it’s being blocked semantically, it treats the prompt rules like a puzzle to solve and immediately looks for backdoors.
You’re 100% right that the only way to actually stop it is hard-gating right at execution in the harness before the OS touches it.
When we were building this, though, we ran straight into the next headache: what happens after you hard-gate it. If you just slap the model with a hard error or block, it either leaves half-written garbage files sitting on disk, or it panics and starts spamming the exact same command with slight tweaks until your token bill explodes.
That’s why we ended up turning the execution harness into a mini database transaction:
Before any tool runs, it takes an in-memory byte snapshot of the workspace. If the invariant gate trips, it rolls the filesystem back in literally 2 microseconds so nothing gets corrupted.
Then, instead of just killing the process, it feeds the model a clean diagnostic hint explaining why the path was blocked so it actually changes direction instead of trying to hack around the gate.
If you have a terminal open, you can actually test the whole harness flow right now without installing anything:
bash
npx btp-guard
For further actions, you may consider blocking this person and/or reporting abuse