Why does the AI always leave a few blocks out?
您的瀏覽器不支援音檔播放。下載音檔
The pain you’ve definitely hit
I asked an AI to build me a brand website. What came back had a header with two links, no “About” page at all, and an admin panel that worked — but with no button anywhere on the site that reached it. Backend 90, frontend 10.
My first guess was that it was being lazy. It wasn’t. This is a structural problem — and the design principle that fixes it turned out to be one I’d reused, without noticing, seven times in my own toolchain.
This piece is about that principle. But first the “leaves a few blocks out” pain, because it has a name — and I suspect no one’s named it.
What’s missing is a completeness baseline
I went digging into why these tools (mine included) “leave a few blocks out,” and found they all verify the same thing:
“Is everything you declared internally consistent?”
and never:
“Does this kind of thing have what it’s supposed to?”
That gap is fatal. You ask for a brand site, mention two pages, and every check faithfully confirms those two pages link to each other — then passes green. It never asks “a brand site is supposed to have an About page,” because nothing in the system holds a standard for what a brand site looks like. Garbage in, verified garbage out.
That’s the missing piece: a completeness baseline — a standard of what a kind of deliverable should contain, that you check the real thing against. Not internal consistency. Whether what’s supposed to be there, is.
I’m not the only one who felt this. In Thomson Reuters’ write-up of Claude Forge, the author drops one line: “Passing tests is necessary, but not enough.” He felt the next layer — and didn’t build it. Everyone’s hit this pain; nobody grew the baseline.
What I did: a completeness axis per deliverable type
My fix isn’t “ship a built-in website template” — that would wreck the people building an API, an automation, a bot, anything non-web. It’s an axis: first classify what kind of thing this is, and let each kind carry its own baseline of “what complete looks like”:
- Brand site → which sections it should have, whether each surface is reachable, whether the trust pages are real.
- Pure API → docs, error codes, versioning, rate limits.
- Automation → does it run, is it observable, does failure alert anyone.
And completeness is three layers, not a yes/no: exists → reachable → substantiated. That last one gets dropped the most: an About page that needs a photo isn’t done until there’s an image slot on the page and a path in the backend to actually put a photo in it — not just a wall of text. Web completeness is only one cell on the axis — which is why “you can’t make ‘web app’ the only standard” is satisfied by design, not by exception.
That’s the completeness baseline.
Then the same law kept showing up
Once that baseline was in, I kept building — wiring the pipeline so it runs from an idea (or an audit of an existing project) all the way to delivery. And I noticed something strange: every piece I added was the same design decision wearing a different outfit.
(1) Capability-gated, not identity-gated.
The tool doesn’t ask “am I Claude?” It asks “is this capability present?” — present, use the rich interaction; absent, degrade gracefully to plain text. Why it matters: the same thing runs on Claude Code, Cursor, or a bare LLM without breaking. Detect the capability, not the identity.
(2) Public engine, private values.
I open-source the engine, but keep my hard-won know-how — house conventions, a list of security traps, table names — in a private repo, injected at runtime through that same capability gate, never leaked. A public user without my private file just gets neutral defaults, completely undisturbed. My habits stay mine: not imposed, not exposed.
(3) Propose, don’t auto-advance.
At the end of each stage, the system proactively offers the next step — but the human decides. What’s automated is the flow, not the judgment. I even refused to make “auto-fix what the audit found” fully automatic: first it triages (genuinely fix-now / deliberately deferred / already correct), the human picks which to fix, and for the dangerous ones (touching row-level security, permissions) it stops and shows you the SQL before applying anything.
The meta-observation is the real one: these aren’t three features. They’re one design philosophy in three outfits. I counted — from the questioning interface, to knowledge injection, to stage handoffs, to a security gate, to the audit-then-fix loop, to the hook that makes a handoff fire deterministically — I made the same decision seven times. When you catch yourself making the same design call a seventh time, that’s not repetition. That’s a law. Name it.
Honestly: how new is any of this?
I won’t claim “no one’s done this,” so I actually checked — English and Chinese. Honest verdict:
- Already said to death (not my selling point): “tools loosely coupled through files” (decades-old Unix philosophy; BMAD-METHOD and Claude Forge both do it) and “ask the human at every step” (Spec Kit, AWS Kiro, Forge; Thoughtworks already named it a 2025 consensus practice). I won’t lead with these.
- No clear prior art I could find: the three above — capability-gating, public-engine/private-values, and the completeness baseline. The closest neighbor, BMAD-METHOD, covers loose coupling + role division, but has no capability gate, no public/private injection, no completeness baseline.
So my novelty isn’t in any single point — it’s in the combination plus the framing of seeing them as one law. I’m saying that conservatively, because someone’s unindexed repo or private essay may well have done something similar. But three rounds of searching didn’t surface it, which is itself a signal.
A small piece of evidence: is the cheap model enough?
Design aside, the rest is cost. I ran a single controlled experiment (N=1, labeled honestly): same spec, same decomposition rules, same prompt — only the model changed. Sonnet vs Opus, one decomposition each.
- Objective: Sonnet 75k tokens, Opus 86k (14% more). ~80% overlap, both valid, both honest.
- The qualitative difference is the signal: on the mechanical parts Sonnet was as good or better (it wrote out the acceptance-example tables more fully, and caught a double-broadcast conflict more explicitly) — and cheaper. Opus earned its extra cost in exactly one place: it caught a hidden prerequisite — “on reconnect, with no account system and a connection ID that changes, how does the server know which player this is?” — and refused to invent a fake fix, flagging it as a genuine gap to take back to the human. Sonnet papered over it with a decision that doesn’t actually solve the problem. Opus also caught an internal spec contradiction Sonnet missed.
Those are precisely the errors that propagate downstream into integration breaks. So this N=1 (I’ll take it to N≥3 before treating it as hard evidence) says: the cheap model carries the mechanical bulk — 80% there and cheaper; the expensive model is worth its cost only on the “refuse to invent, catch the contradiction” judgment that prevents propagation. Which is exactly why my pipeline assigns a model per goal and keeps an independent audit as a safety net.
Not “the expensive one is better.” It’s: automate the flow, tier the judgment, and keep a human gate where errors propagate.
So the stage isn’t “magic app generator”
If I positioned this as “an AI pipeline from idea to delivery,” it’d get compared to Devin, v0, bolt — and I’m not in their quadrant. They sell magic, one-click, hands-off. I built the opposite:
- Swap-safe (loose coupling — each tool is useful alone, replaceable when it breaks)
- Privacy-preserving (public engine, private values — open-sourceable without leaking your moat)
- Actually complete (the completeness baseline — it doesn’t “leave a few blocks out”)
- Human keeps the judgment (propose, don’t auto-advance)
For a one-person shop or small team, this isn’t “a stronger agent.” It’s a moat design pattern: open-source the engine, accumulate your private know-how, take it to clients — instead of betting everything on a black box you can’t see into.
Closing: the seventh time isn’t repetition, it’s a law
The most practical thing I learned: you don’t stop “leaving a few blocks out” with a stronger model or more magic. You stop it by (1) giving the system a completeness baseline that knows what a kind of thing should look like, (2) baking your conventions into reusable parts, and (3) automating the flow but never the judgment.
And — when you catch yourself making the same design decision a seventh time, stop. That’s not you repeating yourself. That’s a law you’ve been using but never said out loud. Name it. It might be exactly the thing everyone else is using too, and nobody’s said yet.
Source: a June 2026 conversation that grew from “the header only has two links” to “a portable security gate + end-to-end handoffs.” The design decisions are recorded as ADR-001 through 007 in the public repos’ DESIGN-NOTES. Prior-art and the full A/B data are kept separately.
답글 남기기