‘추천하지 마세요’ 가 누더기용 가드레일이 아닌 이유

You built a system to surface information so a person could decide. Somewhere it started deciding for them — the output stopped saying “here’s what the documents show” and started saying “you should do X.” Nobody designed that drift. An LLM, when asked a question, produces an answer-shaped thing, and an answer easily becomes a verdict.

What everyone tries

A prompt instruction: “Don’t make recommendations.” “Only state what’s in the documents.” People add the line and assume the boundary is enforced.

Why it doesn’t work

A prompt instruction is a request, not a guardrail. The model follows it most of the time, then on the input that matters produces a confident recommendation anyway, because nothing structurally prevents it. “Please don’t make recommendations” is to a guardrail what a sticky note saying “please don’t enter” is to a locked door.

And the stakes are higher than they look. When output drifts from evidence to verdict, accountability moves. As long as the system returns evidence and a human decides, the human owns the decision. The moment the system returns a verdict and the human defers, the system is deciding things it was never validated to decide — and when one is wrong, accountability is a blank. High-stakes fields separate evidence extraction from judgment on purpose; most RAG systems erase that line by default.

The one shift

Decide what the output is and enforce it structurally. An output should declare itself: answer, evidence, missing facts, or out-of-scope. “Return decision material, not a decision” has to live in the output contract and in gates — not in a polite request to the model. The system supplies frames; the human supplies verdicts.

This is the output boundary — one of three places production RAG dies.

Read the full version on my blog, where this connects to the RAG Failure Diagnosis Kit for teams debugging production RAG.