LLM 에이전트의 다단계 명령 계층 구조

[Submitted on 10 Apr 2026 (v1), last revised 4 Sep 2026 (this version, v4)]

View PDF HTML (experimental)

Abstract:Large language model agents receive instructions from many sources-system messages, user prompts, tool outputs, other agents, and more-each carrying different levels of trust and authority. When these instructions conflict, agents must reliably follow the highest-privilege instruction to remain safe and effective. The dominant paradigm, instruction hierarchy (IH), assumes a fixed, small set of privilege levels (typically fewer than five) defined by rigid role labels (e.g., system > user). This is inadequate for real-world agentic settings, where conflicts can arise across far more sources and contexts. In this work, we propose Many-Tier Instruction Hierarchy (ManyIH), a paradigm for resolving instruction conflicts among instructions with arbitrarily many privilege levels. We introduce ManyIH-Bench, the first benchmark for ManyIH. ManyIH-Bench requires models to navigate up to 12 levels of conflicting instructions with varying privileges, comprising 853 agentic tasks (427 coding and 426 instruction-following). ManyIH-Bench composes constraints developed by LLMs and verified by humans to create realistic and difficult test cases spanning 46 real-world agents. Our experiments show that even the current frontier models perform poorly (~40% accuracy) when instruction conflict scales. This work underscores the urgent need for methods that explicitly target fine-grained, scalable instruction conflict resolution in agentic settings.

Submission history

From: Jingyu Zhang [view email]
[v1] Fri, 10 Apr 2026 16:00:04 UTC (1,536 KB)
[v2] Mon, 13 Apr 2026 15:26:01 UTC (1,536 KB)
[v3] Tue, 14 Apr 2026 15:04:47 UTC (1,536 KB)
[v4] Fri, 4 Sep 2026 23:55:56 UTC (1,592 KB)