← 피드로
[Submitted on 10 Dec 2025 (v1), last revised 18 Jul 2026 (this version, v3)]
Abstract:Autoregressive language models expose one causal token frontier, even when the requested document contains sections that could be developed concurrently. Existing parallel-generation systems arrange external branches around an otherwise unchanged model. We instead formulate model-intrinsic parallel generation: a single trained architecture owns multiple causal frontiers and produces one next-token distribution for each frontier in every synchronized decoding round. The Parallel Decoder Transformer (PDT) retains a frozen shared lower knowledge trunk and replaces the upper trunk with three independently parameterized physical decoder stacks. A prompt-time set planner produces three unordered continuous outlines, each hard-routed to one decoder as persistent Plan-KV memory, while a finite product-quantized notes bus carries block-delayed latent messages among the decoders. Autoregression is preserved within each lane; same-round lane tokens are conditionally independent given the source, plans, private histories, and previously committed messages. We specify source-grounded supervision for long-form historical exposition with single-owner cited facts and token-aligned cross-lane dependencies, a composite objective, a staged curriculum, and preregistered causal evaluations: plan swap and removal, delayed-message ablation, a parameter-matched self-only control, dependency-token likelihood, and blinded human fact audits. The architecture and evaluation pipeline are implemented; scientific training and held-out evaluation are in progress. This paper presents the theory, design, and falsifiable protocol, not a positive empirical result.
Submission history
From: Logan Robbins [view email]
[v1]
Wed, 10 Dec 2025 20:19:10 UTC (171 KB)
[v2]
Mon, 9 Mar 2026 14:35:35 UTC (191 KB)
[v3]
Sat, 18 Jul 2026 20:06:01 UTC (20 KB)
추출 본문 · 출처: arxiv.org · https://arxiv.org/abs/2512.10054
답글 남기기