LLM 심도 압축을 위한 지역 인식 이중화 가지치기

[Submitted on 27 May 2026 (v1), last revised 7 Jun 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Large language models are known to contain representational redundancy across network depth, making depth pruning an effective approach for improving inference efficiency. Existing one-shot pruning methods rely on local layer importance or fixed redundancy assumptions across architectures. We propose Locality-Aware Redundancy Pruning (LoRP), a training-free one-shot depth pruning framework guided by representation locality. We show that inter-layer redundancy can be either localized or globally distributed depending on the LLM architecture. To characterize this phenomenon, we introduce Representation Locality Score (RLS), derived from global inter-layer hidden-state similarity. Using a small calibration set, LoRP computes pairwise layer similarity, clusters layers by representational similarity, and allocates pruning according to residual intra-cluster redundancy. Experiments across diverse LLM families show improvements in both perplexity and downstream task accuracy. Official github repository: this https URL

Submission history

From: Daniel Yun [view email]
[v1] Wed, 27 May 2026 00:09:57 UTC (662 KB)
[v2] Sun, 7 Jun 2026 05:41:35 UTC (671 KB)