RWGBench: Evaluating Scholarly Positioning in Related Work Generation

[Submitted on 30 May 2026 (v1), last revised 11 Jul 2026 (this version, v3)]

View PDF HTML (experimental)

Abstract:Large language models have shown strong fluency in scientific writing, yet the evaluation of related work generation (RWG) remains limited. Existing RWG evaluations largely inherit summarization-oriented metrics, using lexical or semantic similarity to reference sections as proxies for quality. However, related work writing is fundamentally a citation-level scholarly positioning task: it requires selecting, organizing, and framing prior work to clarify how a target paper relates to, differs from, and contributes beyond existing this http URL a result, models may generate coherent and semantically-relevant text while exhibiting academically critical failures, such as inappropriate citation selection or misplaced references, that conventional metrics do not this http URL this end, we introduce \textbf{RWGBench}, a benchmark that evaluates RWG from the perspective of citation decision-making rather than text similarity. RWGBench is constructed from a large-scale collection of 40,108 computer science papers and a retrieval corpus of 1.09 million documents, with a carefully curated test set comprising 100 papers and their corresponding published related work this http URL propose a multi-dimensional evaluation framework that assesses citation selection, contextual appropriateness, organization, and discourse this http URL reveal systematic limitations in current systems that are obscured by standard evaluations, while Oracle studies further disentangle retrieval-level and generation-level bottlenecks. Human evaluation further shows that our citation-centric metrics align substantially better with expert judgment than surface-level text metrics. RWGBench offers a citation-centric testbed for developing and evaluating related work generation systems that are better aligned with scholarly writing practices.

Submission history

From: Anzhe Xie [view email]
[v1] Sat, 30 May 2026 16:53:14 UTC (90 KB)
[v2] Wed, 8 Jul 2026 14:41:34 UTC (94 KB)
[v3] Sat, 11 Jul 2026 10:17:56 UTC (94 KB)