Exploring Information Seeking Agent Consolidation

[Submitted on 31 Jan 2026 (v1), last revised 24 Jun 2026 (this version, v2)]

Authors:Guochen Yan, Jialong Wu, Zhengwei Tao, Bo Li, Qintong Zhang, Jiahao Xu, Haitao Mi, Yuejian Fang, Qingni Shen, Wentao Zhang, Zhonghai Wu

View PDF HTML (experimental)

Abstract:Information-seeking agents have emerged as a powerful paradigm for knowledge-intensive tasks, yet today’s systems remain specialized for the open web, documents, or local knowledge bases, hindering scalable and cross-domain deployment. We present the first systematic empirical study of consolidating these information-seeking agents into a single foundation agentic model. We compare two paradigms — \emph{data-level mixing}, which trains a unified model on a mixture of datasets, and \emph{parameter-level merging}, which merges independently trained experts in parameter space — across 3 training scenarios, evaluating \textbf{26} representative parameter-level methods on \textbf{10} benchmarks. To compare across heterogeneous benchmarks, we introduce a geometric Composite Score and an Imbalance Score that describe overall performance and task skew. Our analysis shows that (i) well-designed parameter-level merging attains parity with data mixing at a fraction of its training cost and is order-agnostic; (ii) parameter-level merging structurally preserves out-of-domain capabilities that data mixing universally forgets; and (iii) cross-scenario stability is strongly tied to consolidation quality. We distil our observations into a method-selection guide and design principles for next-generation merging operators.

Submission history

From: Guochen Yan [view email]
[v1] Sat, 31 Jan 2026 07:59:31 UTC (3,192 KB)
[v2] Wed, 24 Jun 2026 15:29:53 UTC (3,235 KB)