Cluster Attention for Graph Machine Learning

[Submitted on 8 Apr 2026 (v1), last revised 7 Aug 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Message Passing Neural Networks have recently become the most popular approach to graph machine learning tasks; however, their receptive field is limited by the number of message passing layers. To increase the receptive field, Graph Transformers with global attention have been proposed; however, global attention does not take into account the graph topology and thus lacks graph-structure-based inductive biases, which are typically very important for graph machine learning tasks. In this work, we propose an alternative approach: cluster attention (CLATT). We divide graph nodes into clusters with off-the-shelf graph community detection algorithms and let each node attend to all other nodes in each cluster. CLATT provides large receptive fields while still having strong graph-structure-based inductive biases. We show that augmenting Message Passing Neural Networks or Graph Transformers with CLATT significantly improves their performance on a wide range of graph datasets including datasets from the recently introduced GraphLand benchmark representing real-world applications of graph machine learning.

Submission history

From: Oleg Platonov [view email]
[v1] Wed, 8 Apr 2026 18:33:29 UTC (227 KB)
[v2] Fri, 7 Aug 2026 15:11:34 UTC (227 KB)