BiFormer: A Bipartite-stream Information Fusion framework for large-scale graph representation learning.

Abstract

Graph Neural Networks (GNNs) and Graph Transformers (GTs) have shown considerable success in graph-based tasks, each offering distinct strengths: GNNs excel at capturing local details, while GTs are adept at capturing global information. However, both GNNs and GTs face scalability issues when applied to large-scale graphs. To address these challenges, this paper proposes the Graph Transformer Based on Bipartite-stream Information Fusion (BiFormer), a framework designed to integrate the benefits of GTs and GNNs for processing large-scale graphs. BiFormer consists of three modules: a global feature extraction module, which utilizes a Transformer encoder to efficiently capture global information from a small-scale pooled graph; and a local feature extraction module that constructs three parameter-free graph convolution kernels for extracting local features without training; a feature fusion module, which employs a Transformer encoder to fuse extracted local and global features of each node without node-to-node message passing. The complete training of BiFormer requires only the small-scale pooled graph and mini-batched local features to be stored temporarily in memory, allowing for mini-batch training with flexible batch size. Experimental results demonstrate that BiFormer outperforms mainstream GTs and GNNs.