Title:GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training

Abstract:Graph representation learning has emerged as a powerful technique for real-world problems. Various downstream graph learning tasks have benefited from its recent developments, such as node classification, similarity search, graph classification, and link prediction. However, prior arts on graph representation learning focus on domain specific problems and train a dedicated model for each graph, which is usually non-transferable to out-of-domain data. Inspired by recent advances in pre-training from natural language processing and computer vision, we design Graph Contrastive Coding (GCC) -- an unsupervised graph representation learning framework -- to capture the universal network topological properties across multiple networks. We design GCC's pre-training task as subgraph-level instance discrimination in and across networks and leverage contrastive learning to empower the model to learn the intrinsic and transferable structural representations. We conduct extensive experiments on three graph learning tasks and ten graph datasets. The results show that GCC pre-trained on a collection of diverse datasets can achieve competitive or better performance to its task-specific trained-from-scratch counterparts. This suggests that the pre-training and fine-tuning paradigm presents great potential for graph representation learning.