In graph analysis, collecting labeled data for traditional supervised learning methods can be challenging, particularly for academic, social, and biological networks. As a means to navigate this, Graph Self-supervised Pre-training (GSP) techniques have become more prevalent. These methods capitalize on the inherent structures and characteristics of graph data, mining meaningful representations without needing labeled examples to do so.
Two primary types of GSP methods exist: contrastive and generative. Contrastive methods, such as GraphCL and SimGRACE, generate varying graph views through augmentation, learning representations via contrasting positive and negative samples. Alternatively, generative techniques like GraphMAE and MaskGAE place emphasis on learning node representation through a reconstruction objective. These generative GSP techniques are often found to be simpler and more effective than their contrastive counterparts, which are heavily reliant on carefully crafted augmentation and sampling methods.
Current Generativity Graph-masked AutoEncoder (GMAE) models predominantly focus on reconstructing node features, therefore primarily capturing node-level details. However, many graphs, like those used in social networks, recommendation systems, and molecular structures, operate at a multi-scale level. These graphs contain not just node-level information, but subgraph-level details too. This inability of current GMAE models to effectively learn such complex, multi-dimensional structural information often leads to reduced performance.
To better accommodate these multi-scale structures, a group of researchers from institutions, including Wuhan University, unveiled the Hierarchical Graph Masked AutoEncoders (Hi-GMAE) framework. Hi-GMAE consists of three core components explicitly designed to capture hierarchical information within graphs. First, it uses multi-scale coarsening to construct coarse graphs at multiple levels, progressively clustering nodes into super-nodes. Second, the novel Coarse-to-Fine (CoFi) masking strategy is utilized for ensuring the uniformity of masked subgraphs across all scales. Lastly, the Fine- and Coarse-Grained (Fi-Co) encoder and decoder are integrated to capture local and global information at various graph scales.
The effectiveness of Hi-GMAE was assessed through rigorous experimentation using a range of widely-used datasets. The results decidedly demonstrated that Hi-GMAE outperformed current leading models in both contrastive and generative pre-training domains. This points towards the superiority of the multi-scale GMAE approach over single-scale models, specifically regarding their ability to capture and leverage hierarchical graph information.
In conclusion, Hi-GMAE signifies progressive development in self-supervised graph pre-training. By incorporating multi-scale coarsening, a novel masking strategy, and a hierarchy in the encoder-decoder structure, Hi-GMAE effectively accommodates the complexities across varying layers of graph structures. Its outstanding performance in experimental evaluations bolsters its potential as a robust tool for graph learning tasks, setting a new standard and benchmark in graph analysis.
