RELIANT: Fair Knowledge Distillation for Graph Neural Networks
RELIANT: Fair Knowledge Distillation for Graph Neural Networks
复制标题
DOI：
10.48550/arxiv.2301.01150
复制
发表时间：
2023-01
期刊：
影响因子：
--
通讯作者：
Yushun Dong;Binchi Zhang;Yiling Yuan;Na Zou;Qi Wang;Jundong Li
中科院分区：
文献类型：
--
作者：
Yushun Dong;Binchi Zhang;Yiling Yuan;Na Zou;Qi Wang;Jundong Li
图神经网络（GNN）在各种图学习任务上表现出令人满意的性能。为了实现更好的拟合能力，大多数GNN都具有大量参数，这使得这些GNN的计算成本很高。因此，很难将它们部署到具有稀缺计算资源的边缘设备上，例如，移动的电话和可穿戴智能设备。知识蒸馏（KD）是压缩GNN的常见解决方案，其中轻量级模型（即，学生模型）被鼓励模仿计算上昂贵的GNN的行为（即，教师GNN模型）。然而，大多数现有的基于GNN的KD方法缺乏公平性考虑。因此，学生模型通常继承甚至夸大了教师GNN的偏差。为了处理这样的问题，我们采取了初步措施，以实现GNN的公平知识蒸馏。具体来说，我们首先制定了一个新的问题，公平的知识蒸馏GNN为基础的师生框架。然后，我们提出了一个名为RELIANT的原则性框架，以减轻学生模型所表现出的偏见。值得注意的是，RELIANT的设计与任何特定的教师和学生模型结构解耦，因此可以轻松地适应各种基于GNN的KD框架。我们在多个真实世界的数据集上进行了广泛的实验，这证实了RELIANT在保持高预测效用的同时实现了更少偏差的GNN知识提取。
Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably, the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility.