Explaining classification performance and bias via network structure and sampling technique
Explaining classification performance and bias via network structure and sampling technique
复制标题
DOI：
10.1007/s41109-021-00394-3
复制
发表时间：
2021-10-21
影响因子：
2.2
通讯作者：
Wagner, Claudia
中科院分区：
文献类型：
--
作者：
Espin-Noboa, Lisette;Karimi, Fariba;Wagner, Claudia
社交网络是信息的重要载体。例如，我们朋友的政治倾向可以作为识别我们自己政治偏好的代理。这种解释能力在从业务决策到科学研究的许多场景中都得到了利用，从而使用机器学习来推断缺失的属性。然而，影响这些算法的性能和方向的偏见的因素还没有很好地理解。为此，我们系统地研究了网络和训练样本的结构特性如何影响集体分类的结果。我们的主要研究结果表明：（i）平均分类性能可以通过同质性，类平衡，边缘密度和样本大小等结构属性进行经验和分析预测，（ii）小的训练样本足以让heterophilic网络实现高和无偏的分类性能，即使模型估计不完美，（iii）当组大小差异增加时，亲同性网络更容易出现偏差问题和低性能，（iv）当采样预算小时，部分爬行实现最准确的模型估计，并且度采样实现了最高的总体性能。我们的研究结果有助于从业者更好地理解和评估他们的结果时，抽样预算很小，或当没有地面真理是可用的。
Social networks are very important carriers of information. For instance, the political leaning of our friends can serve as a proxy to identify our own political preferences. This explanatory power is leveraged in many scenarios ranging from business decision-making to scientific research to infer missing attributes using machine learning. However, factors affecting the performance and the direction of bias of these algorithms are not well understood. To this end, we systematically study how structural properties of the network and the training sample influence the results of collective classification. Our main findings show that (i) mean classification performance can empirically and analytically be predicted by structural properties such as homophily, class balance, edge density and sample size, (ii) small training samples are enough for heterophilic networks to achieve high and unbiased classification performance, even with imperfect model estimates, (iii) homophilic networks are more prone to bias issues and low performance when group size differences increase, (iv) when sampling budgets are small, partial crawls achieve the most accurate model estimates, and degree sampling achieves the highest overall performance. Our findings help practitioners to better understand and evaluate their results when sampling budgets are small or when no ground-truth is available.