Disparate Impact on Group Accuracy of Linearization for Private Inference
Disparate Impact on Group Accuracy of Linearization for Private Inference
复制标题
DOI：
10.48550/arxiv.2402.03629
复制
发表时间：
2024-02
期刊：
影响因子：
--
通讯作者：
Saswat Das;Marco Romanelli;Ferdinando Fioretto
中科院分区：
文献类型：
--
作者：
Saswat Das;Marco Romanelli;Ferdinando Fioretto
确保对加密安全数据进行隐私保护推理是众所周知的计算挑战。为了缓解非线性激活中昂贵的密码计算的瓶颈，最近的方法建议在神经网络中线性化这些激活的目标部分。这种技术显著减少了运行时间，而对准确性的影响通常可以忽略不计。在本文中，我们证明了这样的计算收益可能会导致公平成本的增加。具体地说，我们发现，与多数群体相比，减少REU激活的数量会不成比例地降低少数群体的准确性。为了解释这些观察结果，我们提供了关于决策边界性质的受限假设下的数学解释，同时也展示了这个问题在广泛使用的数据集和体系结构中的普遍性。最后，我们展示了一个简单的步骤如何改变线性化模型的微调步骤可以作为一种有效的缓解策略。
Ensuring privacy-preserving inference on cryptographically secure data is a well-known computational challenge. To alleviate the bottleneck of costly cryptographic computations in non-linear activations, recent methods have suggested linearizing a targeted portion of these activations in neural networks. This technique results in significantly reduced runtimes with often negligible impacts on accuracy. In this paper, we demonstrate that such computational benefits may lead to increased fairness costs. Specifically, we find that reducing the number of ReLU activations disproportionately decreases the accuracy for minority groups compared to majority groups. To explain these observations, we provide a mathematical interpretation under restricted assumptions about the nature of the decision boundary, while also showing the prevalence of this problem across widely used datasets and architectures. Finally, we show how a simple procedure altering the fine-tuning step for linearized models can serve as an effective mitigation strategy.