Efficient Multiple Objective Optimization for Fair Misinformation Detection
Efficient Multiple Objective Optimization for Fair Misinformation Detection
复制标题
DOI：
10.1109/bigdata55660.2022.10020909
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Eric Enouen;Katja Mathesius;Sean Wang;Arielle K. Carr;Sihong Xie
中科院分区：
文献类型：
--
作者：
Eric Enouen;Katja Mathesius;Sean Wang;Arielle K. Carr;Sihong Xie
多目标优化（MOO）旨在同时优化多个相互冲突的目标，在机器学习中有着重要的应用，如同时最小化分类损失和公平损失。在最优情况下，进一步优化一个目标必然会增加至少另一个目标，决策者需要全面探索多个最优方案，以确定一个最终解决方案。我们解决的效率，探索帕累托前沿，包含所有的最优。首先，随机多梯度下降（SMGD）需要时间来收敛到大型神经网络和数据集的Pareto前沿。相反，我们探索帕累托前作为一个流形从几个初始的最优值，基于预测校正方法。其次，对于每个探索步骤，预测器迭代地求解在模型参数的数量上二次缩放的大规模线性系统，并且需要一个反向传播来评估求解器的每次迭代的二阶Hessian向量积。我们提出了一个高斯-牛顿近似，线性缩放，只需要一阶内积迭代。最后，我们探索了不同的线性系统求解器，包括用于近似求解线性系统的MINRES和共轭梯度方法。这些创新使预测-校正算法对大型网络和数据集有效。在公平错误信息检测任务上的实验表明：1）预测-校正方法能够以更少的时间找到优于或类似于SMGD的Pareto前沿; 2）所提出的一阶方法不会损害由二阶方法识别的Pareto前沿的质量，同时进一步减少了运行时间。
Multiple-objective optimization (MOO) aims to simultaneously optimize multiple conflicting o bjectives a nd has found important applications in machine learning, such as simultaneously minimizing classification a nd f airness l osses. At an optimum, further optimizing one objective will necessarily increase at least another objective, and decision-makers need to comprehensively explore multiple optima to pin-point one final solution. We address the efficiency of exploring the Pareto front that contains all optima. First, stochastic multi-gradient descent (SMGD) takes time to converge to the Pareto front with large neural networks and datasets. Instead, we explore the Pareto front as a manifold from a few initial optima, based on a predictor-corrector method. Second, for each exploration step, the predictor iteratively solves a large-scale linear system that scales quadratically in the number of model parameters, and requires one backpropagation to evaluate a second-order Hessian-vector product per iteration of the solver. We propose a Gauss-Newton approximation that scales linearly, and that requires only first-order i nner-product p er i teration. T hird, we explore different linear system solvers, including the MINRES and conjugate gradient methods for approximately solving the linear systems. The innovations make predictor-corrector efficient for large networks and datasets. Experiments on a fair misinformation detection task show that 1) the predictor-corrector method can find Pareto fronts better than or similar to SMGD with less time, and 2) the proposed first-order method does not harm the quality of the Pareto front identified b y t he second-order method, while further reducing running time.