On Mean-Optimal Robust Linear Discriminant Analysis
On Mean-Optimal Robust Linear Discriminant Analysis
复制标题
DOI：
10.1109/icdm54844.2022.00129
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Xiangyu Li;Hua Wang
中科院分区：
文献类型：
--
作者：
Xiangyu Li;Hua Wang
线性判别分析（LDA）被广泛用于监督学习环境下的降维。传统的LDA目标是最小化平方欧几里德距离的比率，这可能不会在噪声数据集上表现得最佳。已经提出了多个鲁棒LDA目标来解决这个问题，但它们的实现有两个主要的限制。一个是他们的均值计算使用平方$\ell_{2}$-范数距离来集中数据，当目标不使用欧几里得距离时，这是无效的。第二个问题是，没有广义优化算法来解决不同的鲁棒LDA目标。此外，现有的大多数算法只能保证解是局部最优，而不是全局最优。在本文中，我们回顾了多个鲁棒损失函数，并提出了一个新的和广义的鲁棒目标LDA。此外，为了更好地去除数据中的平均值，我们的目标使用了一种最佳的方法来通过学习来集中数据。作为一个重要的算法贡献，我们得到了一个有效的迭代算法来优化所得到的非光滑和非凸的目标函数。我们从理论上证明，我们的解决方案的算法保证了目标和解决方案序列收敛到全局最优解的次线性收敛速度。实验结果表明，我们的新方法的有效性，取得了显着的改善相比，其他竞争的方法。
Linear discriminant analysis (LDA) is widely used for dimensionality reduction under supervised learning settings. Traditional LDA objective aims to minimize the ratio of squared Euclidean distances that may not perform optimally on noisy data sets. Multiple robust LDA objectives have been proposed to address this problem, but their implementations have two major limitations. One is that their mean calculations use the squared $\ell_{2}$-norm distance to center the data, which is not valid when the objective does not use the Euclidean distance. The second problem is that there is no generalized optimization algorithm to solve different robust LDA objectives. In addition, most existing algorithms can only guarantee the solution to be locally optimal, rather than globally optimal. In this paper, we review multiple robust loss functions and propose a new and generalized robust objective for LDA. Besides, to better remove the mean value within data, our objective uses an optimal way to center the data through learning. As one important algorithmic contribution, we derive an efficient iterative algorithm to optimize the resulting non-smooth and non-convex objective function. We theoretically prove that our solution algorithm guarantees that both the objective and the solution sequences converge to globally optimal solutions at a sub-linear convergence rate. The experimental results demonstrate the effectiveness of our new method, achieving significant improvements compared to the other competing methods.