The Geometry of Nonlinear Embeddings in Kernel Discriminant Analysis
The Geometry of Nonlinear Embeddings in Kernel Discriminant Analysis
复制标题
DOI：
10.1109/tpami.2022.3192726
复制
发表时间：
2020-05
影响因子：
23.6
通讯作者：
Jiae Kim;Yoonkyung Lee;Zhiyu Liang
中科院分区：
文献类型：
--
作者：
Jiae Kim;Yoonkyung Lee;Zhiyu Liang
Fisher线性判别分析是一种经典的分类方法，但它仅限于捕获线性特征。核判别分析作为一种扩展，通过非线性特征映射成功地缓解了这一限制。我们研究了多项式核和高斯核判别分析中的非线性嵌入的几何结构，通过识别依赖于数据分布和核的总体级判别函数。为了获得判别函数，我们解决了广义特征值问题与类间和类内协方差算子。多项式判别式捕捉类的差异，通过人口的时刻明确。对于高斯判别式的近似，我们使用一个特定的表示的高斯核，利用指数生成函数的厄米多项式。我们还表明，高斯判别式可以近似使用随机投影的数据。我们的研究结果说明了数据分布和核函数在非线性嵌入判别中的相互作用，并为核函数及其参数的选择提供了指导。
Fisher's linear discriminant analysis is a classical method for classification, yet it is limited to capturing linear features only. Kernel discriminant analysis as an extension is known to successfully alleviate the limitation through a nonlinear feature mapping. We study the geometry of nonlinear embeddings in discriminant analysis with polynomial kernels and Gaussian kernel by identifying the population-level discriminant function that depends on the data distribution and the kernel. In order to obtain the discriminant function, we solve a generalized eigenvalue problem with between-class and within-class covariance operators. The polynomial discriminants are shown to capture the class difference through the population moments explicitly. For approximation of the Gaussian discriminant, we use a particular representation of the Gaussian kernel by utilizing the exponential generating function for Hermite polynomials. We also show that the Gaussian discriminant can be approximated using randomized projections of the data. Our results illuminate how the data distribution and the kernel interact in determination of the nonlinear embedding for discrimination, and provide a guideline for choice of the kernel and its parameters.