Variational Optimization on Lie Groups, with Examples of Leading (Generalized) Eigenvalue Problems
Variational Optimization on Lie Groups, with Examples of Leading (Generalized) Eigenvalue Problems
复制标题
DOI：
--
复制
发表时间：
2020-01
期刊：
影响因子：
--
通讯作者：
Molei Tao;T. Ohsawa
中科院分区：
文献类型：
--
作者：
Molei Tao;T. Ohsawa
本文考虑李群上函数的光滑优化。通过在向量空间中推广NAG变分原理（Wibisono等人，2016）应用于李群，得到了保证收敛于局部最优的连续Lie-NAG动力学.它们对应于李群上梯度流的动量形式。然后详细研究了$\mathsf{SO}（n）$的一种特殊情况，目标函数对应于领先的广义特征值问题：首先在坐标中显式地表达Lie-NAG动力学，然后以结构保持的方式离散化，从而产生具有忠实能量行为（由于共形辛性）的优化算法，并精确地保留在李群上。随机梯度版本也进行了研究。数值实验的合成数据和实际问题（LDA MNIST）证明了所提出的方法作为优化算法的有效性（$不$作为分类方法）。
The article considers smooth optimization of functions on Lie groups. By generalizing NAG variational principle in vector space (Wibisono et al., 2016) to Lie groups, continuous Lie-NAG dynamics which are guaranteed to converge to local optimum are obtained. They correspond to momentum versions of gradient flow on Lie groups. A particular case of $\mathsf{SO}(n)$ is then studied in details, with objective functions corresponding to leading Generalized EigenValue problems: the Lie-NAG dynamics are first made explicit in coordinates, and then discretized in structure preserving fashions, resulting in optimization algorithms with faithful energy behavior (due to conformal symplecticity) and exactly remaining on the Lie group. Stochastic gradient versions are also investigated. Numerical experiments on both synthetic data and practical problem (LDA for MNIST) demonstrate the effectiveness of the proposed methods as optimization algorithms ($not$ as a classification method).