Statistical mechanics of learning multiple orthogonal signals: Asymptotic theory and fluctuation effects
Statistical mechanics of learning multiple orthogonal signals: Asymptotic theory and fluctuation effects
复制标题
DOI：
10.1103/physreve.75.016101
复制
发表时间：
2007-01-01
影响因子：
2.4
通讯作者：
Rattray, M.
中科院分区：
文献类型：
--
作者：
Hoyle, D. C.;Rattray, M.
通过正交分解或主成分分析（PCA）来学习高维数据中的信号方向在物理和工程学科中有许多重要的应用，例如无线通信、信息论和经济物理学。利用平均场理论可以研究正交分解的精度。先前对具有单一信号方向的模型产生的数据的分析预测了一个延迟的学习相变，在该相变下，学习是不可能的，即，如果信号太弱或数据集太小，则不可能学习任何关于信号方向或大小的信息。在这个贡献中，我们证明了结果可以推广到有多个信号方向的情况。每个非退化信号都与一个延迟学习过渡相关联。然而，平均场解周围的波动会导致大的有限尺寸效应，除非信号强度被很好地分离。我们评估了对平均场理论的单环贡献，该理论表明，如果信号方向对应的总体特征值被O（N-tau）与指数tau >(1/)(3)分开，则信号方向彼此不可区分，其中N是数据维数。数值模拟与分析结果一致，表明有限尺寸效应即使对于非常大的数据集也能持续存在。
The learning of signal directions in high-dimensional data through orthogonal decomposition or principal component analysis (PCA) has many important applications in physics and engineering disciplines, e. g., wireless communication, information theory, and econophysics. The accuracy of the orthogonal decomposition can be studied using mean-field theory. Previous analysis of data produced from a model with a single signal direction has predicted a retarded learning phase transition below which learning is not possible, i.e., if the signal is too weak or the data set is too small then it is impossible to learn anything about the signal direction or magnitude. In this contribution we show that the result can be generalized to the case where there are multiple signal directions. Each nondegenerate signal is associated with a retarded learning transition. However, fluctuations around the mean-field solution lead to large finite size effects unless the signal strengths are very well separated. We evaluate the one-loop contribution to the mean-field theory, which shows that signal directions are indistinguishable from one another if their corresponding population eigenvalues are separated by O(N-tau) with exponent tau > (1/)(3), where N is the data dimension. Numerical simulations are consistent with the analysis and show that finite size effects can persist even for very large data sets.