Study on cooperation mechanism and it's dynamic behavior of many learning machines

包围学习算法，如bagging和Ada-boost，试图通过使用许多弱学习机来改善弱学习机的性能;这种学习算法最近受到了相当大的关注。我们已经分析了集成学习的泛化误差的动态，通过使用统计力学方法的框架内的在线学习。在此框架内，教师和初始学生权重向量之间的重叠（或方向余弦）在集成学习中起着重要作用。当教师和学生之间的重叠是同质的，学生输出的简单平均值可以用作集成学习（装袋）的集成方法。从我们的分析中，我们发现，泛化误差是一个单一的线性感知器的线性感知器的数量K成为无限的情况下，无噪声的一半。此外，我们还发现，推广误差收敛于 关于我们 当线性感知器的数目对于无噪声情况和有噪声情况都是有限的时，e无限情况为0（1/K）。在非齐次的情况下，可以通过引入权重来平均学习机的输出来改善泛化误差（即，使用加权平均值而不是简单平均值），并且权重应当适于最小化泛化误差（即，并联升压）。在集成学习中，学生之间没有互动。在相互学习中，学习是在两个学生之间进行的，他们事先从老师那里学习。因此，每个学生从老师那里获得的知识被交换，这可以提高学生的表现。此外，交互可以模仿集成学习的集成机制。我们证明了相互学习渐近收敛到装袋。此外，具有用于相互学习的较大初始重叠的学生在步长极限为零的学习期间短暂地通过并行提升的状态。少

Ensemble learning algorithms, such as bagging and Ada-boost, try to improve upon the performance of a weak learning machine by using many weak learning machines ; such learning algorithms have recently received considerable attention. We have analyzed the dynamics of the generalization error of ensemble learning by using statistical mechanics methods within the framework of on-line learning. Within this framework, the overlap (or direction cosine) between the teacher and the initial student weight vectors plays important roles in ensemble learning. When overlaps between the teacher and the students are homogeneous, a simple average of the student outputs can be used as an integration method for ensemble learning (bagging). From our analysis, we found that the generalization error was equal to half that of a single linear perceptron when the number of linear perceptrons K became infinite for the no noise case. In addition, we found that the generalization error converged with that of th … More e infinite case with 0(1/K) when the number of linear perceptrons was finite for both the no noise case and the noisy case. In an inhomogeneous case, the generalization error can be improved by introducing weights to average the outputs of the learning machines (i.e., to use a weighted average rather than a simple average), and the weights should be adapted to minimize the generalization error (i.e., parallel boosting). In ensemble learning, there is no interaction between the students. In mutual learning, learning is performed between two students who learn from a teacher in advance. Therefore, the knowledge each student has obtained from the teacher is exchanged, which may improve the performance of the students. Moreover, the interaction may mimic the integration mechanism of ensemble learning. We showed that the mutual learning asymptotically converged into bagging. Moreover, a student with a larger initial overlap for mutual learning transiently passes through a state of parallel boosting during the learning in the limit of step size goes to zero. Less