Optimization of the Asymptotic Property of Mutual Learning Involving an Integration Mechanism of Ensemble Learning
Optimization of the Asymptotic Property of Mutual Learning Involving an Integration Mechanism of Ensemble Learning
复制标题
涉及集成学习集成机制的相互学习渐近性质的优化
DOI：
10.1143/jpsj.77.024005
复制
发表时间：
2007
影响因子：
1.7
通讯作者：
T. Yamada
中科院分区：
文献类型：
--
作者：
K. Hara;T. Yamada
提出了一种在线学习框架下收敛到最优集成学习相同状态的互学习优化方法，并利用统计力学方法分析了其渐近性质.该模型包括两个学习步骤：两个学生独立地向教师学习，然后学生通过相互学习相互学习。在交互学习中，即使教师没有参与，学生也可以相互学习，泛化错误得到改善。然而，在教师和学生之间的不同初始重叠（方向余弦）的情况下，具有较大初始重叠的学生倾向于具有比相互学习之前更大的泛化误差。为了克服这个问题，我们提出的相互学习的优化方法优化两个学生的步长，以尽量减少泛化误差的渐近性质。因此，优化的相互学习收敛到与最优集成学习相同的泛化误差。此外，我们之间的相互学习的最佳步长和集成学习的集成机制的关系。
We propose an optimization method of mutual learning which converges into the identical state of optimum ensemble learning within the framework of on-line learning, and have analyzed its asymptotic property through the statistical mechanics method. The proposed model consists of two learning steps: two students independently learn from a teacher, and then the students learn from each other through the mutual learning. In mutual learning, students learn from each other and the generalization error is improved even if the teacher has not taken part in the mutual learning. However, in the case of different initial overlaps (direction cosine) between teacher and students, a student with a larger initial overlap tends to have a larger generalization error than that of before the mutual learning. To overcome this problem, our proposed optimization method of mutual learning optimizes the step sizes of two students to minimize the asymptotic property of the generalization error. Consequently, the optimized mutual learning converges to a generalization error identical to that of the optimal ensemble learning. In addition, we show the relationship between the optimum step size of the mutual learning and the integration mechanism of the ensemble learning.
线性感知的集成学习：在线学习理论
DOI：
--
发表时间：
2005
期刊：
Journal of physical society of Japan 74・11
影响因子：
--
作者：
Hara;K;Okada;M.
通讯作者：
M.