On-line learning of a mixture-of-experts neural network
On-line learning of a mixture-of-experts neural network
复制标题
混合专家神经网络的在线学习
DOI：
10.1088/0305-4470/33/48/306
复制
发表时间：
2000
期刊：
影响因子：
--
通讯作者：
K. Kang
中科院分区：
文献类型：
--
作者：
Nam;J. Oh;K. Kang
在统计物理框架下研究了混合专家系统的在线学习问题。在热力学极限下，分析计算了训练过程中阶参数的时间依赖性。当训练样本数量较少时，每个专家都处于对称状态。当时间步数接近临界点时，对称状态开始瓦解。这种破坏行为是由一个门控网络。在对称状态下，门控网络对学习的影响很小，但当对称性被打破时，门控网络将专家分配到输入空间中适当的子空间。一般化曲线显示对称和破缺对称状态之间的平台。我们还发现，学习曲线显示不同的行为取决于刚度的门控功能。
The on-line learning of a mixture-of-experts system is studied in the framework of statistical physics. The time dependence of the overlap-order parameters during training is calculated analytically in the thermodynamic limit. When the number of training examples is small each expert is in a symmetric state. As the number of time steps approaches a critical point, the symmetric state begins to disintegrate. This symmetry-breaking behaviour is accounted for by means of a gating network. In the symmetric state the gating network has little effect on the learning, but when the symmetry is broken the gating network assigns the experts to appropriate subspaces in the input space. A generalization curve shows a plateau between the symmetric- and broken-symmetry states. We also find that the learning curves show different behaviours depending on the stiffness of the gating function.