On-line Learning of Perceptron from Noisy Data by One and Two Teachers(General)
On-line Learning of Perceptron from Noisy Data by One and Two Teachers(General)
复制标题
一名和两名教师从噪声数据中在线学习感知器（通用）
DOI：
10.1143/jpsj.75.114007
复制
发表时间：
2006
期刊：
影响因子：
--
通讯作者：
Sachi Yamaguchi
中科院分区：
文献类型：
--
作者：
T. Uezu;Y. Maeda;Sachi Yamaguchi
我们分析了感知器从受外部噪声影响的单个感知器或由两个独立的无噪声感知器产生的信号中在线学习的问题。在单教师和双教师的情况下，我们采用典型的三种学习规则。对于单老师的情况，我们处理输入和输出噪声，对于双老师的情况，我们假设信号是由两个老师以确定的概率给出的。在单教师的情况下，为了在学生向量不收敛于教师向量的情况下提高学习效果，我们使用了两种方法：基于最优学习率的方法和平均方法。此外，我们利用三种学习规则的最优学习率得到了泛化误差的渐近形式，并利用模拟数据通过平均方法估计了噪声参数。在双师情况下，对于Hebbian规则，我们给出了序参量的解析解。在此基础上，利用感知机规则对噪声参数进行平均估计。理论计算结果与数值模拟结果吻合较好。
We analyze the on-line learning of a Perceptron from signals produced by a single Perceptron suffering from external noise or by two independent Perceptrons without noise. We adopt typical three learning rules in both single-teacher and two-teacher cases. For the single-teacher case, we treat the input and output noises and for the two-teacher case, we assume that signals are given by two teachers with a definite probability. In the single-teacher case, in order to improve the learning when it does not succeed in the sense that the student vector does not converge to the teacher vector, we use two methods: a method based on the optimal learning rate and an averaging method. Furthermore, we obtain an asymptotic form of the generalization error using an optimal learning rate for the three learning rules, and we estimate noise parameters using the simulation data by the averaging method. In the two-teacher case, for the Hebbian rule, we give analytical solutions of order parameters. Furthermore, we estimate noise parameters using the Perceptron rule by the averaging method. The theoretical results agree quite well with the numerical simulations.