IMPROVEMENT OF CONVERGENCE OF LEARNING OF MULTI-LAYER NEURAL NETWORKS AND APPLICATION FOR SEARCH ENGINE

多层神经网络学习收敛性的提高及搜索引擎的应用

• 批准号: 13680472
• 负责人: HARA Kazuyuki
• 金额: $ 0.83万
• 依托单位: TOKYO METROPOLITAN COLLEGE OF TECHNOLOGY
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 2001
• 资助国家: 日本
• 起止时间: 2001 至 2002
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-13680472/
• 关键词: Multilayer nerural networks; Error distribution; improving convergence of learning; margin; ensemble learning; on-line learning; 単純パーセプトロン; 階層形ニューラルネットワーク; 収束性の改善; 例題数の偏り; 対称性の破壊

In this study, we investigated improvement of convergence of learning of the multi-layer neural networks and its application for search engine. Abstract of the results are follows :(1) Baised Classification with number of the data in the class. Learning method of probability of updating the connection weight is proportional to the mean equated error, is investigated. It keeps balance of the number of the data express large error and small error, then minority class becomes learnable.(2)Learning method to obtain Early symmetry braking.We investigated the learning method for mutilayer neural networks updating only one connection weight to avoid the stopping of the learning.(3)Perceptron learning with a margin.We introduced a margin a la Gardner to improve the perceptron learning. Our algorithm is superior to Hebbian learning at the early stage of the learning.(4)Analysis of ensemble learning through linear learning machine.We analyzed generalization error of ensemble learning related to the number of the weak learner K. As the result, it has been shown that at the limit of K goes to infinity, the generalization error is a half of that of single percentron.

在这项研究中，我们研究了提高学习的多层神经网络的收敛性，并将其应用于搜索引擎。结果摘要如下：（1）基于类内数据个数的分类。研究了连接权更新概率与平均等效误差成正比的学习方法。它保持了表示大误差和小误差的数据的数量的平衡，从而使少数类变得可学习。（2）实现早期对称制动的学习方法研究了多层神经网络只更新一个连接权值以避免学习停止的学习方法。（3）带间隔的感知器学习，引入了Gardner间隔，改进了感知器学习。该算法在学习的早期阶段比Hebbian学习有上级的优势。（4）基于线性学习机的集成学习分析，分析了集成学习的泛化误差与弱学习器数目K的关系。结果表明，当K趋于无穷大时，其推广误差为单一百分位数时的一半。

项目成果

原 一之, 岡田 真人: "オンライン学習理論に基づく線形学習機械のアンサンブル学習の解析"日本物理学会秋期大会概要集. 221 (2002)

Kazuyuki Hara、Masato Okada：“基于在线学习理论的线性学习机的集成学习分析”日本物理学会秋季会议摘要 221（2002）。

KAZUYUKI HARA, MASATO OKADA: "ANALYSIS OF EMSEMBLE LEARNIG THROUGH LINEAR LEARNING MACHINE"MEETING ABSTRACTS OF THE PHYSICAL SOCIETY OF JAPAN. VOL. 57, ISSUE2, PART 2. 221

KAZUYUKI HARA、MASATO OKADA：“通过线性学习机进行 EMSEMBLE 学习的分析”日本物理学会会议摘要。

KAZUYUKI HARA: "A NOVEL LINE SEARCH TYPE ALGORITHM AVOIDABLE OF SMALL LOCAL MINIMA"PROCEEDINGS OF INTERNATIONAL CONFERENCE OF NEURAL NETWORKS. 2048-2053 (2001)

原一之：“一种新颖的线搜索型算法，可避免局部极小值”国际神经网络会议论文集。

Aazuyuki Hara, Yoshihisa Amakata, Ryohei Nukaga, Kenji Nakavama: "A learning method by stochastic connection weight update"Proceedings of International joint conference on neural network. 2036-2041 (2001)

Aazuyuki Hara、Yoshihisa Amakata、Ryohei Nukaga、Kenji Nakavama：“一种基于随机连接权重更新的学习方法”国际神经网络联合会议论文集。

Kazuyuki Hara, Hiroyuki Nose, Megumi Ohwada: "A Novel Line Search Type Algorithm Avoidable of Small Local Minima"Proceedings of International joint conference on neural network. 2048-2053 (2001)

Kazuyuki Hara、Hiroyuki Nose、Megumi Ohwada：“A Novel Line Search Type Algorithm Saving of Small Local Minima”神经网络国际联合会议论文集。