A novel focused crawler based on cell-like membrane computing optimization algorithm
A novel focused crawler based on cell-like membrane computing optimization algorithm
复制标题
一种新型的基于类细胞膜计算优化算法的聚焦爬虫
DOI：
10.1016/j.neucom.2013.06.039
复制
发表时间：
2014
期刊：
影响因子：
6
通讯作者：
YaJun Du
中科院分区：
文献类型：
--
作者：
WenJun Liu;YaJun Du
在许多研究工作中，未访问超链接的主题优先级是基于各种文本的主题相关相似性和相应的加权因子的线性积分来计算的。然而，这些加权因素是根据个人经验确定的，因此这些值可能会使未访问超链接的主题优先级直接严重偏差。为了解决这一问题，本文提出了一种基于类细胞膜计算优化算法（CMCFC）的聚焦爬虫。CMCFC将各文本相似性贡献度对应的所有加权因子作为一个目标，利用膜内的演化规则和传播规则来实现最优加权因子对应的最优目标，使超链接优先级的均方根误差（RMS）达到最小。然后，利用向量空间模型（VSM）对各种文本的最优加权因子和相应主题相似度进行线性整合，计算出未访问超链接的优先级；CMCFC获取更准确的未访问url优先级，引导爬虫收集更高质量的网页。实验结果表明，该方法通过智能确定加权因子，提高了聚焦爬虫的性能。综上所述，该方法对于聚焦爬虫来说是有效且有意义的。
In many research works, topical priorities of unvisited hyperlinks are computed based on linearly integrating topic-relevant similarities of various texts and corresponding weighted factors. However, these weighted factors are determined based on the personal experience, so that these values may make topical priorities of unvisited hyperlinks serious deviations directly. To solve this problem, this paper proposes a novel focused crawler applying the cell-like membrane computing optimization algorithm (CMCFC). The CMCFC regards all weighted factors corresponding to contribution degrees of similarities of various texts as one object, and utilizes evolution regulars and communication regulars in membranes to achieve the optimal object corresponding to the optimal weighted factors, which make the root measure square error (RMS) of priorities of hyperlinks achieve the minimum. Then, it linearly integrates optimal weighted factors and corresponding topical similarities of various texts, which are computed by using a Vector Space Model (VSM), to compute priorities of unvisited hyperlinks. The CMCFC obtains more accurate unvisited URLs' priorities to guide crawlers to collect higher quality web pages. The experimental results indicate that the proposed method improves the performance of focused crawlers by intelligently determining weighted factors. In conclusion, the mentioned approach is effective and significant for focused crawlers.
登录
查看更多内容
DOI：
10.1007/978-3-540-85984-0_101
发表时间：
2008-09
期刊：
--
影响因子：
--
作者：
Yuekui Yang;Yajun Du;Jingyu Sun;Yufeng Hai
通讯作者：
Yuekui Yang;Yajun Du;Jingyu Sun;Yufeng Hai
DOI：
10.1007/978-3-642-23190-2_5
发表时间：
2012
期刊：
Studies in computational intelligence
影响因子：
--
作者：
Yuki Uemura;Tsuyoshi Itokawa;T. Kitasuka;M. Aritsugi
通讯作者：
Yuki Uemura;Tsuyoshi Itokawa;T. Kitasuka;M. Aritsugi
DOI：
10.1016/s1389-1286(99)00052-3
发表时间：
1999-05-17
期刊：
COMPUTER NETWORKS-THE INTERNATIONAL JOURNAL OF COMPUTER AND TELECOMMUNICATIONS NETWORKING
影响因子：
--
作者：
Chakrabarti, S;van den Berg, M;Dom, B
通讯作者：
Dom, B
影响因子：
1.1
作者：
Martín-Vide, C;Paun, G;Rodríguez-Patón, A
通讯作者：
Rodríguez-Patón, A
影响因子：
22.7
作者：
SALTON, G;WONG, A;YANG, CS
通讯作者：
YANG, CS