Mining search engine query logs via suggestion sampling
Mining search engine query logs via suggestion sampling
复制标题
DOI：
10.14778/1453856.1453868
复制
发表时间：
2008-08
期刊：
影响因子：
--
通讯作者：
Ziv Bar-Yossef;M. Gurevich
中科院分区：
文献类型：
--
作者：
Ziv Bar-Yossef;M. Gurevich
许多搜索引擎和其他Web应用程序建议在用户键入查询时自动完成。这些建议是从隐藏的底层数据库中生成的，比如查询日志、目录和词典。这些数据库包含有趣且有用的信息，但通常无法直接访问。在本文中，我们描述了两个算法抽样建议只使用公共建议接口。其中一个算法随机均匀地对建议进行采样，而另一个算法则根据建议的受欢迎程度按比例对建议进行采样。这些算法可以用来挖掘隐藏的建议数据库。示例应用包括比较搜索引擎的查询日志中给定关键字的流行度、估计查询日志中面向商业的查询的量以及评估搜索引擎将其用户暴露于负面内容的程度。我们的算法采用蒙特卡罗方法，以获得无偏的建议数据库中的样本。使用公开的查询日志的实证分析表明，我们的算法是有效的和准确的。还提供了两个主要的建议服务的实验结果。
Many search engines and other web applications suggest auto-completions as the user types in a query. The suggestions are generated from hidden underlying databases, such as query logs, directories, and lexicons. These databases consist of interesting and useful information, but they are typically not directly accessible. In this paper we describe two algorithms for sampling suggestions using only the public suggestion interface. One of the algorithms samples suggestions uniformly at random and the other samples suggestions proportionally to their popularity. These algorithms can be used to mine the hidden suggestion databases. Example applications include comparison of popularity of given keywords within a search engine's query log, estimation of the volume of commercially-oriented queries in a query log, and evaluation of the extent to which a search engine exposes its users to negative content. Our algorithms employ Monte Carlo methods in order to obtain unbiased samples from the suggestion database. Empirical analysis using a publicly available query log demonstrates that our algorithms are efficient and accurate. Results of experiments on two major suggestion services are also provided.