Evaluating How Developers Use General-Purpose Web-Search for Code Retrieval
Evaluating How Developers Use General-Purpose Web-Search for Code Retrieval
复制标题
DOI：
10.1145/3196398.3196425
复制
发表时间：
2018-03
期刊：
影响因子：
--
通讯作者：
Md Masudur Rahman;J. Barson;Sydney Paul;Joshua Kayan;F. Lois;S. Quezada;Chris Parnin;Kathryn T. Stolee;Baishakhi Ray
中科院分区：
文献类型：
--
作者：
Md Masudur Rahman;J. Barson;Sydney Paul;Joshua Kayan;F. Lois;S. Quezada;Chris Parnin;Kathryn T. Stolee;Baishakhi Ray
搜索是软件开发过程中不可或缺的一部分。开发人员经常使用搜索引擎在开发过程中查找信息，包括可重复使用的代码片段，API理解和参考示例。开发人员倾向于更喜欢通用搜索引擎，例如Google，这些搜索引擎通常不会针对与代码相关的文档进行优化，并使用搜索策略和排名技术，这些技术对通用，非代码相关的信息进行了更优化。在本文中，我们探讨了像Google这样的通用搜索引擎是否是与代码相关搜索的最佳选择。特别是，我们调查了使用Google搜索的性能是否有代码与非代码相关搜索的变化。为了进行分析，我们从310个开发人员那里收集搜索日志，其中包含Google的近150,000个搜索查询以及相关的结果点击。要在与代码相关的搜索和与非代码相关的搜索之间进行介绍，我们构建了一个模型，该模型标识了查询的代码。利用此模型，我们构建了一个自动分类器，该分类器检测代码和非代码相关查询。我们在手动注释的查询上确认了分类器的E extiversive，分类器的精度为87％，召回86％，F1分数为87％。我们将此分类器应用于自动注释数据集中的所有查询。分析此数据集时，我们观察到与代码相关的搜索通常需要更多的e？ort（例如，时间，结果点击和查询修改）比一般的非代码搜索搜索，这表明使用一般搜索引擎指示代码搜索性能的效率较小。
Search is an integral part of a software development process. Developers often use search engines to look for information during development, including reusable code snippets, API understanding, and reference examples. Developers tend to prefer general-purpose search engines like Google, which are often not optimized for code related documents and use search strategies and ranking techniques that are more optimized for generic, non-code related information. In this paper, we explore whether a general purpose search engine like Google is an optimal choice for code-related searches. In particular, we investigate whether the performance of searching with Google varies for code vs. non-code related searches. To analyze this, we collect search logs from 310 developers that contains nearly 150,000 search queries from Google and the associated result clicks. To di?erentiate between code-related searches and non-code related searches, we build a model which identifies code intent of queries. Leveraging this model, we build an automatic classifier that detects a code and non-code related query. We confirm the e?ectiveness of the classifier on manually annotated queries where the classifier achieves a precision of 87%, a recall of 86%, and an F1-score of 87%. We apply this classifier to automatically annotate all the queries in the dataset. Analyzing this dataset, we observe that code related searching often requires more e?ort (e.g., time, result clicks, and query modifications) than general non-code search, which indicates code search performance with a general search engine is less effective.