An analysis of correctness for API recommendation: are the unmatched results useless?
An analysis of correctness for API recommendation: are the unmatched results useless?
复制标题
API推荐正确性分析：不匹配的结果就没用了吗？
DOI：
10.1007/s11432-019-2929-9
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Bixin Li
中科院分区：
文献类型：
--
作者：
Xianglong Kong;Weina Han;Li Liao;Bixin Li
API推荐是一种很有前途的方法，在软件开发过程中广泛使用。然而，API 推荐的评估尚未得到足够严格的探索。目前API推荐的评估主要侧重于正确性，通过将推荐结果与ground-truth结果进行匹配来进行衡量。在大多数情况下，每次推荐尝试只有一组真实 API，但目标代码可以通过多种方式实现。忽视代码多样性可能导致评估出现缺陷。为了解决这个问题，我们邀请了 15 名开发人员来分析用户研究中无与伦比的结果。在线评估证实，由于与真实 API 的功能相关性，一些无与伦比的 API 也可以受益于编程。然后，我们根据从API知识图谱、API方法名称和API文档中提取的关系来衡量API功能相关性。此外，我们提出了一种基于 API 功能相关性来改进正确性测量的方法。我们的测量是在 6141 个需求的数据集和相关提交的历史代码片段上进行评估的。结果表明，28.2% 的不匹配 API 有助于我们实验的正确性。
API recommendation is a promising approach which is widely used during software development. However, the evaluation of API recommendation is not explored with sufficient rigor. The current evaluation of API recommendation mainly focuses on correctness, the measurement is conducted by matching recommended results with ground-truth results. In most cases, there is only one set of ground-truth APIs for each recommendation attempt, but the object code can be implemented in dozens of ways. The neglect of code diversity results in a possible defect in the evaluation. To address the problem, we invite 15 developers to analyze the unmatched results in a user study. The online evaluation confirms that some unmatched APIs can also benefit to programming due to the functional correlation with ground-truth APIs. Then we measure the API functional correlation based on the relationships extracted from API knowledge graph, API method name, and API documentation. Furthermore, we propose an approach to improve the measurement of correctness based on API functional correlation. Our measurement is evaluated on a dataset of 6141 requirements and historical code fragments from related commits. The results show that 28.2% of unmatched APIs can contribute to correctness in our experiments.