Hermes: assessment and creation of effective test corpora
Hermes: assessment and creation of effective test corpora
复制标题
Hermes：评估和创建有效的测试语料库
DOI：
10.1145/3088515.3088523
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
M. Mezini
中科院分区：
文献类型：
--
作者：
Michael Reif;Michael Eichberg;Ben Hermann;M. Mezini
开发新分析的一个组成部分是验证其实现的正确性，并在应用于现实的代码时证明其有用性希望收集的项目代表了分析的目标领域，因此确保了合理的评估。此外，该系列可能包含许多项目，这些项目基本上是相同的，因此没有帮助分析的整体评估/测试一个框架，可以对给定语料库进行系统评估，并创建新的Java项目。计算所有QC项目的最小子集，可用于通用数据和控制流分析。
An integral part of developing a new analysis is to validate the correctness of its implementation and to demonstrate its usefulness when applied to real-world code. As a foundation for addressing both challenges developers typically use custom or well-established collections of Java projects. The hope is that the collected projects are representative for the analysis’ target domain and therefore ensure a sound evaluation. But, without proper means to understand how and to which degree the features relevant to an analysis are found in the projects, the evaluation necessarily remains inconclusive. Additionally, it is likely that the collection contains many projects which are – w.r.t. the developed analysis – basically identical and therefore do not help the overall evaluation/testing of the analysis, but still cost evaluation time. To overcome these limitations we propose Hermes, a framework that enables the systematic assessment of given corpora and the creation of new corpora of Java projects. To show the usefulness of Hermes, we used it to comprehend the nature of the projects belonging to the Qualitas Corpus (QC) and then used it to compute a minimal subset of all QC projects useful for generic data- and control-flow analyses. This subset enables effective and efficient integration test suites.