Toward an automated benchmark management system
Toward an automated benchmark management system
复制标题
迈向自动化基准管理系统
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
E. Bodden
中科院分区：
文献类型：
--
作者：
Lisa Nguyen Quang Do;Michael Eichberg;E. Bodden
对程序分析以及软件工程工具的系统评估需要基准套件，这些套件代表了设计工具或分析的域中现实世界项目。此类基准目前仅存在一些研究领域，甚至在存在的地方，由于所需的手动努力，通常无法有效地维护它们。这使得评估依赖当前技术的软件上的新分析和工具通常不可能。我们描述了ABM，这是一种半自动地挖掘软件存储库的方法，以提取属于特定域的最新和代表性的应用程序集。提出的方法促进了此类收藏的创建，并使发布基准套件的更新版本变得更加容易。由于该方法的实例化，我们提供了当前现实世界中的Java业务Web应用程序的集合。该收集和方法是创建当前，有针对性的基准套件的起点，因此有助于更好地评估当前的程序分析和软件工程工具。
The systematic evaluation of program analyses as well as software-engineering tools requires benchmark suites that are representative of real-world projects in the domains for which the tools or analyses are designed. Such benchmarks currently only exist for a few research areas and even where they exist, they are often not effectively maintained, due to the required manual effort. This makes evaluating new analyses and tools on software that relies on current technologies often impossible. We describe ABM, a methodology to semi-automatically mine software repositories to extract up-to-date and representative sets of applications belonging to specific domains. The proposed methodology facilitates the creation of such collections and makes it easier to release updated versions of a benchmark suite. Resulting from an instantiation of the methodology, we present a collection of current real-world Java business web applications. The collection and methodology serve as a starting point for creating current, targeted benchmark suites, and thus helps to better evaluate current program-analysis and software-engineering tools.