STAMINA: a competition to encourage the development and assessment of software model inference techniques
STAMINA: a competition to encourage the development and assessment of software model inference techniques
复制标题
DOI：
10.1007/s10664-012-9210-3
复制
发表时间：
2012-05
影响因子：
4.1
通讯作者：
Neil Walkinshaw;Bernard Lambeau;Christophe Damas;K. Bogdanov;P. Dupont
中科院分区：
文献类型：
--
作者：
Neil Walkinshaw;Bernard Lambeau;Christophe Damas;K. Bogdanov;P. Dupont
模型在软件系统的开发和维护中发挥着至关重要的作用，但由于生成模型需要大量的手动工作，因此在开发过程中经常被忽视。为了解决这个问题，人们开发了许多技术，寻求借助机器学习领域日益精确的算法来自动化模型生成任务。从实证角度来看，这些比较起来极具挑战性；有许多难以控制的因素（例如输入的丰富性和主题系统的复杂性），以及许多同样麻烦的实际问题（例如工具可用性）。本文介绍了 StaMinA（StateMachineInferenceApproaches）竞赛，旨在解决这些问题。此次竞赛吸引了众多参赛作品，其中许多是技术的改进或改编版本，尚未经过广泛的实证评估，也未对其推断软件系统模型的能力进行评估。本文展示了其中有多少技术大大改进了现有技术，提供了对可能支撑最佳技术成功的一些因素的见解。从更一般的意义上来说，它展示了竞赛作为经验软件工程的有用基础的潜力，通过（a）刺激新技术的开发和（b）促进它们的比较评估，达到通常在没有开发人员积极参与的情况下具有挑战性的程度。
Models play a crucial role in the development and maintenance of software systems, but are often neglected during the development process due to the considerable manual effort required to produce them. In response to this problem, numerous techniques have been developed that seek to automate the model generation task with the aid of increasingly accurate algorithms from the domain of Machine Learning. From an empirical perspective, these are extremely challenging to compare; there are many factors that are difficult to control (e.g. the richness of the input and the complexity of subject systems), and numerous practical issues that are just as troublesome (e.g. tool availability). This paper describes the StaMinA (StateMachineInferenceApproaches) competiton, that was designed to address these problems. The competition attracted numerous submissions, many of which were improved or adapted versions of techniques that had not been subjected to extensive empirical evaluations, and had not been evaluated with respect to their ability to infer models of software systems. This paper shows how many of these techniques substantially improve on the state of the art, providing insights into some of the factors that could underpin the success of the best techniques. In a more general sense it demonstrates the potential for competitions to act as a useful basis for empirical software engineering by (a) spurring the development of new techniques and (b) facilitating their comparative evaluation to an extent that would usually be prohibitively challenging without the active participation of the developers.