StaMInA: A Novel Competition to Drive the Comparative Evaluation of State Machine Inference Approaches

StaMINA：推动状态机推理方法比较评估的新颖竞赛

• 批准号: EP/H002456/1
• 负责人: Kirill Bogdanov
• 金额: $ 2.52万
• 依托单位: University of Sheffield
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2009
• 资助国家: 英国
• 起止时间: 2009 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FH002456%2F1
• 关键词: StaMInA; Novel; Competition; Drive; Comparative

Software systems pervade modern life; they control everything from fly-by-wire aircraft and financial transfer systems to ABS breaking systems in cars and cooking modes in microwaves. The ability to understand these complex systems, and to make sure that they behave as expected, is crucial. State machines are a formal, diagrammatic notation that can be used to visualise behaviour of these systems in an accessible way. They can also be used as a basis for several rigorous and automated testing and verification techniques.Currently, state machines have to be designed and maintained by hand. This is an expensive and error-prone task, particularly when the system in question is constantly subject to change. Faced with this challenge, a substantial amount of research has been devoted to solving this problem with automated techniques; to automatically infer state machines of software systems, usually from samples of their behaviour. This has resulted in a multitude of proposed solutions from groups around the world.Although these advances are welcome, they have given rise to an important problem: There is no accepted process by which these techniques can be evaluated and compared against each other. There is no evidence to indicate which technique is better than the others, and why certain techniques excel. This in turn hampers further research in the area.With this project, we will address the above problems by organising an international competition to thoroughly compare and evaluate a diverse range of state machine inference techniques. The competition is especially novel because it will employ a range of techniques to compare the results of different techniques against each other. This will identify (a) which techniques are the most effective ones and (b) shed light on the possible reasons for their effectiveness. It is envisaged that this competition will become a regular event, driving research in the area.

软件系统在现代生活中无处不在;它们控制着从电传飞行器和金融转账系统到汽车中的ABS制动系统和微波炉中的烹饪模式的一切。理解这些复杂系统并确保它们按预期运行的能力至关重要。状态机是一种正式的、图形化的符号，可以用来以一种可访问的方式可视化这些系统的行为。它们也可以作为一些严格的自动化测试和验证技术的基础。目前，状态机必须手工设计和维护。这是一项昂贵且容易出错的任务，特别是当所讨论的系统不断变化时。面对这一挑战，大量的研究一直致力于解决这个问题的自动化技术;自动推断软件系统的状态机，通常从他们的行为样本。这导致了来自世界各地的团体提出了大量的解决方案。尽管这些进步是受欢迎的，但它们也引发了一个重要的问题：没有一个公认的过程可以对这些技术进行评估和比较。没有证据表明哪种技术比其他技术更好，以及为什么某些技术优于其他技术。这反过来又阻碍了该领域的进一步研究。在这个项目中，我们将通过组织一个国际比赛来彻底比较和评估各种状态机推理技术，以解决上述问题。比赛特别新颖，因为它将采用一系列技术来比较不同技术的结果。这将确定（a）哪些技术是最有效的技术，以及（B）阐明其有效性的可能原因。预计这一竞赛将成为一项定期活动，推动该领域的研究。

项目成果

A framework for the competitive evaluation of model inference techniques

模型推理技术竞争性评估框架

• DOI: 10.1145/1868044.1868045
• 发表时间: 2010
• 作者: Walkinshaw N

Using compression algorithms to support the comprehension of program traces

• DOI: 10.1145/1868321.1868323
• 发表时间: 2010-07
• 作者: Neil Walkinshaw;S. Afshan;Phil McMinn