II-NEW: Collaborative Research: COMET: A Web Infrastructure for Research and Experimentation in User Interactive Event Driven Testing

如今，用户交互事件驱动的软件非常普遍。最终用户通过指向、点击或触摸界面与应用程序交互，当发生这种情况时，程序会做出响应。通过软件测试确保这些系统的可靠性是至关重要的，因为测试技术不足目前每年给美国经济造成数十亿美元的损失。然而，这些系统的灵活性，使它们对用户具有吸引力，也增加了测试的难度。这一困难推动了大量关于用户交互事件驱动测试的研究，但新开发的技术通常使用孤立的案例研究和实验进行评估。用户交互事件驱动测试社区缺乏一套通用的基准和工具来评估他们的新方法，导致实验不匹配；一个研究的结果很难与另一个进行比较。缺乏共同的基准也意味着我们不能轻易地将多项研究的结果结合起来建立更大的知识体系。该项目通过开发名为Comet的共享研究和实验网络基础设施来减少不匹配，并正在推进用户交互事件驱动测试研究。彗星的初步概念证明是通过美国国家科学基金会的早期支持而开发的。导致不匹配的因素包括特定于平台的测试方法的开发、需要为每个测试对象应用程序和每个操作系统进行定制的测试工具，以及彼此不兼容的模型。这项研究将设计新的技术来控制测试环境，将可能影响实验结果的因素与环境联系起来，并将允许人工制品随着时间的推移而演变和变化。它正在建立一个共享和可扩展的网络基础设施，其中包括基准、工具、模型和测试人工制品，这将使科学发现在最先进的用户交互事件驱动测试中成为可能。Comet是一个公共资源，将向更广泛的社区开放。它的影响不仅将扩展到用户交互事件测试研究社区，还将扩展到其他使用用户界面的人，例如那些研究可用性的人，以及整个行业和软件测试社区。项目工作将涉及研究生和本科生。来自Comet网站的文物将在课堂上用于教育目的。

User interactive event driven software is pervasive today. End users interact with applications by pointing, clicking or touching the interface, and as this happens, the programs respond. Ensuring the dependability of these systems through software testing is paramount, because insufficient testing techniques currently cost the US economy billions of dollars annually. Yet the flexibility of these systems, which make them appealing to users, also increases the difficulty of testing them. This difficulty has fueled a large body of research on user interactive event driven testing, but the newly developed techniques are often evaluated using isolated case studies and experiments. The user interactive event driven testing community lacks a common set of benchmarks and tools for evaluating their new methods, leading to experimental mismatch; the results of one study are difficult to compare with another. The lack of common benchmarks also means that we cannot easily combine results of multiple studies to build a larger body of knowledge. This project reduces the mismatch and is advancing user interactive event driven testing research by developing a shared research and experimentation web infrastructure called COMET. An initial proof of concept for COMET was developed through earlier support from NSF. Factors that contribute to the mismatch include the development of platform specific test methods, test harnesses that require customizations for each test subject application and each operating system, and models that are incompatible with one another. The research will devise new techniques to control the testing environment, contextualizing factors that may affect experimental outcome and will allow for evolution and change of the artifacts over time. It is building a shared and extensible web infrastructure of benchmarks, tools, models and test artifacts that will enable scientific discovery in the state-of-the-art of user interactive event driven testing. COMET is a public resource that will be available to a broader community. Its impact will extend not only to the user interactive event testing research community, but also to others that work with user interfaces such as those who study usability, and to industry and the software testing community at large. The project work will involve both graduate and undergraduate students. Artifacts from the COMET website will be utilized for educational purposes in the classroom.