CCRI: ENS: BugSwarm: Enhancing an Infrastructure and Dataset to Support the Software Engineering Research Community

随着科技的进步，软件在我们的日常生活中扮演着越来越重要的角色。缺陷的突出是软件开发中一个令人烦恼的事实。软件缺陷可能会导致严重的后果，导致企业财务资本的重大损失，甚至是生命的损失。这些事实导致了丰富的研究领域，研究先前发现的缺陷及其修复，以防止或发现未来的缺陷。这些研究领域跨越了许多学科，从安全到软件工程和子领域，如程序分析、验证、软件测试和自动程序修复。对于这些领域来说，重要的是大量现实世界缺陷的研究和评估的可用性。最近，BugSwarm基础设施被开发出来，可以自动创建一个持续增长的可重复的现实世界故障和修复数据集。BugSwarm从使用持续集成服务Travis CI的开源GitHub项目中挖掘成对的故障和修复。BugSwarm的新颖之处在于自动生成脚本来编译和测试代码，并使用Docker映像提供所需的环境来重现每个故障及其相应的修复。BugSwarm基础设施导致了BugSwarm数据集的创建，该数据集目前包含了超过3000个可重复的故障和用Java或Python编写的项目的修复。这项研究将增强BugSwarm基础设施和数据集，以实现研究社区所需的规模和方向的增长，并拥有此类基础设施继续发展所需的长期可持续性。具体来说，该研究旨在解决使数据集更大、更健壮、更多样化和更有用所需的五个挑战：(1)设计新技术以提高从开源项目中挖掘的故障和修复的再现率；(2)结合用户反馈来指导对研究社区感兴趣的故障和修复的挖掘；(3)为数据集提供更丰富的分类模式；(4)构建对JavaScript等其他编程语言的支持；(5)提供工具生态系统以促进BugSwarm数据集的使用。作为这一努力的一部分，研讨会和教程将组织收集来自社区的反馈，并分享BugSwarm资源。展望未来，BugSwarm不仅将促进实验，还将避免软件工程研究人员之间的重复工作。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

As technology advances, software plays an ever-increasing role in our daily lives. The prominence of defects is a troublesome fact of software development. Software defects can have serious consequences leading to significant loss of financial capital for businesses or even loss of life. These facts have led to rich areas of research studying previously found defects and their fixes in order to prevent or find future defects. These areas of research span many disciplines from security to software engineering and sub-fields such as program analysis, verification, software testing, and automated program repair. Important to these fields is the availability of a large number of real-world defects for study and evaluation. Recently, the BugSwarm infrastructure was developed to automatically create a continuously growing dataset of reproducible real-world failures and fixes. BugSwarm mines pairs of failures and fixes from open-source GitHub projects that use the continuous integration service Travis CI. The novelty of BugSwarm lies in the automated generation of scripts to compile and test the code, and the use of Docker images to provide the required environment to reproduce each failure and its corresponding fix. The BugSwarm infrastructure has led to the creation of the BugSwarm dataset, which currently includes over 3,000 reproducible failures and fixes from projects written in Java or Python.This research will enhance the BugSwarm infrastructure and dataset to enable growth at the scale and direction required by the research community, and to possess the long-term sustainability required for such an infrastructure to continue evolving. Specifically, the research aims to solve five challenges necessary to make the dataset larger, more robust, diverse, and useful: (1) devise novel techniques to increase the reproduction rate of failures and fixes mined from open-source projects, (2) incorporate user feedback to guide the mining of failures and fixes of interest to the research community, (3) provide a richer classification schema for the dataset, (4) build support for additional programming languages such as JavaScript, and (5) provide a tool ecosystem to facilitate the use of the BugSwarm dataset. As part of this effort, workshops and tutorials will be organized to gather feedback from the community, and to share the BugSwarm resources. Going forward, BugSwarm will not only facilitate experimentation, but also avoid the duplication of work among software engineering researchers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.