CI-New: BugSwarm: A Large-Scale Repository of Replicable Defects, Tests, and Patches to Support the Software Engineering Research Community

提高软件质量是我们这个时代的中心任务。了解、诊断、定位和修复软件缺陷的方法通常基于过去缺陷/修复的数据集。有几个这样的数据集可用；然而，这些数据集很难创建，并且通常没有代表正在使用的软件的大小、规模和多样性。该项目的目标是创建BUGsarm，这是一个可复制的缺陷、测试和补丁的大型存储库。我们建议从开源项目中记录的缺陷历史中提取，以创建前所未有的大小和多样性的数据集，同时保持足够的细节保真度，以便仔细研究和复制这些历史缺陷。持续集成(CI)开发实践，即构建和测试过程在云中执行，并将结果存档，这为构建缺陷数据集提供了一个新的机会。我们建议利用CI编码实践来创建BUG群。持续的在线集成实践本身就会创建构建和测试尝试的存档记录，包括那些导致构建和/或测试失败以及后续修复的尝试。这些做法产生于以下需求：a)严格和自动地构建、集成和全面测试来自志愿开发人员的大量代码提交；以及b)需要在虚拟的、可配置的、基于云的环境中大量测试这些提交。基于云的虚拟化测试使这些缺陷更加可用和可复制。我们将利用这些归档记录来创建DRP(缺陷复制包)和DFRP(缺陷和修复复制包)，包括错误版本的代码、失败的回归测试和错误修复。DRP和DFRP将包括完整的虚拟机来再现真实的测试失败。BUG群将促进实验，并避免编程语言和软件工程方面的研究人员之间大量工作的重复。

Improving software quality is a central priority of our time.Approaches to understand, diagnose, localize, and fix software defectsare usually empirically grounded in datasets of past defects/repairs.A few such datasets are available; however, these datasets aredifficult to create, and are typically not of the size, scale, anddiversity that is representative of the software in use. The goal ofthis project is to create BugSwarm, a large-scale repository ofreplicable defects, tests, and patches. We propose to draw from therecorded history of defects in open-source projects, to create thisdataset of unprecedented size and diversity, while retainingsufficient fidelity of detail to allow careful study and replicationof these historical defects. BugSwarm will amplify the size ofavailable defect datasets by several orders of magnitude.Continuous integration (CI) development practices, where build andtest processes are carried out in the cloud, with archived results,offer a novel opportunity to construct defect datasets. We propose toexploit CI coding practices to create BugSwarm. Continuous, onlineintegration practices inherently create archived records of build andtest attempts, including those that result in build and/or testfailures, and subsequent repairs. These practices emerged from theimperatives to: a) rigorously and automatically build, integrate, andfully test numerous code submissions from volunteer developers; and b)the need to test these submissions at large volumes, in virtualized,configurable, cloud-based settings. The virtualized, cloud-basedtesting makes these defects much more available and replicable. Wewill exploit these archived records to create DRPs (Defect ReplicationPackages) and DFRPs (Defect & Fix Replication Packages), comprisingbuggy versions of the code, failing regression tests, and bug fixes.DRPs and DFRPs will include complete virtual machines to reproducereal test failures. BugSwarm will facilitate experimentation, andavoid the duplication of a tremendous amount of work among researchersin programming languages, and software engineering.