CI-New: BugSwarm: A Large-Scale Repository of Replicable Defects, Tests, and Patches to Support the Software Engineering Research Community

点击翻译按钮获取中文摘要

Improving software quality is a central priority of our time.Approaches to understand, diagnose, localize, and fix software defectsare usually empirically grounded in datasets of past defects/repairs.A few such datasets are available; however, these datasets aredifficult to create, and are typically not of the size, scale, anddiversity that is representative of the software in use. The goal ofthis project is to create BugSwarm, a large-scale repository ofreplicable defects, tests, and patches. We propose to draw from therecorded history of defects in open-source projects, to create thisdataset of unprecedented size and diversity, while retainingsufficient fidelity of detail to allow careful study and replicationof these historical defects. BugSwarm will amplify the size ofavailable defect datasets by several orders of magnitude.Continuous integration (CI) development practices, where build andtest processes are carried out in the cloud, with archived results,offer a novel opportunity to construct defect datasets. We propose toexploit CI coding practices to create BugSwarm. Continuous, onlineintegration practices inherently create archived records of build andtest attempts, including those that result in build and/or testfailures, and subsequent repairs. These practices emerged from theimperatives to: a) rigorously and automatically build, integrate, andfully test numerous code submissions from volunteer developers; and b)the need to test these submissions at large volumes, in virtualized,configurable, cloud-based settings. The virtualized, cloud-basedtesting makes these defects much more available and replicable. Wewill exploit these archived records to create DRPs (Defect ReplicationPackages) and DFRPs (Defect & Fix Replication Packages), comprisingbuggy versions of the code, failing regression tests, and bug fixes.DRPs and DFRPs will include complete virtual machines to reproducereal test failures. BugSwarm will facilitate experimentation, andavoid the duplication of a tremendous amount of work among researchersin programming languages, and software engineering.