CCRI: ENS: Evolving the Defects4J Benchmark and Infrastructure to Enable Sustained Innovation and Reproducibility in Software Engineering Research

CCRI：ENS：发展 Defects4J 基准和基础设施以实现软件工程研究的持续创新和可重复性

• 批准号: 2120070
• 负责人: Rene Just
• 金额: $ 50万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2120070&HistoricalAwards=false
• 关键词: CCRI; ENS; Evolving; Defects4J; Benchmark

Software engineering research aims to increase the quality of the software that pervades modern technology. To make significant advances, reproducibility and comparability of empirical results, using realistic artifacts, are essential in many software engineering research areas, including software testing and debugging, program comprehension, software evolution, and machine learning in software engineering. The Defects4J database of real software faults and a supporting infrastructure were developed to support the software engineering research community, in particular to further reproducibility, to enable faster and better completion of downstream research, and to free researchers from the burden of (re-)developing an experiment infrastructure.This project evolves and extends Defects4J, which has been widely adopted by the software engineering research community. It focuses on enabling sustained innovation and reproducibility in software engineering research by making Defects4J more broadly available and meeting the growing interest and feature requests from the software engineering research community. Specifically, this project aims to (1) grow the current set of artifacts in Defects4J, based on community feedback, (2) develop a fully automated framework for mining additional artifacts, (3) future-proof and containerize all artifacts for use with modern compilers and runtime environments, and (4) provide comprehensive tutorials and templates to support researchers and educators. A high degree of automation and structured pathways to community contributions will support sustained growth. Additionally, this project advances scientific knowledge about defect patterns, prediction, and prevention, applicability and limitations of research techniques, and best practices for empirical evaluations. Finally, this project supports software engineering education at the graduate and undergraduate level. The benchmark and infrastructure allow students to reproduce published results, experiment with, and compare, existing tools and techniques, and quickly evaluate new ideas.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

软件工程研究旨在提高遍布现代技术的软件的质量。为了取得重大进展，使用现实的工件在许多软件工程研究领域中必不可少，包括软件测试和调试，程序理解，软件演变和机器学习在软件工程中至关重要。开发了实际软件故障和支持基础架构的缺陷4J数据库，以支持软件工程研究社区，尤其是为了进一步的可重复性，以使下游研究的更快，更好地完成，以使研究人员免于（重新）开发实验基础架构的负担。它着重于通过使缺陷4J更广泛地获得并满足软件工程研究社区的不断增长的兴趣和功能请求，从而在软件工程研究中持续创新和可重复性。具体而言，该项目的目的是（1）根据社区反馈，（2）开发一个完全自动化的框架，以挖掘额外的工件，（3）将未来的所有工件与现代编译器和运行时环境一起使用，以及（4）提供全面的Tutorials tutorials和Tumplepleters和TempleplateStress和Templepletars和Templepletars和Templeter的教育者。高度的自动化和结构化途径将支持持续的增长。此外，该项目还提高了有关缺陷模式，预测和预防，研究技术的适用性和局限性以及经验评估的最佳实践的科学知识。最后，该项目支持研究生和本科级别的软件工程教育。基准和基础设施使学生能够复制已发表的结果，实验和比较现有工具和技术，并快速评估新想法。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估的评估来评估的。