Stable Prediction of Defect-Inducing Software Changes (SPDISC)

背景：软件系统变得越来越大，越来越复杂。这不可避免地会导致软件缺陷，据估计，软件缺陷的调试每年要花费全球经济3120亿美元。减少软件缺陷的数量是一个具有挑战性的问题，考虑到快速交付的强大压力，这一问题尤为重要。这种压力阻碍了软件源代码的不同部分都接受同样大量的检查和测试工作。考虑到这一点，已经提出了机器学习方法，一旦这些更改完成实现，就可以预测这些更改导致的源代码中的缺陷。这种方法可以使软件工程师将特别的测试和检查注意力集中在最有可能导致缺陷的源代码部分，从而降低提交有缺陷的更改的风险。问题：现有方法的预测性能不稳定，因为正在建模的潜在缺陷生成过程可能会随着时间的推移而变化(即，可能存在概念漂移)。这意味着从业者不能对现有方法的预测能力充满信心--在任何给定的时间点，预测模型可能表现得非常好，或者明显失败。目的和愿景：SPDISC的目标是通过开发一种新的机器学习方法来自动适应概念漂移，创建更稳定的模型来预测缺陷诱发的变化。当与软件版本控制系统集成时，这些模型将在软件项目的整个生命周期中提供早期、可靠和自动化的、导致缺陷的更改警报。影响：SPDISC将使软件开发人员审查和提交他们的更改的方式发生转变。通过创建稳定的模型，使软件开发人员在实现这些更改时立即意识到缺陷引发的更改，它将允许在软件项目的整个生命周期中对缺陷引发的代码进行有针对性的检查和测试。这将降低调试成本，并最终带来更好的软件质量。建议的方法：将开发一种在线学习算法，在输入数据变得可用时对其进行处理，使其能够对概念漂移做出快速反应。将使用设计用于处理类不平衡的方法来检测概念漂移，类不平衡通常发生在预测导致缺陷的软件变更中。类不平衡指的是导致缺陷的更改的数量远远少于安全更改的数量的问题。该方法还将利用来自不同项目的数据(即，域之间的迁移学习)来加快对概念漂移的适应。大多数以前的工作忽略了这样一个事实，即随着时间的推移，预测是必要的，没有注意到预测性能在这个问题上的不稳定性。为了应对不稳定性，SPDISC将开发第一种在线迁移学习方法，用于预测导致缺陷的软件变更。雄心壮志：概念漂移领域之间的在线迁移学习不仅是软件工程中一个非常新的研究领域，而且在机器学习中也是一个非常新的领域。解决这一问题的方法很少，而且没有一种方法能处理阶级不平衡问题。因此，SPDISC不仅将通过实现软件开发人员审查和提交更改的方式的转变来促进软件工程，而且还将促进机器学习本身的领域。及时性：鉴于目前软件系统的规模和复杂性、对生命至关重要的应用程序数量的增加以及软件业的高度竞争力，提高软件质量和降低软件生产和维护成本的方法目前至关重要。

Context: software systems have become ever larger and more complex. This inevitably leads to software defects, whose debugging is estimated to cost the global economy 312 billion USD annually. Reducing the number of software defects is a challenging problem, and is particularly important considering the strong pressure towards rapid delivery. Such pressure impedes different parts of the software source code to all receive equally large amount of inspection and testing effort. With that in mind, machine learning approaches have been proposed for predicting defect-inducing changes in the source code as soon as these changes finish being implemented. Such approaches could enable software engineers to target special testing and inspection attention towards parts of the source code most likely to induce defects, reducing the risk of committing defective changes. Problem: the predictive performance of existing approaches is unstable, because the underlying defect generating process being modelled may vary over time (i.e., there may be concept drift). This means that practitioners cannot be confident about the prediction ability of existing approaches -- at any given point in time, predictive models may be performing very well or failing dramatically.Aim and vision: SPDISC aims at creating more stable models for predicting defect-inducing changes, through the development of a novel machine learning approach for automatically adapting to concept drift. When integrated with software versioning systems, the models will provide early, reliable and automated defect-inducing change alerts throughout the lifetime of software projects. Impact: SPDISC will enable a transformation in the way software developers review and commit their changes. By creating stable models to make software developers aware of defect-inducing changes as soon as these are implemented, it will allow targeted inspection and testing attention towards defect-inducing code throughout the lifetime of software projects. This will reduce the debugging cost and ultimately lead to better software quality. Proposed approach: an online learning algorithm will be developed to process incoming data as they become available, enabling fast reaction to concept drift. Concept drift will be detected using methods designed to cope with class imbalance, which typically occurs in prediction of defect-inducing software changes. Class imbalance refers to the issue of having a much smaller number of defect-inducing changes than the number of safe changes. The proposed approach will also make use of data from different projects (i.e., transfer learning between domains) to speed up adaptation to concept drift.Novelty: SPDISC is the first proposal to look into the stability of predictive performance over time in the context of defect-inducing software changes. Most previous work ignored the fact that predictions are required over time, being oblivious of the instability of predictive performance in this problem. To deal with instability, SPDISC will develop the first online transfer learning approach for predicting defect-inducing software changes. Ambitiousness: online transfer learning between domains with concept drift is not only a very new area of research in software engineering, but also in machine learning. Very few approaches exist for that, and none of them can deal with class-imbalanced problems. Therefore, SPDISC will not only advance software engineering by enabling a transformation in the way software developers review and commit their changes, but also advance the area of machine learning itself. Timeliness: given the current size and complexity of software systems, the increased number of life-critical applications, and the high competitiveness of the software industry, approaches for improving software quality and reducing the cost of producing and maintaining software are currently of utmost importance.