Stable Prediction of Defect-Inducing Software Changes (SPDISC)

背景：软件系统变得越来越大，越来越复杂。这不可避免地导致软件缺陷，据估计，软件缺陷的调试每年要耗费全球经济3120亿美元。减少软件缺陷的数量是一个具有挑战性的问题，考虑到快速交付的强大压力，这一点尤为重要。这种压力阻碍了软件源代码的不同部分都接受同样大量的检查和测试工作。考虑到这一点，已经提出了机器学习方法来预测源代码中导致缺陷的更改，一旦这些更改完成实现。这样的方法可以使软件工程师将特殊的测试和检查注意力集中在最可能引起缺陷的源代码部分，从而减少提交有缺陷的更改的风险。问题：现有方法的预测性能是不稳定的，因为被建模的潜在缺陷生成过程可能随着时间的推移而变化（例如，可能存在概念漂移）。这意味着从业者不能对现有方法的预测能力充满信心——在任何给定的时间点，预测模型可能表现得非常好，也可能表现得非常糟糕。目标和愿景：SPDISC旨在通过开发一种新的机器学习方法来自动适应概念漂移，为预测缺陷诱导变化创建更稳定的模型。当与软件版本控制系统集成时，这些模型将在软件项目的整个生命周期中提供早期的、可靠的和自动的缺陷诱导变更警报。影响：SPDISC将在软件开发人员审查和提交变更的方式上实现转换。通过创建稳定的模型，使软件开发人员在实现这些变更时就能意识到导致缺陷的变更，它将允许在软件项目的整个生命周期中对导致缺陷的代码进行有针对性的检查和测试。这将减少调试成本，并最终带来更好的软件质量。建议的方法：将开发一种在线学习算法来处理可用的传入数据，从而对概念漂移做出快速反应。概念漂移将使用设计用于处理类不平衡的方法来检测，类不平衡通常发生在预测导致缺陷的软件更改中。类不平衡指的是导致缺陷的更改数量远远少于安全更改的数量。所提出的方法还将利用来自不同项目的数据（即领域之间的迁移学习）来加速对概念漂移的适应。新颖性：SPDISC是第一个在引起缺陷的软件更改的背景下研究预测性能随时间变化的稳定性的建议。大多数先前的工作都忽略了这样一个事实，即随着时间的推移，预测是必需的，忽略了这个问题中预测性能的不稳定性。为了处理不稳定性，SPDISC将开发第一个在线迁移学习方法来预测导致缺陷的软件更改。雄心：概念漂移领域之间的在线迁移学习不仅是软件工程领域的一个非常新的研究领域，也是机器学习领域的一个非常新的研究领域。这方面的方法很少，而且没有一种方法能解决阶级失衡问题。因此，SPDISC不仅将通过软件开发人员审查和提交变更的方式进行转换来推进软件工程，而且还将推进机器学习本身的领域。及时性：考虑到当前软件系统的规模和复杂性，生命关键应用程序数量的增加，以及软件行业的高竞争力，提高软件质量和降低生产和维护软件的成本的方法是当前最重要的。

Context: software systems have become ever larger and more complex. This inevitably leads to software defects, whose debugging is estimated to cost the global economy 312 billion USD annually. Reducing the number of software defects is a challenging problem, and is particularly important considering the strong pressure towards rapid delivery. Such pressure impedes different parts of the software source code to all receive equally large amount of inspection and testing effort. With that in mind, machine learning approaches have been proposed for predicting defect-inducing changes in the source code as soon as these changes finish being implemented. Such approaches could enable software engineers to target special testing and inspection attention towards parts of the source code most likely to induce defects, reducing the risk of committing defective changes. Problem: the predictive performance of existing approaches is unstable, because the underlying defect generating process being modelled may vary over time (i.e., there may be concept drift). This means that practitioners cannot be confident about the prediction ability of existing approaches -- at any given point in time, predictive models may be performing very well or failing dramatically.Aim and vision: SPDISC aims at creating more stable models for predicting defect-inducing changes, through the development of a novel machine learning approach for automatically adapting to concept drift. When integrated with software versioning systems, the models will provide early, reliable and automated defect-inducing change alerts throughout the lifetime of software projects. Impact: SPDISC will enable a transformation in the way software developers review and commit their changes. By creating stable models to make software developers aware of defect-inducing changes as soon as these are implemented, it will allow targeted inspection and testing attention towards defect-inducing code throughout the lifetime of software projects. This will reduce the debugging cost and ultimately lead to better software quality. Proposed approach: an online learning algorithm will be developed to process incoming data as they become available, enabling fast reaction to concept drift. Concept drift will be detected using methods designed to cope with class imbalance, which typically occurs in prediction of defect-inducing software changes. Class imbalance refers to the issue of having a much smaller number of defect-inducing changes than the number of safe changes. The proposed approach will also make use of data from different projects (i.e., transfer learning between domains) to speed up adaptation to concept drift.Novelty: SPDISC is the first proposal to look into the stability of predictive performance over time in the context of defect-inducing software changes. Most previous work ignored the fact that predictions are required over time, being oblivious of the instability of predictive performance in this problem. To deal with instability, SPDISC will develop the first online transfer learning approach for predicting defect-inducing software changes. Ambitiousness: online transfer learning between domains with concept drift is not only a very new area of research in software engineering, but also in machine learning. Very few approaches exist for that, and none of them can deal with class-imbalanced problems. Therefore, SPDISC will not only advance software engineering by enabling a transformation in the way software developers review and commit their changes, but also advance the area of machine learning itself. Timeliness: given the current size and complexity of software systems, the increased number of life-critical applications, and the high competitiveness of the software industry, approaches for improving software quality and reducing the cost of producing and maintaining software are currently of utmost importance.