Stable Prediction of Defect-Inducing Software Changes (SPDISC)

导致缺陷的软件变更的稳定预测 (SPDISC)

• 批准号: EP/R006660/1
• 负责人: Leandro Minku
• 金额: $ 12.81万
• 依托单位: University of Leicester
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2018
• 资助国家: 英国
• 起止时间: 2018 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FR006660%2F1
• 关键词: Stable; Prediction; Defect; Inducing; Software

Context: software systems have become ever larger and more complex. This inevitably leads to software defects, whose debugging is estimated to cost the global economy 312 billion USD annually. Reducing the number of software defects is a challenging problem, and is particularly important considering the strong pressure towards rapid delivery. Such pressure impedes different parts of the software source code to all receive equally large amount of inspection and testing effort. With that in mind, machine learning approaches have been proposed for predicting defect-inducing changes in the source code as soon as these changes finish being implemented. Such approaches could enable software engineers to target special testing and inspection attention towards parts of the source code most likely to induce defects, reducing the risk of committing defective changes. Problem: the predictive performance of existing approaches is unstable, because the underlying defect generating process being modelled may vary over time (i.e., there may be concept drift). This means that practitioners cannot be confident about the prediction ability of existing approaches -- at any given point in time, predictive models may be performing very well or failing dramatically.Aim and vision: SPDISC aims at creating more stable models for predicting defect-inducing changes, through the development of a novel machine learning approach for automatically adapting to concept drift. When integrated with software versioning systems, the models will provide early, reliable and automated defect-inducing change alerts throughout the lifetime of software projects. Impact: SPDISC will enable a transformation in the way software developers review and commit their changes. By creating stable models to make software developers aware of defect-inducing changes as soon as these are implemented, it will allow targeted inspection and testing attention towards defect-inducing code throughout the lifetime of software projects. This will reduce the debugging cost and ultimately lead to better software quality. Proposed approach: an online learning algorithm will be developed to process incoming data as they become available, enabling fast reaction to concept drift. Concept drift will be detected using methods designed to cope with class imbalance, which typically occurs in prediction of defect-inducing software changes. Class imbalance refers to the issue of having a much smaller number of defect-inducing changes than the number of safe changes. The proposed approach will also make use of data from different projects (i.e., transfer learning between domains) to speed up adaptation to concept drift.Novelty: SPDISC is the first proposal to look into the stability of predictive performance over time in the context of defect-inducing software changes. Most previous work ignored the fact that predictions are required over time, being oblivious of the instability of predictive performance in this problem. To deal with instability, SPDISC will develop the first online transfer learning approach for predicting defect-inducing software changes. Ambitiousness: online transfer learning between domains with concept drift is not only a very new area of research in software engineering, but also in machine learning. Very few approaches exist for that, and none of them can deal with class-imbalanced problems. Therefore, SPDISC will not only advance software engineering by enabling a transformation in the way software developers review and commit their changes, but also advance the area of machine learning itself. Timeliness: given the current size and complexity of software systems, the increased number of life-critical applications, and the high competitiveness of the software industry, approaches for improving software quality and reducing the cost of producing and maintaining software are currently of utmost importance.

背景：软件系统变得越来越大，越来越复杂。这不可避免地导致软件缺陷，其调试估计每年花费全球经济3120亿美元。减少软件缺陷的数量是一个具有挑战性的问题，考虑到快速交付的强大压力，这一点尤为重要。这种压力阻碍了软件源代码的不同部分都接受同样大量的检查和测试工作。考虑到这一点，机器学习方法已经被提出来预测源代码中的缺陷诱导更改，一旦这些更改完成实现。这种方法可以使软件工程师能够针对最有可能导致缺陷的源代码部分进行特殊的测试和检查，从而降低提交有缺陷的更改的风险。问题：现有方法的预测性能是不稳定的，因为被建模的潜在缺陷产生过程可能随时间变化（即，可能存在概念漂移）。这意味着从业者无法对现有方法的预测能力充满信心--在任何给定的时间点，预测模型都可能表现得非常好，也可能会出现严重的失败。目标和愿景：SPDISC旨在通过开发一种新的机器学习方法来自动适应概念漂移，从而创建更稳定的模型来预测导致缺陷的变化。当与软件版本控制系统集成时，这些模型将在软件项目的整个生命周期内提供早期、可靠和自动化的缺陷诱导变更警报。影响：SPDISC将使软件开发人员审查和提交更改的方式发生转变。通过创建稳定的模型，使软件开发人员在实施后立即意识到导致缺陷的更改，它将允许在软件项目的整个生命周期中对导致缺陷的代码进行有针对性的检查和测试。这将降低调试成本，并最终导致更好的软件质量。拟议办法：将开发一种在线学习算法，以便在收到数据时对其进行处理，从而能够对概念漂移作出快速反应。概念漂移将使用设计用于科普类不平衡的方法来检测，这通常发生在预测缺陷诱导软件更改时。类不平衡指的是导致缺陷的更改的数量比安全更改的数量少得多。拟议的方法还将利用来自不同项目的数据（即，新奇：SPDISC是第一个在缺陷诱导软件变更的背景下研究预测性能随时间变化的稳定性的提案。大多数以前的工作忽略了一个事实，即预测需要随着时间的推移，被遗忘的预测性能在这个问题中的不稳定性。为了应对不稳定性，SPDISC将开发第一个在线迁移学习方法来预测导致缺陷的软件更改。野心：在具有概念漂移的领域之间的在线迁移学习不仅是软件工程中的一个非常新的研究领域，而且在机器学习中也是如此。很少有方法可以解决这个问题，而且没有一种方法可以处理类不平衡的问题。因此，SPDISC不仅将通过改变软件开发人员审查和提交更改的方式来推进软件工程，还将推进机器学习本身的领域。时间：考虑到软件系统目前的规模和复杂性、对生命至关重要的应用程序数量的增加以及软件行业的高度竞争性，用于提高软件质量和降低生产和维护软件的成本的方法目前至关重要。

项目成果

Software Effort Interval Prediction via Bayesian Inference and Synthetic Bootstrap Resampling

• DOI: 10.1145/3295700
• 发表时间: 2019-01
• 期刊: ACM Transactions on Software Engineering and Methodology (TOSEM)
• 作者: Liyan Song;Leandro L. Minku;X. Yao

A novel online supervised hyperparameter tuning procedure applied to cross-company software effort estimation

• DOI: 10.1007/s10664-019-09686-w
• 发表时间: 2019-02
• 期刊: Empirical Software Engineering
• 影响因子: 4.1
• 作者: Leandro L. Minku

Multi-Source Transfer Learning for Non-Stationary Environments

• DOI: 10.1109/ijcnn.2019.8852024
• 发表时间: 2019-01
• 期刊: 2019 International Joint Conference on Neural Networks (IJCNN)
• 作者: Honghui Du;Leandro L. Minku;Huiyu Zhou

GMM-VRD: A Gaussian Mixture Model for Dealing With Virtual and Real Concept Drifts

GMM-VRD：处理虚实概念漂移的高斯混合模型

• DOI: 10.1109/ijcnn.2019.8852097
• 发表时间: 2019
• 作者: Oliveira G

Data-Driven Search-Based Software Engineering

• DOI: 10.1145/3196398.3196442
• 发表时间: 2018-01
• 期刊: 2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR)
• 作者: V. Nair;Amritanshu Agrawal;Jianfeng Chen;Wei Fu;George Mathew;T. Menzies;Leandro L. Minku;Markus W