USING FAULT CHARACTERISTICS TO IMPROVE SOFTWARE FAULT PREDICTION

重要性：软件代码中的错误是公司的重大成本，也是人类安全和商业成功的风险。查找和修复代码中的错误每年要花费英国软件业数十亿英镑。在将系统交付给用户之前，我们发现故障的能力即使有很小的改进，也可以显著节省成本。背景：我们之前的工作表明，在过去的10年里，208项研究发表了数百种不同的断层预测模型。这些研究通常是由研究人员将许多建模技术中的一种或多种应用于许多可用数据集中的一个或多个，然后应用性能度量来报告该模型预测故障的效果。问题：模型的表现不能一直超过目前80%召回率的预测性能上限。我们认为造成这种性能不佳的一个重要因素是模型将所有的错误都视为同质的。以前没有人尝试了解什么特征使故障可预测，或者模型需要什么特征来预测具有特定特征的故障。目的：建立一种以故障特征为中心，性能始终高于现有性能上限的故障预测模型集。方法：这个为期36个月的项目是基于分析来自六个商业系统和六个开源系统的代码和故障数据。我们将对这些系统中的故障特征进行详细的定量和定性分析，例如确定故障特征是否属于代码接口问题、算法问题、结构问题、排版问题等。我们将构建一组具有多种特征的预测模型（例如，不同的建模技术，不同的自变量等）。我们将使用这些模型来经验地识别故障特征和单个模型特征之间的关系。这意味着我们将确定预测模型的哪些特征可以预测具有特定特征的断层。我们将构建具有覆盖最广泛故障特征的特征的模型集合。我们将与一家公司合作，在工业系统上评估这些模型。

SIGNIFICANCE: Faults in software code are a significant cost to companies, as well as a risk to human safety and business success. Finding and fixing faults in code costs the UK software industry billions of pounds every year. Significant cost savings are available with even small improvements in our capability to find faults before systems are delivered to users. BACKGROUND: Our previous work shows that during the last 10 years, 208 studies have published hundreds of different fault prediction models. These studies are usually typified by researchers applying one or more of the many modeling techniques to one or more of the many available data sets, then applying performance measures to report how well that model predicts faults.PROBLEM: Models do not perform consistently above the current predictive performance ceiling of about 80% recall. We propose that an important contributor to this underperformance is that models treat all faults as homogeneous. No previous attempt has been made to understand what characteristics make a fault predictable or what features a model needs in order to predict faults with particular characteristics. AIM: To build a fault prediction model ensemble which is focused on the characteristics of faults and which consistently performs above the current performance ceiling.METHOD: This 36 month project is based on analysing the code and fault data from six commercial systems and from six open source systems. We will conduct detailed quantitative and qualitative analysis of the characteristics of the faults in these systems, identifying for example whether the characteristics of faults are problems in code interfaces, algorithmic problems, structural problems, typographic problems, etc. We will construct a set of prediction models with a large variety of features (e.g. different modeling techniques, different independent variables, etc.). We will use these models to empirically identify relationships between fault characteristics and the features of individual models. This means that we will identify what features of prediction models predict faults with particular characteristics. We will build ensembles of models with features that cover the widest range of fault characteristics. We will evaluate those models on industrial systems in collaboration with a company.