A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models
A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models
复制标题
评估类比和回归模型估计益处的对照实验
DOI：
10.1109/32.799947
复制
发表时间：
1999
期刊：
影响因子：
--
通讯作者：
E. Stensrud
中科院分区：
文献类型：
--
作者：
I. Myrtveit;E. Stensrud
要具有普遍的有效性，经验结果必须收敛。为了可信，实验科学必须了解其局限性，并能够解释经验结果的不一致之处。我们描述了一个复制先前研究的实验，这些研究声称类比估计优于回归模型。在实验中，68名经验丰富的从业者从48个工业COTS项目的数据集中估计了一个项目。我们采用了类比工具和回归模型两种处理方法，并以历史数据辅助下的估计绩效作为对照。我们发现我们的结果与以前的结果并不收敛。原因是以前的研究使用了其他数据集和部分不同的数据分析方法，最后但同样重要的是，这些工具是在与工具用户隔离的情况下进行验证的。这意味着结果对实验设计是敏感的：数据集的特征，从原始数据集中去除离群值和其他数据点的规范，测试指标，显著水平，以及人类受试者的使用和他们的专业水平。因此，我们的结果和以前的结果都不够稳健，不足以声称任何普遍的有效性。
To have general validity, empirical results must converge. To be credible, an experimental science must understand the limitations and be able to explain the disagreements of empirical results. We describe an experiment to replicate previous studies which claim that estimation by analogy outperforms regression models. In the experiment, 68 experienced practitioners each estimated a project from a dataset of 48 industrial COTS projects. We applied two treatments, an analogy tool and a regression model, and we used the estimating performance when aided by the historical data as the control. We found that our results do not converge with previous results. The reason is that previous studies have used other datasets and partially different data analysis methods, and last but not least, the tools have been validated in isolation from the tool users. This implies that the results are sensitive to the experimental design: the characteristics of the dataset, the norms for removing outliers and other data points from the original dataset, the test metrics, significance levels, and the use of human subjects and their level of expertise. Thus, neither our results nor previous results are robust enough to claim any general validity.