Comparing Detection Methods For Software Requirements Inspections: A Replication Using Professional Subjects
Comparing Detection Methods For Software Requirements Inspections: A Replication Using Professional Subjects
复制标题
比较软件需求检查的检测方法：使用专业主题的复制
DOI：
10.1023/a:1009776104355
复制
发表时间：
1998
影响因子：
4.1
通讯作者：
L. Votta
中科院分区：
文献类型：
--
作者：
A. Porter;L. Votta
软件需求规范（SRS）通常是手工验证的。一个这样的过程是检查，在这个过程中，几个审阅者独立地分析全部或部分规范并查找缺陷。然后在审稿人和作者的会议上收集这些错误。通常，审查者使用Ad Hoc或Checklist方法来发现错误。这些方法迫使所有评审人员依靠非系统技术来搜索各种各样的错误。我们假设基于场景的方法，其中每个审稿人使用不同的、系统的技术来搜索不同的、特定类别的错误，将具有显著更高的成功率。在之前的工作中，我们以48名计算机科学研究生为研究对象评估了这一假设。现在我们用来自朗讯科技的18名专业开发人员作为实验对象重复了这个实验。我们的目标是：(1)通过研究专业开发人员来扩大我们结果的外部可信度，以及(2)将专业人员的表现与研究生的表现进行比较，以更好地了解成本较低的学生实验的结果有多普遍。对于每个检查，我们执行了四个度量：(1)个人故障检出率，(2)团队故障检出率，(3)在收集会议上首次发现的故障百分比（会议增益率），以及(4)个人首次发现但从未在收集会议上报告的故障百分比（会议损失率）。对于专业人员和学生来说，实验结果是：(1)场景方法比Ad Hoc或Checklist方法具有更高的故障检出率，(2)Checklist评审者并不比Ad Hoc评审者更有效，(3)集合会议没有产生故障的净改进，并且检出率会议的收益被会议损失所抵消。最后，尽管专业人员和学生群体之间的具体措施有所不同，几乎所有统计检验的结果都是相同的。这表明，研究生提供了一个充分的专业人群的模型，与专业人员一起进行研究的更大的费用可能并不总是必要的。
Software requirements specifications (SRS) are often validated manually. One such process is inspection, in which several reviewers independently analyze all or part of the specification and search for faults. These faults are then collected at a meeting of the reviewers and author(s).Usually, reviewers use Ad Hoc or Checklist methods to uncover faults. These methods force all reviewers to rely on nonsystematic techniques to search for a wide variety of faults. We hypothesize that a Scenario-based method, in which each reviewer uses different, systematic techniques to search for different, specific classes of faults, will have a significantly higher success rate.In previous work we evaluated this hypothesis using 48 graduate students in computer science as subjects.We now have replicated this experiment using 18 professional developers from Lucent Technologies as subjects. Our goals were to (1) extend the external credibility of our results by studying professional developers, and to (2) compare the performances of professionals with that of the graduate students to better understand how generalizable the results of the less expensive student experiments were.For each inspection we performed four measurements: (1) individual fault detection rate, (2) team fault detection rate, (3) percentage of faults first identified at the collection meeting (meeting gain rate), and (4) percentage of faults first identified by an individual, but never reported at the collection meeting (meeting loss rate).For both the professionals and the students the experimental results are that (1) the Scenario method had a higher fault detection rate than either Ad Hoc or Checklist methods, (2) Checklist reviewers were no more effective than Ad Hoc reviewers, (3) Collection meetings produced no net improvement in the fault, and detection rate—meeting gains were offset by meeting losses,Finally, although specific measures differed between the professional and student populations, the outcomes of almost all statistical tests were identical. This suggests that the graduate students provided an adequate model of the professional population and that the much greater expense of conducting studies with professionals may not always be required.