A comparative evaluation of two user feedback techniques for requirements trace retrieval
A comparative evaluation of two user feedback techniques for requirements trace retrieval
复制标题
DOI：
10.1145/2245276.2231943
复制
发表时间：
2012-03
期刊：
影响因子：
--
通讯作者：
Yonghee Shin;J. Cleland-Huang
中科院分区：
文献类型：
--
作者：
Yonghee Shin;J. Cleland-Huang
在自动化要求中，可以通过合并用户反馈来实现重大改进。在本文中，我们介绍了一种名为Direct查询操作（DQM）的相对较新的技术，并将其对Rocchio的有效性进行比较，Rocchio是将用户反馈集成到自动跟踪方法中的当前DefaTSO标准。这两种技术通过一系列模拟和用户研究进行经验评估，该技术是通过追踪Worldvista的需求来进行的，WorldVista是一种电子医疗信息系统，可根据医疗保健信息技术认证委员会的要求。我们的结果表明，与矢量空间模型相比，Rocchio和DQM均返回痕量质量的显着改善，这是一种全自动技术。在痕量质量方面，DQM的表现略好，人类努力的差异最小。混合方法对DQM和Rocchio的各个方法都提供了进一步的改进。
In automated requirements trace retrieval, significant improvements can be realized through incorporating user feedback. In this paper we introduce a relatively new technique named Direct Query Manipulation (DQM) and compare its effectiveness against Rocchio, the current defacto standard for integrating user feedback into automated tracing methods. The two techniques are evaluated empirically through a series of simulations and a user study, conducted by tracing requirements for WorldVista, an electronic healthcare information system against requirements from the Certification Commission for Healthcare Information Technology. Our results show that both Rocchio and DQM return significant improvements in trace quality in comparison to the vector space model, a fully automated technique. DQM performs slightly better than Rocchio in terms of trace quality with minimal difference in human effort. The hybrid approach provides further improvement over both individual approaches of DQM and Rocchio.