Automated summative usability studies: an empirical evaluation
Automated summative usability studies: an empirical evaluation
复制标题
DOI：
10.1145/1124772.1124867
复制
发表时间：
2006-04
期刊：
影响因子：
--
通讯作者：
R. West;K. Lehman
中科院分区：
文献类型：
--
作者：
R. West;K. Lehman
本文评估了一种使用自动数据收集系统进行总结性可用性测试的方法。我们发现自动总结性测试是一种简单而有效的替代基于实验室的总结性测试，并且可以成功地远程执行。在我们的研究中，一个基于网络的控制窗口引导参与者完成总结性研究，提供要执行的任务，并询问有关用户体验的后续问题。使用组内比较，我们发现可用性工程师收集的数据和通过性能度量的自动测试系统收集的数据之间没有主要差异。通过组间比较，我们发现自动化总结性研究可以远程进行，与基于实验室的测试相比，在完成任务的时间和放弃任务的可能性方面存在微小但可接受的差异。任务成功和任务满意度评分在远程和实验室总结性测试之间没有差异。参与者通过测试系统提供的书面评论足以确定导致任务失败的主要可用性问题，但并没有像观察会议的可用性工程师那样全面地揭示一系列问题。
This paper evaluates a method for summative usability testing using an automated data collection system. We found automated summative testing to be a simple and effective alternative to lab-based summative testing and could be successfully conducted remotely. In our study, a web-based control window led participants through the summative study, provided tasks to perform, and asked follow up questions about the user experience. Using a within-group comparison, we found no major differences between data collected by a usability engineer and that collected through an automated testing system for performance metrics. Using a between-group comparison, we found automated summative studies could be conducted remotely with minor but acceptable differences in time on task and likelihood to give up on a task compared to lab-based testing. Task success and task satisfaction ratings were not different between remote and lab-based summative testing. Written comments provided by participants through the testing system were sufficient to identify the major usability problems that led to task failure but did not reveal as comprehensive a set of issues as did a usability engineer observing the sessions.