An investigation of the efficacy of collaborative virtual reality systems for moderated remote usability testing
An investigation of the efficacy of collaborative virtual reality systems for moderated remote usability testing
复制标题
DOI：
10.1016/j.apergo.2017.02.011
复制
发表时间：
2017-11-01
影响因子：
3.2
通讯作者：
Greenstein, Joel S.
中科院分区：
文献类型：
--
作者：
Madathil, Kapil Chalil;Greenstein, Joel S.
基于协作虚拟现实的系统将高保真语音通信、沉浸式音频和屏幕共享工具集成到虚拟环境中。这种三维协作虚拟环境可以反映可用性测试参与者和促进者之间的协作，当他们在物理上并在一起时，当促进者和参与者位于不同的地方时，可能会有效地进行适度的可用性测试。我们开发了一个虚拟协作三维远程调节可用性测试实验室，并将其用于一项对照研究，以评估在基于虚拟现实的协作环境中调节可用性测试的有效性，以及其他两种调节可用性测试方法：传统实验室方法和Cisco WebEx（一种基于网络的会议和屏幕共享方法）。采用混合方法实验设计，36名测试参与者和12名测试主持人被要求在一个模拟的网上购物网站上完成代表性任务。因变量包括完成任务所需的时间；识别的可用性缺陷及其严重程度；以及对工作量指数、存在感和满意度问卷的主观评分。使用协作虚拟现实系统的远程调节可用性测试方法，在确定的缺陷总数、确定的高严重性缺陷数量以及使用其他两种方法完成任务所花费的时间方面执行类似。在传统的实验室条件下，测试参与者和引导者所经历的总体工作量最少。在虚拟现实和WebEx条件下所经历的工作量没有显著差异。然而，与WebEx条件相比，测试参与者在虚拟环境中体验到更多的参与和更身临其境的体验。虚拟环境条件的评分与传统实验室条件的评分没有显著差异。这项研究的结果表明，参与者是富有成效的，并且喜欢虚拟实验室的条件，这表明了基于虚拟世界的方法作为同步可用性测试传统方法的替代方法的潜力。(C) 2017 Elsevier Ltd.版权所有。
Collaborative virtual reality-based systems have integrated high fidelity voice-based communication, immersive audio and screen-sharing tools into virtual environments. Such three-dimensional collaborative virtual environments can mirror the collaboration among usability test participants and facilitators when they are physically collocated, potentially enabling moderated usability tests to be conducted effectively when the facilitator and participant are located in different places. We developed a virtual collaborative three-dimensional remote moderated usability testing laboratory and employed it in a controlled study to evaluate the effectiveness of moderated usability testing in a collaborative virtual reality-based environment with two other moderated usability testing methods: the traditional lab approach and Cisco WebEx, a web-based conferencing and screen sharing approach. Using a mixed methods experimental design, 36 test participants and 12 test facilitators were asked to complete representative tasks on a simulated online shopping website. The dependent variables included the time taken to complete the tasks; the usability defects identified and their severity; and the subjective ratings on the workload index, presence and satisfaction questionnaires. Remote moderated usability testing methodology using a collaborative virtual reality system performed similarly in terms of the total number of defects identified, the number of high severity defects identified and the time taken to complete the tasks with the other two methodologies. The overall workload experienced by the test participants and facilitators was the least with the traditional lab condition. No significant differences were identified for the workload experienced with the virtual reality and the WebEx conditions. However, test participants experienced greater involvement and a more immersive experience in the virtual environment than in the WebEx condition. The ratings for the virtual environment condition were not significantly different from those for the traditional lab condition. The results of this study suggest that participants were productive and enjoyed the virtual lab condition, indicating the potential of a virtual world based approach as an alternative to conventional approaches for synchronous usability testing. (C) 2017 Elsevier Ltd. All rights reserved.