Determination of the effectiveness of two methods for usability evaluation using a CPOE medication ordering system
Determination of the effectiveness of two methods for usability evaluation using a CPOE medication ordering system
复制标题
DOI：
10.1016/j.ijmedinf.2011.02.005
复制
发表时间：
2011-05-01
影响因子：
4.9
通讯作者：
Jaspers, M. W. M.
中科院分区：
文献类型：
--
作者：
Khajouei, R.;Hasman, A.;Jaspers, M. W. M.
目的：为了评估两种可用性评估方法，认知走查（CW）和大声思考（TA），识别可用性问题，并比较CW和TA在识别不同类型的可用性problems.Methods的性能的有效性：CW进行了两个可用性评估和10名医生被招募来执行一个CPOE系统（Medicator）的TA可用性测试。确定已识别可用性问题的严重性，并根据用户操作框架（UAF）对可用性问题进行分类。还确定了可用性问题导致用药错误的可能性。这两种方法的彻底性，有效性和有效性进行了比较。结果：57个不同严重程度的独特的可用性问题，分布在四个阶段的相互作用所定义的UAF，被确定。TA方法识别可用性问题的有效性比CW高0.08（0.70 vs. 0.62）。TA的彻底性（方法可以识别现有可用性问题的程度）在“规划”和“评估”阶段较高，在“翻译”阶段较低（由UAF定义）。TA识别可能导致用药错误的问题的彻底性高于CW（0.81 vs. 0.68）。通过每种方法识别的可用性问题数量显著少于在Medicator中检测到的真实的可用性问题总数（p < 0.001）。观察到的CW和TA识别的真实的可用性问题数量之间的差异（38 vs. 41），CW和TA检测到的问题的平均严重性之间的差异（2.37 vs. 2.41），以及识别可能导致用药错误的问题的差异（15 vs. 18）无统计学显著性（p > 0.4）。结论：这项研究表明，虽然TA显示出略好的效果，在识别的可用性问题的数量和这些问题的平均严重性方面，CW方法和TA方法的性能没有显著差异问题由于没有一种单一的评估方法可以发现所有的可用性问题，因此建议将多种方法结合起来作为最合适的方法，特别是如果可用性问题可能导致潜在的致命结果。(C)2011爱思唯尔爱尔兰有限公司保留所有权利。
Objectives: To assess the effectiveness of two usability evaluation methods, cognitive walkthrough (CW) and think aloud (TA), for identifying usability problems and to compare the performance of CW and TA in identifying different types of usability problems.Methods: A CW was performed by two usability evaluators and 10 physicians were recruited to perform a TA usability testing of a CPOE system (Medicator). The severity of identified usability problems was determined and the usability problems were categorized based on the User Action Framework (UAF). The potential of usability problems to cause medication errors was also determined. The thoroughness, validity and effectiveness of the two methods were compared.Results: Fifty seven unique usability problems of different severity, spread over the four phases of interaction as defined by the UAF, were identified. The effectiveness of the TA method for identifying usability problems was 0.08 higher than that of the CW (0.70 vs. 0.62). The thoroughness (the extent to which a method can identify existing usability problems) of the TA was higher for the "Planning" and "Assessment" phases and lower for the "Translation" phase (as defined by UAF). The thoroughness of TA for identifying problems that may potentially result in medication errors was higher than that of CW (0.81 vs. 0.68). The number of usability problems identified by each of the methods was significantly less than the total number of detected real usability problems in Medicator (p < 0.001). The observed differences between the number of real usability problems identified by CW and TA (38 vs. 41), the difference between the average severity of the detected problems by CW and TA (2.37 vs. 2.41), and the difference for identifying problems potentially resulting in medication errors (15 vs. 18) were not statistically significant (p > 0.4).Conclusions: This study shows that although TA showed a slightly better effectiveness, there is no significant difference between the performance of the CW and the TA methods in terms of number of usability problems identified and the mean severity of these problems. Since no single evaluation method will uncover all of the usability problems a combination of methods is advised as the most appropriate approach, especially if usability problems can lead to potentially fatal outcomes. (C) 2011 Elsevier Ireland Ltd. All rights reserved.