Usability of quality measures for online health information: Can commonly used technical quality criteria be reliably assessed?
Usability of quality measures for online health information: Can commonly used technical quality criteria be reliably assessed?
复制标题
DOI：
10.1016/j.ijmedinf.2005.02.002
复制
发表时间：
2005-08-01
影响因子：
4.9
通讯作者：
Meric-Bernstam, F
中科院分区：
文献类型：
--
作者：
Bernstam, EV;Sagaram, S;Meric-Bernstam, F
目的：已经制定了许多标准来评估在线健康信息的质量。为了有效评估质量，消费者必须使用可以可靠评估的质量标准。然而，很少有工具经过验证以保证评估者之间的一致性。因此，我们评估了两名评估者能够可靠地评估 42 个补充和替代医学网站样本中的 22 条普遍引用的质量标准的程度。方法：我们通过计算一致性百分比、Cohen 的 kappa 以及患病率和偏倚调整的 kappa (PABAK) 来确定评估者间的一致性程度。结果：我们未经校准的分析显示，评估者间对 22 项质量标准中的 8 项的一致性较差。因此，我们为每个标准创建了操作定义，减少了评估选择的数量，并定义了信息的获取位置。结果，22 项质量标准中的 18 项得到了可靠评估（评估者间一致性 >= 0.6）。结论：我们得出的结论是，即使有精确的定义，一些常用的质量标准也无法得到可靠评估。然而，可以通过精确的操作定义来改善评估者之间的一致性。 (C) 2005 Elsevier Ireland Ltd. 保留所有权利。
Purpose: Many criteria have been developed to rate the quality of online health information. To effectively evaluate quality, consumers must use quality criteria that can be reliably assessed. However, few instruments have been validated for inter-rater agreement. Therefore, we assessed the degree to which two raters could reliably assess 22 popularly cited quality criteria on a sample of 42 complementary and alternative medicine Web sites.Methods: We determined the degree of inter-rater agreement by calculating the percentage agreement, Cohen's kappa, and prevalence- and bias-adjusted kappa (PABAK).Results: Our un-calibrated analysis showed poor inter-rater agreement on eight of the 22 quality criteria. Therefore, we created operational definitions for each of the criteria, decreased the number of assessment choices and defined where to took for the information. As a result 18 of the 22 quality criteria were reliably assessed (inter-rater agreement >= 0.6).Conclusions: We conclude that even with precise definitions, some commonly used quality criteria cannot be reliably assessed. However, inter-rater agreement can be improved with precise operational definitions. (C) 2005 Elsevier Ireland Ltd. All rights reserved.