Reliability of Health Information on the Internet: An Examination of Experts' Ratings
Reliability of Health Information on the Internet: An Examination of Experts' Ratings
复制标题
DOI：
10.2196/jmir.4.1.e2
复制
发表时间：
2002-01-01
影响因子：
7.4
通讯作者：
Muncer, Steven
中科院分区：
文献类型：
--
作者：
Craigie, Mark;Loader, Brian;Muncer, Steven
背景：近年来，利用医学专家对互联网上与健康有关的网站的内容进行评级的做法越来越多。在这项研究中，它一直是常见的做法，使用一个单一的医学专家的网站的内容进行评级。在许多情况下，专家将互联网上的健康信息评为差，甚至是潜在的危险。然而，这种方法的一个问题是，不能保证其他医学专家将以类似的方式对网站进行评级。Objectives：目的是评估医学专家对互联网新闻组中与常见疾病相关的主题的判断的可靠性。一个次要目的是显示的局限性，常用的统计测量的可靠性（例如，Kappa）。方法：在这项研究中的参与者是5名医生，谁在一个专门的单位工作，致力于治疗的疾病。他们每个人都使用专家自己设计的6分制对新闻组线程中包含的信息进行评分。他们的评级进行了分析，使用一些统计数据的可靠性：科恩的卡帕，伽马，肯德尔的W，和克朗巴赫的alpha.Results：可靠性是不存在的问题的评级，和低的评级的反应。所用的各种可靠性措施给出了相互矛盾的结果。没有措施产生高reliability.Conclusions：医学专家表现出较低的协议时，从新闻组的帖子评级。因此，在评估互联网上与健康有关的信息的准确性和质量的研究中，测试评分者之间的可靠性是很重要的。对可以使用的不同一致性衡量标准的讨论表明，统计数据的选择可能存在问题。因此，在使用可靠性度量之前，必须考虑其背后的假设。通常，"三角测量"需要使用多个度量。
Background: The use of medical experts in rating the content of health-related sites on the Internet has flourished in recent years. In this research, it has been common practice to use a single medical expert to rate the content of the Web sites. In many cases, the expert has rated the Internet health information as poor, and even potentially dangerous. However, one problem with this approach is that there is no guarantee that other medical experts will rate the sites in a similar manner.Objectives: The aim was to assess the reliability of medical experts' judgments of threads in an Internet newsgroup related to a common disease. A secondary aim was to show the limitations of commonly-used statistics for measuring reliability (eg, kappa).Method: The participants in this study were 5 medical doctors, who worked in a specialist unit dedicated to the treatment of the disease. They each rated the information contained in newsgroup threads using a 6-point scale designed by the experts themselves. Their ratings were analyzed for reliability using a number of statistics: Cohen's kappa, gamma, Kendall's W, and Cronbach's alpha.Results: Reliability was absent for ratings of questions, and low for ratings of responses. The various measures of reliability used gave conflicting results. No measure produced high reliability.Conclusions: The medical experts showed a low agreement when rating the postings from the newsgroup. Hence, it is important to test inter-rater reliability in research assessing the accuracy and quality of health-related information on the Internet. A discussion of the different measures of agreement that could be used reveals that the choice of statistic can be problematic. It is therefore important to consider the assumptions underlying a measure of reliability before using it. Often, more than one measure will be needed for "triangulation" purposes.