User evaluation of the effects of a text simplification algorithm using term familiarity on perception, understanding, learning, and information retention.
User evaluation of the effects of a text simplification algorithm using term familiarity on perception, understanding, learning, and information retention.
复制标题
DOI：
10.2196/jmir.2569
复制
发表时间：
2013-07-31
影响因子：
7.4
通讯作者：
Just M
中科院分区：
文献类型：
--
作者：
Leroy G;Endicott JE;Kauchak D;Mouradi O;Just M
适当的卫生知识对于人们保持良好的健康和管理疾病和伤害非常重要。教育文本，无论是从互联网上检索或由医生的办公室提供，是一种流行的方法来传达健康相关的信息。不幸的是，很难写出易于理解的文本，并且现有的方法，主要是可读性公式的应用，还没有令人信服地证明可以降低文本的难度。开发一个基于证据的写作支持工具，以改善感知和实际的文本难度。为此，我们正在开发和测试算法，自动识别文本中的困难部分，并提供适当的，更容易的替代方案;有效降低文本难度的算法将包括在支持工具中。这项工作描述了一个独立的作家使用术语熟悉度的自动化简化算法的用户评价。术语熟悉度表示读者对单词的熟悉程度，并使用Google Web Corpus中的术语频率进行估计。不熟悉的单词被算法识别并标记为潜在的替换。由同义词、上位词、定义和语义类型组成的替代词从WordNet、统一医学语言系统（UMLS）和维基词典中提取，并排名供作者选择以简化文本。我们进行了一项对照用户研究，其中一位代表性作家使用我们的简化算法来简化文本。我们测试了具有代表性的消费者的影响。我们的研究的关键自变量是词汇简化，我们测量了它对感知和实际文本难度的影响。参与者是从亚马逊的土耳其机器人网站上招募的。感知难度用1个度量标准（5分制李克特量表）测量。实际难度用3个指标来衡量：5个多项选择题与每个文本一起测量理解，7个多项选择题没有文本用于学习，2个自由回忆题用于信息保留。99名参与者完成了这项研究。我们发现了强烈的有益效果，感知和实际的困难。简化后，文本被认为更简单（P<0.001），简化文本在5分制李克特量表上得分为2.3，原始文本为3.2（得分1：最简单）。这也导致了对文本的更好理解（P<.001），与原始文本（52%正确）相比，简化文本（63%正确）的正确答案增加了11%。在阅读简化文本后，正确答案增加了18%，而在阅读原文后，正确答案增加了9%（P= 0.003）。对自由回忆没有显著影响。术语熟悉性是简化文本的一个有价值的特征。虽然文本的主题影响效应大小，但结果令人信服且一致。
Adequate health literacy is important for people to maintain good health and manage diseases and injuries. Educational text, either retrieved from the Internet or provided by a doctor’s office, is a popular method to communicate health-related information. Unfortunately, it is difficult to write text that is easy to understand, and existing approaches, mostly the application of readability formulas, have not convincingly been shown to reduce the difficulty of text. To develop an evidence-based writer support tool to improve perceived and actual text difficulty. To this end, we are developing and testing algorithms that automatically identify difficult sections in text and provide appropriate, easier alternatives; algorithms that effectively reduce text difficulty will be included in the support tool. This work describes the user evaluation with an independent writer of an automated simplification algorithm using term familiarity. Term familiarity indicates how easy words are for readers and is estimated using term frequencies in the Google Web Corpus. Unfamiliar words are algorithmically identified and tagged for potential replacement. Easier alternatives consisting of synonyms, hypernyms, definitions, and semantic types are extracted from WordNet, the Unified Medical Language System (UMLS), and Wiktionary and ranked for a writer to choose from to simplify the text. We conducted a controlled user study with a representative writer who used our simplification algorithm to simplify texts. We tested the impact with representative consumers. The key independent variable of our study is lexical simplification, and we measured its effect on both perceived and actual text difficulty. Participants were recruited from Amazon’s Mechanical Turk website. Perceived difficulty was measured with 1 metric, a 5-point Likert scale. Actual difficulty was measured with 3 metrics: 5 multiple-choice questions alongside each text to measure understanding, 7 multiple-choice questions without the text for learning, and 2 free recall questions for information retention. Ninety-nine participants completed the study. We found strong beneficial effects on both perceived and actual difficulty. After simplification, the text was perceived as simpler (P<.001) with simplified text scoring 2.3 and original text 3.2 on the 5-point Likert scale (score 1: easiest). It also led to better understanding of the text (P<.001) with 11% more correct answers with simplified text (63% correct) compared to the original (52% correct). There was more learning with 18% more correct answers after reading simplified text compared to 9% more correct answers after reading the original text (P=.003). There was no significant effect on free recall. Term familiarity is a valuable feature in simplifying text. Although the topic of the text influences the effect size, the results were convincing and consistent.
登录
查看更多内容
影响因子：
5
作者：
COHEN, S;KAMARCK, T;MERMELSTEIN, R
通讯作者：
MERMELSTEIN, R
影响因子：
2.5
作者：
Landi, Nicole
通讯作者：
Landi, Nicole
DOI：
10.1197/jamia.m2599
发表时间：
2008-07-01
影响因子：
6.4
作者：
Keselman, Alla;Smith, Catherine Arnott;Zeng-Treitler, Qing
通讯作者：
Zeng-Treitler, Qing
影响因子：
4.3
作者：
Bailey, Marc A.;Coughlin, Patrick A.;Scott, D. Julian A.
通讯作者：
Scott, D. Julian A.
影响因子：
18.4
作者：
Ahmed, Osman H.;Sullivan, S. John;McCrory, Paul R.
通讯作者：
McCrory, Paul R.