Evaluating the effects of machine pre-annotation and an interactive annotation interface on manual de-identification of clinical text.
Evaluating the effects of machine pre-annotation and an interactive annotation interface on manual de-identification of clinical text.
复制标题
DOI：
10.1016/j.jbi.2014.05.002
复制
发表时间：
2014-08
影响因子：
4.5
通讯作者：
Chapman WW
中科院分区：
文献类型：
--
作者：
South BR;Mowery D;Suo Y;Leng J;Ferrández Ó;Meystre SM;Chapman WW
健康保险流通与责任法案 (HIPAA) 安全港方法要求从临床文件中删除 18 种受保护的健康信息 (PHI)，以便在用于研究目的之前将其视为“去识别化”。对大量临床文档中的 PHI 元素进行人工审查可能非常乏味且容易出错。事实上，可能需要多个注释者来一致地编辑代表每个 PHI 类的信息。自动去识别化有可能提高注释质量并减少注释时间。例如，通过结合用作预注释的去识别系统输出和交互式注释界面来使用机器辅助注释，为注释者提供用于“管理”的 PHI 注释，而不是在原始临床文档上从“从头开始”进行手动注释。为了评估机器辅助标注是否提高了参考标准质量的可靠性和准确性并减少了标注工作量，我们进行了标注实验。在这项注释研究中，我们评估了 VA 医疗信息学研究联盟 (CHIR) 注释模式和指南应用于名为 MTSamples 的公开临床文档语料库的普遍性。具体来说，我们的目标是 (1) 表征手动注释风险排名 PHI 和其他注释类型（临床名字和人际关系）的异质临床文档语料库，(2) 评估注释者将 CHIR 模式应用于异质语料库的效果，(3) 比较机器辅助注释（实验）与手动注释（对照）相比是否提高了注释质量并减少了注释时间，以及 (4) 评估每个添加的参考标准覆盖范围质量的变化注释者的注释。
The Health Insurance Portability and Accountability Act (HIPAA) Safe Harbor method requires removal of 18 types of protected health information (PHI) from clinical documents to be considered “de-identified” prior to use for research purposes. Human review of PHI elements from a large corpus of clinical documents can be tedious and error-prone. Indeed, multiple annotators may be required to consistently redact information that represents each PHI class. Automated de-identification has the potential to improve annotation quality and reduce annotation time. For instance, using machine-assisted annotation by combining de-identification system outputs used as pre-annotations and an interactive annotation interface to provide annotators with PHI annotations for “curation” rather than manual annotation from “scratch” on raw clinical documents. In order to assess whether machine-assisted annotation improves the reliability and accuracy of the reference standard quality and reduces annotation effort, we conducted an annotation experiment. In this annotation study, we assessed the generalizability of the VA Consortium for Healthcare Informatics Research (CHIR) annotation schema and guidelines applied to a corpus of publicly available clinical documents called MTSamples. Specifically, our goals were to (1) characterize a heterogeneous corpus of clinical documents manually annotated for risk-ranked PHI and other annotation types (clinical eponyms and person relations), (2) evaluate how well annotators apply the CHIR schema to the heterogeneous corpus, (3) compare whether machine-assisted annotation (experiment) improves annotation quality and reduces annotation time compared to manual annotation (control), and (4) assess the change in quality of reference standard coverage with each added annotator’s annotations.
登录
查看更多内容
影响因子：
4
作者：
Meystre SM;Friedlin FJ;South BR;Shen S;Samore MH
通讯作者：
Samore MH
影响因子：
3.5
作者：
Neamatullah, Ishna;Douglass, Margaret M.;Clifford, Gari D.
通讯作者：
Clifford, Gari D.
影响因子：
5.8
作者：
Settles, B
通讯作者：
Settles, B
DOI：
10.1197/jamia.m1733
发表时间：
2005-05-01
影响因子：
6.4
作者：
Hripcsak, G;Rothschild, AS
通讯作者：
Rothschild, AS
影响因子：
4
作者：
Ferrández O;South BR;Shen S;Friedlin FJ;Samore MH;Meystre SM
通讯作者：
Meystre SM