Evaluating current automatic de-identification methods with Veteran's health administration clinical documents.
Evaluating current automatic de-identification methods with Veteran's health administration clinical documents.
复制标题
DOI：
10.1186/1471-2288-12-109
复制
发表时间：
2012-07-27
影响因子：
4
通讯作者：
Meystre SM
中科院分区：
文献类型：
--
作者：
Ferrández O;South BR;Shen S;Friedlin FJ;Samore MH;Meystre SM
电子健康记录（EHR）的使用和采用的增加导致对临床医生，研究人员和许多其他操作目的有用的数字信息的巨大增长。然而，这些信息中含有大量受保护的健康信息，这严重限制了其访问和可能的使用。许多研究人员已经开发出通过删除PHI来自动去识别EHR文档的方法，如健康保险便携性和责任法案“安全港”方法中所规定的。本研究的重点是评估现有的自动文本去识别方法和工具，适用于退伍军人健康管理局（VHA）的临床文件，以评估哪些方法对我们的临床笔记中发现的每一类PHI表现更好;以及何时需要新的方法来提高性能。我们使用VHA临床文档语料库安装并评估了五个文本去识别系统“开箱即用”。基于机器学习方法的系统使用2006年i2 b2去识别语料库进行训练，并使用我们的VHA语料库进行评估，还使用我们的VHA语料库进行了十倍交叉验证实验。我们统计了与参考注释的精确、部分和完全包含的匹配，分别考虑每个PHI类型，或仅考虑一个唯一的“PHI”类别。使用召回率（相当于灵敏度）和精确度（相当于阳性预测值）指标以及F2测量评估系统的性能。总的来说，基于规则和模式匹配的系统实现了更好的召回率，并且基于机器学习方法的系统的精确度总是更好。对于部分匹配，最高的“开箱即用”F2测量值为67%;最好的精确度和召回率分别为95%和78%。最后，10倍交叉验证实验允许F2测量增加到79%，具有部分匹配。文本去识别系统的“开箱即用”评估为我们提供了关于VHA临床文档去识别的最佳方法的令人信服的见解。错误分析表明，需要定制特定于VHA文档的PHI格式。该研究为VHA临床文本的“同类最佳”自动去识别应用程序的规划和开发提供了信息。
The increased use and adoption of Electronic Health Records (EHR) causes a tremendous growth in digital information useful for clinicians, researchers and many other operational purposes. However, this information is rich in Protected Health Information (PHI), which severely restricts its access and possible uses. A number of investigators have developed methods for automatically de-identifying EHR documents by removing PHI, as specified in the Health Insurance Portability and Accountability Act “Safe Harbor” method. This study focuses on the evaluation of existing automated text de-identification methods and tools, as applied to Veterans Health Administration (VHA) clinical documents, to assess which methods perform better with each category of PHI found in our clinical notes; and when new methods are needed to improve performance. We installed and evaluated five text de-identification systems “out-of-the-box” using a corpus of VHA clinical documents. The systems based on machine learning methods were trained with the 2006 i2b2 de-identification corpora and evaluated with our VHA corpus, and also evaluated with a ten-fold cross-validation experiment using our VHA corpus. We counted exact, partial, and fully contained matches with reference annotations, considering each PHI type separately, or only one unique ‘PHI’ category. Performance of the systems was assessed using recall (equivalent to sensitivity) and precision (equivalent to positive predictive value) metrics, as well as the F2-measure. Overall, systems based on rules and pattern matching achieved better recall, and precision was always better with systems based on machine learning approaches. The highest “out-of-the-box” F2-measure was 67% for partial matches; the best precision and recall were 95% and 78%, respectively. Finally, the ten-fold cross validation experiment allowed for an increase of the F2-measure to 79% with partial matches. The “out-of-the-box” evaluation of text de-identification systems provided us with compelling insight about the best methods for de-identification of VHA clinical documents. The errors analysis demonstrated an important need for customization to PHI formats specific to VHA documents. This study informed the planning and development of a “best-of-breed” automatic de-identification application for VHA clinical text.
登录
查看更多内容
影响因子：
4
作者：
Meystre SM;Friedlin FJ;South BR;Shen S;Samore MH
通讯作者：
Samore MH
影响因子：
3.5
作者：
Neamatullah, Ishna;Douglass, Margaret M.;Clifford, Gari D.
通讯作者：
Clifford, Gari D.
DOI：
10.1197/jamia.m2444
发表时间：
2007-09-01
影响因子：
6.4
作者：
Uzuner, Oezlem;Luo, Yuan;Szolovits, Peter
通讯作者：
Szolovits, Peter
影响因子：
3.5
作者：
Gupta, D;Saul, M;Gilbertson, J
通讯作者：
Gilbertson, J
影响因子：
7.5
作者：
Uzuner, Oezlem;Sibanda, Tawanda C.;Szovits, Peter
通讯作者：
Szovits, Peter