A Systematic Approach to Configuring MetaMap for Optimal Performance.
A Systematic Approach to Configuring MetaMap for Optimal Performance.
复制标题
配置元模型的系统方法，以实现最佳性能。
DOI：
10.1055/a-1862-0421
复制
发表时间：
2022-12
影响因子：
1.7
通讯作者：
中科院分区：
文献类型：
--
作者：
背景MetaMap是处理生物医学文本以识别概念的宝贵工具。 虽然MetaMap是高度可配置的，但配置决策并不简单。 目的开发一种系统的、数据驱动的方法来配置MetaMap以获得最佳性能。 方法采用MetaMap、word 2 vec模型和短语模型构建流水线。 对于无监督训练，短语和word 2 vec模型使用与临床决策支持相关的摘要作为输入。在测试过程中，MetaMap配置了默认选项、一个行为选项和两个行为选项。对于每种配置，确定的实体和黄金标准的条款之间的余弦和软余弦相似性得分计算40注释摘要（422句）。相似性评分用于计算和比较每种配置的摘要中精确匹配、相似匹配和缺失金标准术语的总体百分比。手动抽查结果。计算精确度、召回率和F-测量（β =1）。 结果一个行为选项的完全匹配率和金标准项缺失率分别为0.6-0.79和0.09-0.3;两个行为选项的完全匹配率和金标准项缺失率分别为0.56-0.8和0.09-0.3。 软余弦相似性分数的精确匹配和缺失项的百分比超过余弦相似性分数。精确匹配的平均精确度、召回率和F-测度分别为0.59、0.82和0.68，缺失项的平均精确度、召回率和F-测度分别为1.00、0.53和0.69。 结论我们展示了一种系统的方法，提供了客观和准确的证据，指导MetaMap配置优化性能。 在MetaMap配置中，将客观证据与使用原则、经验和直觉的当前实践相结合，优于单一策略。我们的方法、参考代码、测量、结果和工作流程是优化和配置MetaMap的宝贵参考。
Background MetaMap is a valuable tool for processing biomedical texts to identify concepts. Although MetaMap is highly configurative, configuration decisions are not straightforward. Objective To develop a systematic, data-driven methodology for configuring MetaMap for optimal performance. Methods MetaMap, the word2vec model, and the phrase model were used to build a pipeline. For unsupervised training, the phrase and word2vec models used abstracts related to clinical decision support as input. During testing, MetaMap was configured with the default option, one behavior option, and two behavior options. For each configuration, cosine and soft cosine similarity scores between identified entities and gold-standard terms were computed for 40 annotated abstracts (422 sentences). The similarity scores were used to calculate and compare the overall percentages of exact matches, similar matches, and missing gold-standard terms among the abstracts for each configuration. The results were manually spot-checked. The precision, recall, and F-measure ( β =1) were calculated. Results The percentages of exact matches and missing gold-standard terms were 0.6–0.79 and 0.09–0.3 for one behavior option, and 0.56–0.8 and 0.09–0.3 for two behavior options, respectively. The percentages of exact matches and missing terms for soft cosine similarity scores exceeded those for cosine similarity scores. The average precision, recall, and F-measure were 0.59, 0.82, and 0.68 for exact matches, and 1.00, 0.53, and 0.69 for missing terms, respectively. Conclusion We demonstrated a systematic approach that provides objective and accurate evidence guiding MetaMap configurations for optimizing performance. Combining objective evidence and the current practice of using principles, experience, and intuitions outperforms a single strategy in MetaMap configurations. Our methodology, reference codes, measurements, results, and workflow are valuable references for optimizing and configuring MetaMap.
登录
查看更多内容
影响因子：
3
作者：
Tsai RT;Wu SH;Chou WC;Lin YC;He D;Hsiang J;Sung TY;Hsu WL
通讯作者：
Hsu WL
影响因子：
14.4
作者：
Ferrucci, David;Levas, Anthony;Mueller, Erik T.
通讯作者：
Mueller, Erik T.
影响因子：
3.5
作者：
Peng J;Zhao M;Havrilla J;Liu C;Weng C;Guthrie W;Schultz R;Wang K;Zhou Y
通讯作者：
Zhou Y
DOI：
10.1093/jamia/ocw177
发表时间：
2017-07-01
影响因子：
6.4
作者：
Demner-Fushman, Dina;Rogers, Willie J.;Aronson, Alan R.
通讯作者：
Aronson, Alan R.
影响因子：
14.9
作者：
Bodenreider, O
通讯作者：
Bodenreider, O