Using Nonexperts for Annotating Pharmacokinetic Drug-Drug Interaction Mentions in Product Labeling: A Feasibility Study.
Using Nonexperts for Annotating Pharmacokinetic Drug-Drug Interaction Mentions in Product Labeling: A Feasibility Study.
复制标题
DOI：
10.2196/resprot.5028
复制
发表时间：
2016-04-11
影响因子：
1.7
通讯作者：
Boyce RD
中科院分区：
文献类型：
--
作者：
Hochheiser H;Ning Y;Hernandez A;Horn JR;Jacobson R;Boyce RD
由于潜在的药代动力学药物相互作用的重要细节通常在自由文本结构化的产品标签中描述，因此手动管理是开发电子药物相互作用信息资源的必要但昂贵的步骤。使用非专家注释药品标签注释中提及的潜在药物相互作用（PDDI）可能是减轻人工管理负担的一种方法。我们的目标是探索使用非专家参与者从结构化产品标签中注释药物相互作用描述的实用性。通过向药学专家和相对幼稚的参与者提出注释任务，我们希望证明使用非专家注释者进行药物-药物信息注释的可行性。我们也有兴趣探索自然语言处理（NLP）预注释是否以及在多大程度上有助于提高任务完成时间，准确性和主观满意度。要求两名专家和4名非专家在4种条件下按顺序完成注释208个结构化产品标签部分：（1）无NLP辅助，（2）药物提及的预注释，（3）药物提及和PDDI的预注释，以及（4）重复无注释条件。在2组内并相对于现有金标准评价结果。参与者被要求提供完成任务所需的时间和他们对任务难度的看法的报告。一名专家和三名非专家完成了所有任务。来自非专家组的注释结果在每个场景中都相对较强，并且优于NLP管道的性能。专家和2名非专家能够在不到3小时的时间内完成大多数任务。可用性的看法一般是积极的（3.67为专家，平均3.33为非专家）。结果表明，非专家注释可能是一个可行的选择，在更广泛的药品标签的注释PDDI的全面标签。药物提及的预先注释可以简化注释任务。然而，在本研究中操作的PDDI的预先注释给参与者带来了困难。未来的工作应该测试这些问题是否可以通过使用性能更好的NLP和不同的方法在注释工作流期间向用户呈现PDDI预注释来解决。
Because vital details of potential pharmacokinetic drug-drug interactions are often described in free-text structured product labels, manual curation is a necessary but expensive step in the development of electronic drug-drug interaction information resources. The use of nonexperts to annotate potential drug-drug interaction (PDDI) mentions in drug product label annotation may be a means of lessening the burden of manual curation. Our goal was to explore the practicality of using nonexpert participants to annotate drug-drug interaction descriptions from structured product labels. By presenting annotation tasks to both pharmacy experts and relatively naïve participants, we hoped to demonstrate the feasibility of using nonexpert annotators for drug-drug information annotation. We were also interested in exploring whether and to what extent natural language processing (NLP) preannotation helped improve task completion time, accuracy, and subjective satisfaction. Two experts and 4 nonexperts were asked to annotate 208 structured product label sections under 4 conditions completed sequentially: (1) no NLP assistance, (2) preannotation of drug mentions, (3) preannotation of drug mentions and PDDIs, and (4) a repeat of the no-annotation condition. Results were evaluated within the 2 groups and relative to an existing gold standard. Participants were asked to provide reports on the time required to complete tasks and their perceptions of task difficulty. One of the experts and 3 of the nonexperts completed all tasks. Annotation results from the nonexpert group were relatively strong in every scenario and better than the performance of the NLP pipeline. The expert and 2 of the nonexperts were able to complete most tasks in less than 3 hours. Usability perceptions were generally positive (3.67 for expert, mean of 3.33 for nonexperts). The results suggest that nonexpert annotation might be a feasible option for comprehensive labeling of annotated PDDIs across a broader range of drug product labels. Preannotation of drug mentions may ease the annotation task. However, preannotation of PDDIs, as operationalized in this study, presented the participants with difficulties. Future work should test if these issues can be addressed by the use of better performing NLP and a different approach to presenting the PDDI preannotations to users during the annotation workflow.