Testing the face validity and inter-rater agreement of a simple approach to drug-drug interaction evidence assessment.
Testing the face validity and inter-rater agreement of a simple approach to drug-drug interaction evidence assessment.
复制标题
DOI：
10.1016/j.jbi.2019.103355
复制
发表时间：
2020-01
影响因子：
4.5
通讯作者：
Boyce RD
中科院分区：
文献类型：
--
作者：
Grizzle AJ;Hines LE;Malone DC;Kravchenko O;Hochheiser H;Boyce RD
药物相互作用（DDI）知识库之间的一致性低是一个有据可查的问题。不一致的一个潜在原因是药物专家在评估潜在DDI证据的方法上存在差异。在这项研究中，我们检查了一个新的DDI证据评估工具，设计简单，易于使用的表面效度和评分者间的信度。招募了具有评估DDI证据的专业经验的参与者的便利样本。参与者使用新仪器独立评估了5种药物对的预选证据项。对于每个药物对，参与者根据仪器提供的证据类别将每个证据项标记为足以或不足以确定DDI的存在。参与者还决定是否整体证据支持涉及药物对的DDI。在证据项和药物对水平上计算一致性。选择≥ 70%的临界值作为一致性百分比的一致性阈值，而使用> 0.6的系数作为机会校正一致性的临界值。开放式的意见收集和编码，以确定主题相关的参与者的经验，使用新的方法。新工具的表面有效性通过两轮评估确定，共涉及6名专家。15名专家同意参加可靠性评估，14名专家完成了研究。参与者对34个证据项目中的22个（65%）的充分性的一致性没有超过先验一致性阈值。同样，5种药物对中有3种（60%）的证据充分性一致性较差。在药物对水平的机会校正一致性进一步证实了该工具的评价者间可靠性较差（Gwet的AC 1 = 0.24，Conger的Kappa =0.24）。参与者评论提出了几个可能的分歧原因，包括评估证据项目类型和研究设计时未解决的主观性，将证据评价与临床相关性考虑分离不可行，以及与DDI病例报告评价相关的潜在问题。尽管关键发现是负面的，但该研究的结果揭示了专家如何处理DDI证据评估，包括在考虑临床相关性的背景下定位证据评估的重要性。在负面调查结果的背景下，参与者的意见分析确定了几个有前途的未来研究方向，包括：新的基于计算机的证据评估支持;一个更全面的证据评估方法，需要考虑具体的，明确的，临床后果的正式评价;和DDI病例报告评估工具的更正式的调查。
Low concordance between drug-drug interaction (DDI) knowledge bases is a well-documented concern. One potential cause of inconsistency is variability between drug experts in approach to assessing evidence about potential DDIs. In this study, we examined the face validity and inter-rater reliability of a novel DDI evidence evaluation instrument designed to be simple and easy to use. A convenience sample of participants with professional experience evaluating DDI evidence was recruited. Participants independently evaluated pre-selected evidence items for 5 drug pairs using the new instrument. For each drug pair, participants labeled each evidence item as sufficient or insufficient to establish the existence of a DDI based on the evidence categories provided by the instrument. Participants also decided if the overall body of evidence supported a DDI involving the drug pair. Agreement was computed both at the evidence item and drug pair levels. A cut-off of ≥ 70% was chosen as the agreement threshold for percent agreement, while a coefficient > 0.6 was used as the cut-off for chance-corrected agreement. Open ended comments were collected and coded to identify themes related to the participants’ experience using the novel approach. The face validity of the new instrument was established by two rounds of evaluation involving a total of 6 experts. Fifteen experts agreed to participate in the reliability assessment, and 14 completed the study. Participant agreement on the sufficiency of 22 of the 34 evidence items (65%) did not exceed the a priori agreement threshold. Similarly, agreement on the sufficiency of evidence for 3 of the 5 drug pairs (60%) was poor. Chance-corrected agreement at the drug pair level further confirmed the poor interrater reliability of the instrument (Gwet’s AC1 = 0.24, Conger’s Kappa =0.24). Participant comments suggested several possible reasons for the disagreements including unaddressed subjectivity in assessing an evidence item’s type and study design, an infeasible separation of evidence evaluation from the consideration of clinical relevance, and potential issues related to the evaluation of DDI case reports. Even though the key findings were negative, the study’s results shed light on how experts approach DDI evidence assessment, including the importance situating evidence assessment within the context of consideration of clinical relevance. Analysis of participant comments within the context of the negative findings identified several promising future research directions including: novel computer-based support for evidence assessment; formal evaluation of a more comprehensive evidence assessment approach that requires consideration of specific, explicitly stated, clinical consequences; and more formal investigation of DDI case report assessment instruments.
登录
查看更多内容
影响因子：
3.5
作者：
Romagnoli KM;Nelson SD;Hines L;Empey P;Boyce RD;Hochheiser H
通讯作者：
Hochheiser H
影响因子：
4.2
作者：
Scheife RT;Hines LE;Boyce RD;Chung SP;Momper JD;Sommer CD;Abernethy DR;Horn JR;Sklar SJ;Wong SK;Jones G;Brown ML;Grizzle AJ;Comes S;Wilkins TL;Borst C;Wittie MA;Malone DC
通讯作者：
Malone DC
影响因子：
3.7
作者：
Seden K;Gibbons S;Marzolini C;Schapiro JM;Burger DM;Back DJ;Khoo SH
通讯作者：
Khoo SH
影响因子：
4.2
作者：
van Roon, EN;Flikweert, S;Brouwers, JRBJ
通讯作者：
Brouwers, JRBJ
为临床决策支持选择药物相互作用的建议。
DOI：
10.2146/ajhp150565
发表时间：
2016-04-15
期刊：
American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists
影响因子：
--
作者：
Tilson H;Hines LE;McEvoy G;Weinstein DM;Hansten PD;Matuszewski K;le Comte M;Higby-Baker S;Hanlon JT;Pezzullo L;Vieson K;Helwig AL;Huang SM;Perre A;Bates DW;Poikonen J;Wittie MA;Grizzle AJ;Brown M;Malone DC
通讯作者：
Malone DC