Using cloud-based mobile technology for assessment of competencies among medical students.
Using cloud-based mobile technology for assessment of competencies among medical students.
复制标题
作者：
Ferenchick GS;Solomon D
临床环境下对医学生能力的有效、直接观察仍然具有挑战性，并限制了促进以成绩为基础的学生进步的机会。直接观察的基本原理是确定学生已经获得了护理病人所需的核心临床能力。学生的观察往往会导致高度多变的评估，这些评估受到学生实际表现以外的因素的歪曲。有效的直接观察和评估的障碍包括缺乏有效的工具和战略，以确保在真实的临床环境中使用透明的标准来判断临床能力。我们开发了一个基于Web的内容管理系统，名为Just In Time Medicine(JIT)，以解决其中的许多问题。JIT的目标有四个：第一，创建一个自助服务界面，使具备一般计算技能的教师可以编写可在包括移动设备在内的支持互联网的设备上显示的可定制内容和基于标准的评估工具；第二，创建一个能够捕获学员与数百项临床技能相关的进展的评估和反馈工具；第三，使教员能够在真实的临床环境中轻松访问和利用这些工具进行学习者评估，以实现及时的师资发展；第四，创建受训人员观察到的技能的永久记录，这些技能对学员和项目评估均有用。从2010年7月到2012年10月，我们在367名内科三年级学生中实施了JIT启动的临床评估练习(CEX)。观察者(主治医生和住院医生)使用JIT进行CEX评估，以指导和记录他们的观察，记录他们观察的时间并向学生提供反馈，以及他们的总体满意度。17名观察者观看了6个学生与患者的会面录像，并通过测量学生CEX分数与他们在随后标准化的患者欧安组织考试中的分数之间的相关性来评估评分者之间的可靠性和有效性。516名观察员共完成了3567次CEX。每个学生的平均评估次数为9.7次(±1.8 SD)，每个观察者平均完成的CEX次数为6.9次(±15.8 SD)。观察者在43-50%的CEX上花费的时间不到10分钟，68.6%的人在反馈会议上花费不到10分钟。大多数观察员(92%)对CEX表示满意。在观看录像带的所有观察员中，评分员之间的可信度为0.69，这些评级充分区分了合格和不合格的表现。测量的CEX成绩与随后的学生在欧安组织年终考试中的表现相关。我们的结论是，JIT的使用在获取离散的临床表现数据方面是可行的，并且具有高度的用户满意度。我们的嵌入式核对表具有足够的评分者之间的可靠性以及同时和预测的有效性。
Valid, direct observation of medical student competency in clinical settings remains challenging and limits the opportunity to promote performance-based student advancement. The rationale for direct observation is to ascertain that students have acquired the core clinical competencies needed to care for patients. Too often student observation results in highly variable evaluations which are skewed by factors other than the student’s actual performance. Among the barriers to effective direct observation and assessment include the lack of effective tools and strategies for assuring that transparent standards are used for judging clinical competency in authentic clinical settings. We developed a web-based content management system under the name, Just in Time Medicine (JIT), to address many of these issues. The goals of JIT were fourfold: First, to create a self-service interface allowing faculty with average computing skills to author customizable content and criterion-based assessment tools displayable on internet enabled devices, including mobile devices; second, to create an assessment and feedback tool capable of capturing learner progress related to hundreds of clinical skills; third, to enable easy access and utilization of these tools by faculty for learner assessment in authentic clinical settings as a means of just in time faculty development; fourth, to create a permanent record of the trainees’ observed skills useful for both learner and program evaluation. From July 2010 through October 2012, we implemented a JIT enabled clinical evaluation exercise (CEX) among 367 third year internal medicine students. Observers (attending physicians and residents) performed CEX assessments using JIT to guide and document their observations, record their time observing and providing feedback to the students, and their overall satisfaction. Inter-rater reliability and validity were assessed with 17 observers who viewed six videotaped student-patient encounters and by measuring the correlation between student CEX scores and their scores on subsequent standardized-patient OSCE exams. A total of 3567 CEXs were completed by 516 observers. The average number of evaluations per student was 9.7 (±1.8 SD) and the average number of CEXs completed per observer was 6.9 (±15.8 SD). Observers spent less than 10 min on 43–50% of the CEXs and 68.6% on feedback sessions. A majority of observers (92%) reported satisfaction with the CEX. Inter-rater reliability was measured at 0.69 among all observers viewing the videotapes and these ratings adequately discriminated competent from non-competent performance. The measured CEX grades correlated with subsequent student performance on an end-of-year OSCE. We conclude that the use of JIT is feasible in capturing discrete clinical performance data with a high degree of user satisfaction. Our embedded checklists had adequate inter-rater reliability and concurrent and predictive validity.
登录
查看更多内容
影响因子：
4.6
作者：
Ferenchick, Gary S.;Foreback, Jami;Mohmand, Asad
通讯作者：
Mohmand, Asad
影响因子：
2.5
作者：
Ferenchick, Gary;Fetters, Moses;Carse, A. Mervyn
通讯作者：
Carse, A. Mervyn
影响因子：
39.2
作者：
Holmboe, ES;Hawkins, RE;Huot, SJ
通讯作者：
Huot, SJ
影响因子：
7.4
作者：
Hasnain, M;Connell, KJ;Yudkowsky, R
通讯作者：
Yudkowsky, R
影响因子：
4
作者：
Solomon, David J
通讯作者：
Solomon, David J