The educational effects of portfolios on undergraduate student learning: A Best Evidence Medical Education (BEME) systematic review. BEME Guide No. 11
The educational effects of portfolios on undergraduate student learning: A Best Evidence Medical Education (BEME) systematic review. BEME Guide No. 11
复制标题
DOI：
10.1080/01421590902889897
复制
发表时间：
2009-01-01
期刊：
影响因子：
4.7
通讯作者：
Sayers, Jayne
中科院分区：
文献类型：
--
作者：
Buckley, Sharon;Coleman, Jamie;Sayers, Jayne
近年来，使用档案作为学习和评估工具在卫生专业范围内变得更加普遍。虽然伴随着这些趋势的文献越来越多，但在本科教育中使用档案的教育效果的证据没有明确的整理总结。这篇系统综述是我们工作的结果，提供了这样一个总结。方法：我们根据最佳证据医学教育（BEME）合作的建议制定了一套方案。通过10个数据库的电子检索检索到的引文由两名独立审稿人根据预先定义的纳入/排除标准进行评估，并获得潜在相关文章的全文。通过两名独立审稿人对全文文章的审查，确定了纳入审查的研究。在所有阶段，分歧都以协商一致方式解决。使用试点数据提取表收集有关学生群体特征、干预措施、结果测量、学生设计和结果的数据。每项研究都根据11项质量指标进行评估，这些指标旨在提供有关其设计和实施情况的信息；以及针对教育环境修改的柯克帕特里克等级制度。不同组间比较采用Kruskal-Wallis检验（非参数方差分析）或Mann-Whitney U检验。结果：电子搜索产生了2348条引用。另外23条引文是通过手工检索参考文献列表获得的。大约554篇全文被检索并根据我们的纳入标准进行评估。在纳入我们综述的69项研究中，18项来自医学，32项来自护理，19项来自其他联合卫生专业，包括牙科、物理治疗和放射学。在所有专业小组中，档案主要用于临床环境，完成是强制性的，需要反思和评估（形成性，总结性或两者的结合）是规范。三项研究使用了电子投资组合。虽然许多研究使用了数据收集方法的组合，但所有纳入的研究中有一半以上使用问卷调查，三分之一使用焦点小组访谈，还有三分之一使用直接评估组合。大多数研究评估了学生或导师对使用档案对他们学习的影响的看法。五项研究采用了比较设计，其中一项是随机对照试验。研究最有可能满足与研究对象的适当性、研究问题的清晰度和数据的完整性有关的质量指标。然而，在许多研究中，报告的方法不够详细，无法作出判断。69项纳入的研究中约有19项（27%）达到7项或更多质量指标。在所有职业中，这种“高质量”的研究更有可能是最近发表的。“质量得分”的中位数（达到指标的数量）从2000年或更早发表的研究的2分上升到2005年或更晚发表的研究的7分。在2000年或之前发表的研究与2001年至2004年发表的研究（p = 0.027）、2000年或之前发表的研究与2005年或之后发表的研究（p = 0.002）以及所有研究之间（p = 0.004）均观察到显著差异。类似的趋势也出现在所有专业群体中。在纳入的研究中，约有59项（85%）是在修改后的柯克帕特里克层次的第一级进行评估的（即“参与”效应，包括对学生对保存文件夹对学习的影响的看法的“事后”评估）。约有9项（13%）研究报告直接测量了学生技能或态度的变化，一项研究报告了学生行为的变化。被纳入的研究确定的文件夹使用的主要影响是：提高学生的知识和理解（28项研究，6项在柯克帕特里克水平2或以上），更强的自我意识和鼓励反思（44项研究，7项在柯克帕特里克水平2或以上）和独立学习的能力（10项研究，1项在柯克帕特里克水平2）。高质量研究的结果也确定了这些领域的益处。他们报告说，提高了学生的知识和理解能力，特别是理论与实践相结合的能力，尽管与其他评估成绩的提高之间的相关性并不总是很明显。也注意到更大的自我意识和参与反思，尽管一些研究质疑所进行的反思的质量。高质量的研究还表明，使用作品集可以改善对学生的反馈，使导师更好地了解学生的需求，可以帮助学生应对不确定或情感要求高的情况，并为学生在需要反思实践的研究生环境中做好准备。整理作品集所需的时间投入是确定的主要缺点。在其中的两项研究中，这被发现会减损其他临床学习。结论：目前，在本科背景下，档案的教育效果的证据基础的强度和程度是有限的。然而，有证据表明，报告研究的质量有改善的趋势。“更高质量”的论文认为，知识和理解的提高、自我意识的提高和反思的参与以及学生与导师关系的改善是使用论文组合的主要好处。然而，他们还建议，虽然作品集鼓励学生参与反思，但这些反思的质量不能被假设，并且完成作品集所需的时间承诺可能会减损其他学习或阻止学生参与该过程，除非评估要求这样做。需要进一步的工作来加强组合使用的证据基础，特别是直接观察学生知识和能力变化的比较研究，而不是在组合完成后报告他们的看法。
Introduction: In recent years, the use of portfolios as learning and assessment tools has become more widespread across the range of health professions. Whilst a growing body of literature has accompanied these trends, there is no clear collated summary of the evidence for the educational effects of the use of portfolios in undergraduate education. This systematic review is the result of our work to provide such a summary. Methods: We developed a protocol based on the recommendations of the Best Evidence Medical Education (BEME) collaboration. Citations retrieved by electronic searches of 10 databases were assessed against pre-defined inclusion/exclusion criteria by two independent reviewers and full texts of potentially relevant articles were obtained. Studies were identified for inclusion in the review by examination of full text articles by two independent reviewers. At all stages, discrepancies were resolved by consensus. Data relating to characteristics of the student population, intervention, outcome measures, student design and outcomes were collected using a piloted data extraction form. Each study was assessed against 11 quality indicators designed to provide information about how well it was designed and conducted; and against the Kirkpatrick hierarchy as modified for educational settings. Comparisons between different groups were carried out using the Kruskal-Wallis test (non-parametric ANOVA) or the Mann-Whitney U test as appropriate. Results: Electronic searches yielded 2,348 citations. A further 23 citations were obtained by hand searching of reference lists. About 554 full articles were retrieved and assessed against our inclusion criteria. Of the 69 studies included in our review, 18 were from medicine, 32 from nursing and 19 from other allied health professions, including dentistry, physiotherapy and radiography. In all professional groups, portfolios were used mainly in the clinical setting, completion was compulsory, reflection required and assessment (either formative, summative or a combination of both) the norm. Three studies used electronic portfolios. Whilst many studies used a combination of data collection methods, over half of all included studies used questionnaires, a third used focus group interviews and another third used direct assessment of portfolios. Most studies assessed student or tutor perceptions of the effect of the use of portfolios on their learning. Five studies used a comparative design, one of which was a randomized controlled trial. Studies were most likely to meet the quality indicators relating to appropriateness of study subjects, clarity of research question and completeness of data. However, in many studies, methods were not reported in sufficient detail to allow a judgement to be made. About 19 of the 69 included studies (27%) met seven or more quality indicators. Across all professions, such 'higher quality' studies were more likely to have been published recently. The median 'quality score' (number of indicators met) rose from two for studies published in 2000 or earlier to seven for studies published in 2005 or later. Significant differences were observed between the quality scores for studies published in or before 2000 and those published between 2001 and 2004 (p = 0.027), those published in or before 2000 and those published in 2005 or later (p = 0.002) and between all studies (p = 0.004). Similar trends were seen in all professional groups. About 59 (85%) of the included studies were assessed at level 1 of the modified Kirkpatrick hierarchy (i.e.'participation' effects, incluing 'post hoc' evaluations of student perceptions of the effects of keeping a portfolio on their learning). About 9 (13%) of the studies reported direct measurement of changes in student skills or attitudes and one study reported a change in student behaviour. The main effects of portfolio use identified by the included studies were: Improvement in student knowledge and understanding (28 studies, six at Kirkpatrick level 2 or above), greater self-awareness and encouragement to reflection (44 studies, seven at Kirkpatrick level 2 or above) and the ability to learn independently (10 studies, one at Kirkpatrick level 2). The findings of higher quality studies also identified benefits in these areas. They reported improved student knowledge and understanding, particularly the ability to integrate theory with practice, although a correlation with improved scores in other assessments was not always apparent. Greater self-awareness and engagement in reflection were also noted, although some studies questioned the quality of the reflection undertaken. Higher quality studies also suggest that use of portfolios improves feedback to students and gives tutors a greater awareness of students' needs, may help students to cope with uncertain or emotionally demanding situations and prepares students for postgraduate settings in which reflective practice is required. Time commitment required to collate a portfolio was the major drawback identified. In two of the studies, this was found to detract from other clinical learning. Conclusions: At present, the strength and extent of the evidence base for the educational effects of portfolios in the undergraduate setting is limited. However, there is evidence of an improving trend in the quality of reported studies. 'Higher quality' papers identify improvements in knowledge and understanding, increased self-awareness and engagement in reflection and improved student-tutor relationships as the main benefits of portfolio use. However, they also suggest that whilst portfolios encourage students to engage in reflection, the quality of those reflections cannot be assumed and that the time commitment required for portfolio completion may detract from other learning or deter students from engaging with the process unless required to do so by the demands of assessment. Further work is needed to strengthen the evidence base for portfolio use, particularly comparative studies which observe changes in student knowledge and abilities directly, rather than reporting on their perceptions once a portfolio has been completed.