Designing a Peer Evaluation Instrument that is Simple, Reliable, and Valid

一种简单、可靠的合作学习小组同伴评价方法是本项目的重点。该项目解决了一个问题，这是重要的有效利用和更广泛地接受合作学习在工程教育。正在设计和测试的工具将不会提供与更复杂和更难使用的工具相同的反馈水平，但其可靠性和有效性应该与目前可用的最佳工具相当或更高，其简单性将鼓励尚未准备好承诺更雄心勃勃的方法的教师广泛采用。这个项目建立在合作学习的基础上。 一个多项目的仪器与良好的管理有很好的潜力，成为一个成功的仪器。 该项目正在研究同行评价（分数膨胀、相同分数和偏见）和可靠性（评分者人数、管理人数）方面的关键问题。 由于没有真正衡量团队合作质量的标准，该项目正在使用多种评估方法，并寻求它们之间的一致性。通过口头协议分析、行为观察以及与其他经验证的仪器获得的结果的一致性来建立仪器有效性。 重测信度和评分者间信度都在评估中。该项目正在五个参与机构中以分布式方式进行。 这些机构共有9个教师，在11门不同的课程中尝试不同的评估方法。课程和教师的广度承诺使开发的方法稳健。更广泛的影响这同行评价工具，预计将显着改善以团队为基础的工程教育和评估活动。使用它的教师能够通过给予学生形成性的同伴反馈来提高团队技能的教学。使用它的工程项目可以证明他们的学生有团队合作的能力。 通过出版物和专题介绍成功地传播了原型文书，这一点很有希望。该工具的简单性、有效性和可靠性有望加快其在其他机构的采用。来自多个工程教育联盟的学校的参与也可能加速其传播。评估工具及其相关的标题和说明也对大量的学术课程有潜在的兴趣。 正在开发的工具在任何使用合作社的地方都是有用的。 该工具的验证和可靠性评估的广泛测试还应尽量减少任何意外的性别或种族偏见。

Intellectual MeritA simple, reliable cooperative learning group peer evaluation method is the focus of this project. The project addresses an issue that is significant for the effective use of and wider acceptance of cooperative learning in engineering education. The instrument being designed and tested will not provide the same level of feedback as more complex and difficult- to-use instruments, but its reliability and validity should be comparable or higher than those of the best instruments currently available, and its simplicity will encourage widespread adoption by faculty who are not ready to make the commitment to more ambitious approaches. This project builds from what is known about cooperative learning. A multi-item instrument with good administration has good potential for becoming a successful instrument. The project is working on crucial issues in peer evaluation (inflation of scores, identical scores, and bias) and in reliability (number of raters, number of administrations). Because there is no true measure of the quality of teamwork, the project is using multiple assessment methodologies and seeking concurrence among them. Instrument validity is being established through verbal protocol analysis, behavioral observation, and concurrence with results obtained with other validated instruments. Both test-retest reliability and inter-rater reliability are being assessed. The project is being conducted in a distributive fashion in five participating institutions. There are nine faculty across these institutions, experimenting with different methods of assessment in eleven different courses. The breadth of courses and instructors promises to make the developed method robust. Broader impact This peer evaluation instrument is expected to significantly improve both team-based engineering education and assessment activities. Instructors using it are able to improve the teaching of team skills by giving students formative peer feedback. Engineering programs using it can document that their students have the ability to work in teams. The successful dissemination of the prototype instrument through publications and presentations has been promising. The simplicity, validity, and reliability of this instrument can be expected to speed its adoption at other institutions. The involvement of schools from multiple Engineering Education Coalitions is also likely to accelerate its dissemination. The assessment instrument and its associated rubric and instructions are also of potential interest to a large number of academic programs. The instrument under development is useful anywhere cooperative groups are employed. Extensive testing for validation and reliability evaluation for this tool should also minimize any unintended gender or racial bias.