Factors Considered in the Assessment of Computer Science Engineering Capstone Projects and Their Influence on Discrepancies Between Assessors
Factors Considered in the Assessment of Computer Science Engineering Capstone Projects and Their Influence on Discrepancies Between Assessors
复制标题
计算机科学工程顶点项目评估中考虑的因素及其对评估者之间差异的影响
DOI：
10.1145/3381836
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
J. J. Olarte
中科院分区：
文献类型：
--
作者：
C. Domínguez;A. Jaime;F. J. García;J. J. Olarte
一个顶点项目是一个广泛的学习经验，传统上在学生的最后一个学年开发。评估这样一个复杂的任务涉及到几个挑战，通常是基于至少两个不同的人的评估：顶点项目顾问，和一个或多个其他评估员。进行了定量研究，只比较不同评估员的等级，并进行了定性研究，调查可能存在的差异的根源。在这两种情况下，得出了相互矛盾的结论。本研究的目的是分析工程顶点项目评估人员所考虑的因素，以及这些因素对同一项目不同评估人员意见差异的影响。本研究定量研究了162个计算机科学工程顶点项目开发的一个学生和一个顾问的监督。每个项目都由项目顾问和一个委员会进行评估。对于每个项目，顾问和委员会被要求完成一份关于产品特征、学生能力和项目监督的额外问卷。学生表现出的能力被认为是最相关的元素时，顶点项目由顾问和委员会进行评估;产品特性被发现是第二个影响。此外，顾问们认为咨询师参与的部分并不重要。等级之间的差异似乎与一个评估员可以访问的方面有关，而另一个则没有，例如学生在项目开发过程中展示的技能或他们在口头辩护中的表现。顾问和委员会的观点在评估这一复杂任务时都很重要，它们相辅相成。
A capstone project is an extensive learning experience traditionally developed during a student's final academic year. Assessing such a complex assignment involves several challenges and is usually based upon the evaluations of at least two different people: the capstone project advisor, and one or more other assessors. Quantitative studies comparing only different assessors' grades and qualitative studies investigating the origin of possible discrepancies have been conducted. In both cases, contradictory conclusions were reached. The objective of this study is to analyze the factors that are given consideration by assessors of engineering capstone projects and the influence of these factors on the discrepancies between different assessors’ opinions of the same project. This study quantitatively examined 162 computer science engineering capstone projects developed by one student and supervised by one advisor. Each project was assessed by the project advisor and a committee. For each project, the advisor and the committee were asked to complete an additional questionnaire on product characteristics, student competences, and project supervision. Competences demonstrated by the student were found to be the most relevant element when a capstone project was evaluated by the advisor and the committee; product characteristics were found to be second in influence. Furthermore, advisors grant minor significance to the advisor-involvement component. Discrepancies between grades seem to be associated with those aspects to which one assessor has access, while the other does not, such as student skills demonstrated during project development or their performance in the oral defense. Both the advisor's and the committee's perspectives are important in the assessment of this complex task and they complement one another.