Combining GitHub, Chat, and Peer Evaluation Data to Assess Individual Contributions to Team Software Development Projects
Combining GitHub, Chat, and Peer Evaluation Data to Assess Individual Contributions to Team Software Development Projects
复制标题
结合 GitHub、聊天和同行评估数据来评估个人对团队软件开发项目的贡献
DOI：
10.1145/3593592
复制
发表时间：
2023
影响因子：
2.4
通讯作者：
Tariq, Ahsun
中科院分区：
文献类型：
--
作者：
Hundhausen, Christopher;Conrad, Phill;Adesope, Olusola;Tariq, Ahsun
评估团队软件开发项目是出了名的困难，并且通常基于主观度量。为了帮助评估更加严格，我们进行了一项实证研究，以探索基于同行和教师评估的主观指标与基于GitHub和聊天数据的客观指标之间的关系。我们研究了23个本科软件团队（n= 117名学生），从两个本科计算课程在两个北美研究型大学。我们收集了团队的数据：（a）来自GitHub代码库的提交和问题，（B）来自Slack和Microsoft Teams频道的聊天消息，（c）来自CATME同行评估系统的同行评估评级，以及（d）来自课程的个人作业评分。我们从（a）和（B）中推导出度量标准来衡量团队成员对团队的贡献以及团队成员贡献的平等性。然后，我们进行皮尔森分析，以确定指标之间的相关性，同行评价评级，和个人成绩。我们发现团队成员的GitHub贡献、聊天贡献和同行评估评级之间存在显著的正相关性。此外，团队的GitHub贡献的平等性与团队的平均同行评价评分呈正相关，与这些评分的方差呈负相关。然而，在团队聊天贡献的平等性和他们的同行评价评级之间没有检测到这种正相关。我们的研究扩展了以前的研究结果，提供了证据表明：（a）团队成员的聊天贡献，就像他们的GitHub贡献一样，与他们的同行评价评级呈正相关;（B）团队成员的聊天贡献与他们的GitHub贡献呈正相关;（c）团队GitHub贡献的平等性与他们的同行评价评级呈正相关。这些结果进一步支持了这样一种观点，即结合客观和主观度量可以使团队软件项目的评估更加全面和严格。
Assessing team software development projects is notoriously difficult and typically based on subjective metrics. To help make assessments more rigorous, we conducted an empirical study to explore relationships between subjective metrics based on peer and instructor assessments, and objective metrics based on GitHub and chat data. We studied 23 undergraduate software teams (n= 117 students) from two undergraduate computing courses at two North American research universities. We collected data on teams’ (a) commits and issues from their GitHub code repositories, (b) chat messages from their Slack and Microsoft Teams channels, (c) peer evaluation ratings from the CATME peer evaluation system, and (d) individual assignment grades from the courses. We derived metrics from (a) and (b) to measure both individual team members’contributionsto the team, and theequalityof team members’ contributions. We then performed Pearson analyses to identify correlations among the metrics, peer evaluation ratings, and individual grades. We found significant positive correlations between team members’ GitHub contributions, chat contributions, and peer evaluation ratings. In addition, the equality of teams’ GitHub contributions was positively correlated with teams’ average peer evaluation ratings and negatively correlated with the variance in those ratings. However, no such positive correlations were detected between the equality of teams’ chat contributions and their peer evaluation ratings. Our study extends previous research results by providing evidence that (a) team members’ chat contributions, like their GitHub contributions, are positively correlated with their peer evaluation ratings; (b) team members’ chat contributions are positively correlated with their GitHub contributions; and (c) the equality of team’ GitHub contributions is positively correlated with their peer evaluation ratings. These results lend further support to the idea that combining objective and subjective metrics can make the assessment of team software projects more comprehensive and rigorous.
登录
查看更多内容
评估个人对软件工程项目的贡献：复制研究
影响因子：
2.7
作者：
Hundhausen, C.D.;Conrad, P.T.;Carter, A.S.;Adesope, O.
通讯作者：
Adesope, O.
与工业客户合作的软件工程项目课程
DOI：
10.1145/2732155
发表时间：
2015
期刊：
ACM Trans. Comput. Educ.
影响因子：
--
作者：
B. Brügge;Stephan Krusche;Lukas Alperowitz
通讯作者：
Lukas Alperowitz
计算机科学工程顶点项目评估中考虑的因素及其对评估者之间差异的影响
DOI：
10.1145/3381836
发表时间：
2020
期刊：
ACM Transactions on Computing Education (TOCE)
影响因子：
--
作者：
C. Domínguez;A. Jaime;F. J. García;J. J. Olarte
通讯作者：
J. J. Olarte
通过服务学习项目和开源方法丰富软件工程课程
DOI：
10.1145/1062455.1062566
发表时间：
2005
期刊：
Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005.
影响因子：
--
作者：
Chang Liu
通讯作者：
Chang Liu
影响因子：
1.7
作者：
J. Defranco;P. Laplante
通讯作者：
P. Laplante