Automated competency measurement and feedback generation for object-oriented programming

基于到目前为止的项目过程，已经有可能通过对学生编程代码的教学和实证分析来识别有能力的候选人，并以这种方式将编程代码的自动分析和评估作为编程任务的解决方案与面向对象编程的循证定义和能力测量联系起来。这为创建相应的评价项目并将其应用于教学奠定了科学的基础。然而，这些项目的成功使用需要对迄今为止发现的能力候选人进行实证验证，特别是对这些项目的效果进行调查，并在此基础上自动生成以能力为导向的反馈。此外，能力候选者的推导绝不能被认为是到目前为止的结果所完成的，因此，进一步的能力候选者可以并且应该使用到目前为止在项目期间成功建立的方法和定义的能力测量相关项目来推导。因此，该项目追求教学和教育技术的目标。在教学部分，进一步的能力候选人的推导以及发现的能力候选人的实证验证是在前景，其次是自动反馈和提交的解决方案的质量之间的相互联系的调查。这些项目的使用将产生广泛的数据，从而可以得出关于知识差距或误解的结论。对有用反馈的分析，反过来，将允许得出关于触发反馈的技能缺陷的性质和结构的结论，以及关于项目的质量和功能。由于项目持续数年，对即将到来的学期和现有数据集的比较将不仅关注短期影响，还关注跨学期的影响，以便能够观察由于几年使用相同任务而产生的磨损效应。在目标的教育技术部分，重点是具体主题的技术程序的进一步发展，这涉及对大量方案代码的教学分析的支持和对具体的、面向能力的反馈的自动产生。其目的是使教学部分所需的学生课程代码的定性分析自动化，从而使识别知识差距、误解和能力缺陷的机制成为JACK使用的电子评估系统的一个组成部分，从而使在项目学期结束后，特别是在讲座的日常操作中，能够进一步发展以能力为导向的反馈。

Based on the course of the project so far, it has been possible to identify competence candidates through the didactic and empirical analysis of student programming code and in this way to link the automated analysis and evaluation of programming code as a solution to programming tasks with the evidence-based definition and measurement of competences for object-oriented programming. This lays the scientific foundation for creating corresponding assessment items and using them in teaching. Nevertheless, the successful use of these items requires the empirical validation of the competence candidates found so far and, in particular, the investigation of the effect of these items and the automatically generated, competence-oriented feedback based on them. In addition, the derivation of competence candidates can by no means be regarded as completed with the results so far, so that further competence candidates can and should be derived with the method successfully established during the project so far and associated items for competence measurement defined. The project thus pursues both didactic and educational technology goals. In the didactic part, the derivation of further competence candidates as well as the empirical validation of the found competence candidates are in the foreground, followed by the investigation of the mutual connection between automated feedback and the quality of the submitted solutions. The use of the items will generate extensive data that will allow conclusions to be drawn about knowledge gaps or misconceptions. The analysis of helpful feedback, in turn, will allow conclusions to be drawn about the nature and structure of the skills deficits that triggered the feedback, as well as about the quality and functioning of the items. Due to the project's duration of several years, a comparison of upcoming semesters and existing data sets will not only focus on short-term effects, but also on effects across semesters, in order to be able to observe wear-and-tear effects due to the use of identical tasks over several years. In the educational technology part of the objective, the focus is on the subject-specific further development of technical procedures, which concern both the support for the didactic analysis of large amounts of programme code and the automatic generation of specific, competence-oriented feedback. The aim is to automate the qualitative analysis of student programme code necessary for the didactic part to such an extent that mechanisms for recognising knowledge gaps, misconceptions and competence deficits can become an integral part of the e-assessment system JACK used and thus enable the further development of competence-oriented feedback beyond the end of the project term, especially in the everyday operation of a lecture.