CAREER: Combining Human Judgment and Data-Driven Approaches for the Development of Interpretable Models of Student Behaviors: Applications to Computer Science Education

职业：结合人类判断和数据驱动的方法来开发可解释的学生行为模型：在计算机科学教育中的应用

• 批准号: 1942962
• 负责人: Luc Paquette
• 金额: $ 69.59万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-02-01 至 2025-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1942962&HistoricalAwards=false
• 关键词: CAREER; Combining; Human; Judgment; Data

Debugging, the process of identifying and resolving defects in computer programs, is an important skill to acquire while learning to program. However, many novice programmers, with good understanding of programming, struggle with the debugging process. Despite this, debugging is rarely explicitly taught. This project will use data-driven approaches to study the debugging processes of novice programmers enrolled in a college level introductory computer science course, identify the meaningful elements of their debugging behaviors and how those elements combine to form common debugging strategies. This knowledge will be used to create computer algorithms that are able to identify which debugging strategies students use when programming. These algorithms will be designed to be interpretable by students and instructors. These algorithms will be used to support students in learning efficient debugging strategies, assist instructors in monitoring their students' debugging practices, and help future K-12 teachers in learning about the meaningful elements of the debugging process. The results of this project will positively impact the state of computer science education in both college level introductory computer science courses and in the K-12 level by supporting future teachers in learning how to foster important debugging skills.The project will contribute to the state of the art in student behavior modeling by formalizing an approach that combines knowledge engineering and machine learning to create interpretable models of student behavior. It will provide empirical evidence illustrating how the interpretability of a student behavior model can provide powerful pedagogical advantages beyond its accuracy at predicting a student's behavior. The proposed approach will be applied to study debugging strategies in college level introductory computer science courses through the log data collected from an online problem-solving platform named PrairieLearn. The benefits of interpretable models will be compared to those of traditional machine learning approaches, using rigorous research to identify the best methods for supporting students and instructors. Specifically, the project will apply this approach, leveraging the increased interpretability of the models it creates, to: (1) better understand students' debugging behaviors; (2) support students in self-reflecting about their debugging strategies and developing efficient debugging practices; (3) provide instructors with actionable information about their students' debugging processes; and (4) support future teachers in acquiring expertise in formulating hypotheses about students' debugging strategies. By doing so, the project will contribute to general knowledge about debugging processes for novice programmers, and establish methods to support college students and future K-12 teachers in acquiring explicit knowledge about the debugging process.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

调试是识别和解决计算机程序中的缺陷的过程，是学习编程时需要掌握的一项重要技能。然而，许多对编程有很好了解的新手程序员却在调试过程中遇到困难。尽管如此，调试却很少被明确地教导。该项目将使用数据驱动的方法来研究参加大学水平计算机科学入门课程的新手程序员的调试过程，识别他们的调试行为的有意义的元素以及这些元素如何结合起来形成常见的调试策略。这些知识将用于创建计算机算法，能够识别学生在编程时使用的调试策略。这些算法将被设计为可供学生和教师解释。这些算法将用于支持学生学习有效的调试策略，协助教师监控学生的调试实践，并帮助未来的 K-12 教师了解调试过程中有意义的元素。该项目的结果将通过支持未来教师学习如何培养重要的调试技能，对大学计算机科学入门课程和 K-12 级别的计算机科学教育状况产生积极影响。该项目将通过形式化一种结合知识工程和机器学习来创建可解释的学生行为模型的方法，为学生行为建模的发展做出贡献。它将提供经验证据，说明学生行为模型的可解释性如何在预测学生行为的准确性之外提供强大的教学优势。所提出的方法将用于通过从名为 PrairieLearn 的在线问题解决平台收集的日志数据来研究大学计算机科学入门课程中的调试策略。将可解释模型的优点与传统机器学习方法进行比较，通过严格的研究来确定支持学生和教师的最佳方法。具体来说，该项目将应用这种方法，利用其创建的模型增强的可解释性，以：（1）更好地理解学生的调试行为； (2) 支持学生自我反思自己的调试策略并开发有效的调试实践； (3) 向教师提供有关学生调试过程的可操作信息； (4) 支持未来的教师获得制定有关学生调试策略的假设的专业知识。通过这样做，该项目将为新手程序员提供关于调试过程的一般知识，并建立方法来支持大学生和未来的 K-12 教师获得有关调试过程的明确知识。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Using submission log data to investigate novice programmers’ employment of debugging strategies

使用提交日志数据调查新手程序员调试策略的使用

• DOI: 10.1145/3576050.3576094
• 发表时间: 2023
• 期刊: ACM
• 作者: Liu, Qianhui;Paquette, Luc
• 通讯作者: Paquette, Luc

Combining latent profile analysis and programming traces to understand novices’ differences in debugging

结合潜在的配置文件分析和编程跟踪来了解新手在调试中的差异

• DOI: 10.1007/s10639-022-11343-7
• 发表时间: 2022
• 期刊: Education and Information Technologies
• 影响因子: 5.5
• 作者: Zhang, Yingbin;Paquette, Luc;Pinto, Juan D.;Liu, Qianhui;Fan, Aysa Xuemo
• 通讯作者: Fan, Aysa Xuemo

Investigating the reliability of aggregate measurements of learning process data: From theory to practice

• DOI: 10.1111/jcal.12951
• 发表时间: 2024-02
• 期刊: J. Comput. Assist. Learn.
• 作者: Yingbin Zhang;Yafei Ye;Luc Paquette;Yibo Wang;Xiaoyong Hu

Investigating the Relationship Between Programming Experience and Debugging Behaviors in an Introductory Computer Science Course

在计算机科学入门课程中调查编程经验与调试行为之间的关系

• 发表时间: 2023
• 期刊: Springer Nature Switzerland
• 作者: Pinto, Juan D.;Liu, Qianhui;Paquette, Luc;Zhang, Yingbin;Fan, Aysa X.
• 通讯作者: Fan, Aysa X.

Utilizing programming traces to explore and model the dimensions of novices' code‐writing skill

利用编程痕迹探索和建模新手代码维度——写作能力

• DOI: 10.1002/cae.22622
• 发表时间: 2023
• 期刊: Computer Applications in Engineering Education
• 影响因子: 2.9
• 作者: Zhang, Yingbin;Paquette, Luc;Pinto, Juan D.;Fan, Aysa Xuemo
• 通讯作者: Fan, Aysa Xuemo