Helping Improve and Scale Introductory Programming Courses through Automated Code-Reading Exercises

通过自动代码阅读练习帮助改进和扩展入门编程课程

• 批准号: 2121424
• 负责人: Craig Zilles
• 金额: $ 30万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2121424&HistoricalAwards=false
• 关键词: Helping; Improve; Scale; Introductory; Programming

This project aims to serve the national interest by developing a tool that will help computing departments to prepare more students to enter the technical workforce, in particular to improve their ability to program. Explain in plain English" (EipE) questions are considered by many to play an important role in helping students to learn to program by emphasizing “code reading,” an important developmental skill that supports “code writing.” However, while many programming course activities (e.g., code writing) can be objectively graded in a straight-forward manner, activities like "Explain in plain English" (EipE) questions that ask students to read a given piece of code and describe its function in English are difficult to grade consistently. While EipE questions are well regarded by researchers, they are not in widespread use instructionally, presumably due to the burden of manually grading them and the slow feedback provided to students. The goal of this project is to develop a natural language processing (NLP) based autograder for EipE questions that will provide students with immediate feedback, ease the instructional burden of using EPiE questions in the classroom, and help increase the use of this effective pedagogical approach.Starting from a bag-of-words and bigram-based implementation that is already accurate enough for use in low stakes assessments, this project will refine this implementation using publicly available pre trained Transformer architectures. The project centers around two research questions: (1) How does student performance on code reading activities relate to performance on other activities in introductory programming courses? and (2) Can we improve student success rates in introductory programming courses by introducing automated formative EipE assessments? The proposed qualitative and quantitative studies will contribute to the understanding of how novices learn to program and the role that learning to read code plays in learning. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在通过开发一种工具来服务于国家利益，该工具将帮助计算部门为更多的学生进入技术劳动力市场做好准备，特别是提高他们的编程能力。“用简单英语解释”（EipE）问题被许多人认为是通过强调“代码阅读”（一种支持“代码编写”的重要发展技能）来帮助学生学习编程的重要角色。然而，虽然许多编程课程活动（例如，代码编写）可以以直接的方式客观地评分，但是像“用简单英语解释”（EipE）问题这样的活动，要求学生阅读给定的代码并用英语描述其功能，很难一致地评分。虽然EipE问题受到研究人员的好评，但它们并没有被广泛使用，大概是由于手动评分的负担以及向学生提供的反馈缓慢。这个项目的目标是开发一个基于自然语言处理（NLP）的EipE问题自动评分器，为学生提供即时反馈，减轻在课堂上使用EPiE问题的教学负担，并帮助增加这种有效教学方法的使用。从一个词袋和基于双语法的实现开始，它已经足够准确，可以用于低风险评估，这个项目将使用公开的预训练的Transformer体系结构来改进这个实现。 该项目围绕两个研究问题：（1）学生在代码阅读活动中的表现如何与入门编程课程中其他活动的表现相关？以及（2）我们能否通过引入自动化的EipE形成性评估来提高学生在入门编程课程中的成功率？所提出的定性和定量研究将有助于理解新手如何学习编程以及学习阅读代码在学习中所起的作用。NSF IUSE：EHR计划支持研究和开发项目，以提高所有学生STEM教育的有效性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Am I Wrong, or Is the Autograder Wrong? Effects of AI Grading Mistakes on Learning

• DOI: 10.1145/3568813.3600124
• 发表时间: 2023-08
• 期刊: Proceedings of the 2023 ACM Conference on International Computing Education Research - Volume 1
• 作者: T. Li;Silas Hsu;Max Fowler;Zhilin Zhang;C. Zilles;Karrie Karahalios

Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication study

在重复研究中重新评估 CS1 中解释、追踪和写作技能之间的关系

• DOI: 10.1080/08993408.2022.2079866
• 发表时间: 2022
• 期刊: Computer Science Education
• 影响因子: 2.7
• 作者: Fowler, Max;Smith IV, David H.;Hassan, Mohammed;Poulsen, Seth;West, Matthew;Zilles, Craig
• 通讯作者: Zilles, Craig

On Students' Usage of Tracing for Understanding Code

论学生使用追踪来理解代码

• DOI: 10.1145/3545945.3569741
• 发表时间: 2023
• 期刊: SIGCSE 2023: Proceedings of the 54th ACM Technical Symposium on Computer Science Education
• 作者: Hassan, Mohammed;Zilles, Craig
• 通讯作者: Zilles, Craig

Evaluating Beacons, the Role of Variables, Tracing, and Abstract Tracing for Teaching Novices to Understand Program Intent

评估信标、变量的作用、跟踪和抽象跟踪，以帮助新手理解程序意图

• DOI: 10.1145/3568813.3600140
• 发表时间: 2023
• 期刊: Proceedings of the 2023 ACM Conference on International Computing Education Research
• 作者: Hassan, Mohammed;Cunningham, Kathryn;Zilles, Craig
• 通讯作者: Zilles, Craig

Using Context-Free Grammars to Scaffold and Automate Feedback in Precise Mathematical Writing

使用上下文无关语法来构建和自动化精确数学写作中的反馈

• DOI: 10.1145/3545945.3569728
• 发表时间: 2023
• 期刊: SIGCSE 2023: Proceedings of the 54th ACM Technical Symposium on Computer Science Education
• 作者: Xia, Jason;Zilles, Craig
• 通讯作者: Zilles, Craig