EXP: Automatically Synthesizing Valid, Personalized, Formative Assessments of CS1 Concepts

全世界有数百万人正在学校、学院、大学和网上学习编程。为了成功地做到这一点，他们需要大量的实践和有意义的反馈，以回应他们的具体困惑，并建立在他们已经拥有的知识基础上。不幸的是，良好的实践内容是具有挑战性的规模和熟练的教师能够提供有意义的反馈是罕见的，往往无法访问。此外，流行的在线学习技术只提供固定数量的静态内容，几乎没有提供有意义的反馈。正因为如此，许多人放弃了学习编程，只有那些有特权接触朋友、家人、导师或老师的人才能提供这种支持。这限制了对这一关键的世纪文化的访问，并最终损害了我们计算劳动力的性别、种族、民族和智力多样性。该项目旨在解决这一问题的一部分，应用编程语言研究的进步，使创建无限数量的多样化的实践内容，以及机器学习的进步，以建立学习者做什么和不知道什么的模型。通过应用计算机科学的这两项进步，该项目将创建一种新的在线学习技术，自动生成评估内容，提供有关解决方案的详细即时反馈，并使用学习者表现的评估信息来生成更个性化的实践，单独针对学习者正在努力掌握的概念。在创建这样一个系统，新的技术产生的实践问题和新的方法来建模学习者的知识介绍编程概念将实现。该项目还将探索学生目前的实践方法，然后将新系统部署到一个大型的入门课堂环境中，以实验方式衡量其对学习和信心的影响。如果该系统是有效的，这些发现有可能为学习者提供更有效的实践和学习。这反过来又可以提高进一步从事计算教育的学习者的能力和多样性。

Millions of people worldwide are trying to learn to code in schools, colleges, universities, and online. To do so successfully, they need significant practice and meaningful feedback that responds to their specific confusions and builds upon the knowledge they already have. Unfortunately, good practice content is challenging to create at scale and skilled teachers capable of providing meaningful feedback are rare and often inaccessible. Moreover, popular online learning technologies only provide a fixed amount of static content and do little to provide meaningful feedback. Because of this, many people give up learning to code, and only those with privileged access to friends, family, mentors, or teachers who can provide this support persist. This limits access to this critical 21st century literacy and ultimately harms the gender, racial, ethnic, and intellectual diversity of our computing workforce. This project seeks to address part of this problem, applying advances in programming languages research that enable the creation of infinite amounts of diverse practice content, and advances in machine learning to build models of what learners do and do not know. By applying these two advances in computer science, the project will create a novel online learning technology that automatically generates assessment content, provides detailed immediate feedback about solutions, and uses assessment information of learners' performance to generate more personalized practice that individually targets concepts that learners are struggling to master. In creating such a system, new techniques for generating practice problems and new approaches to modeling learner knowledge of introductory programming concepts will be realized. The project will also explore students' current approach to practice, then deploy the new system into a large introductory classroom setting to experimentally measure its impact on learning and confidence. If the system is effective, the discoveries have the potential to provide learners in a range of settings more effective practice and learning. This may in turn improve both the competency and diversity of learners who further engage in computing education.