CAREER: Improving Machine Learning Education through Data-driven Support for Pipeline Design and Implementation

机器学习（ML）是一种强大的计算方法，允许计算机自动从数据中学习模式或模型，以完成复杂的任务，例如从医学图像诊断疾病，翻译语言或进行人类级别的聊天机器人对话。这些技术在改善生活方面具有令人难以置信的潜力，但如果设计不当，也会造成伤害。因此，学习如何创建ML模型，以及如何以道德的方式这样做，是STEM学科中越来越重要的技能。然而，ML对于大学生来说也是一个具有挑战性的课题，研究人员才刚刚开始探索ML教育的有效实践。因此，该项目的目标是发现可以改善学生在ML教室中学习的新技术，并促进我们对学生如何学习ML的理解。具体来说，该项目将开发新的算法，生成个性化的反馈，提示和示例，以支持学生设计有效的ML模型，将其作为代码实现，并反思其设计选择的道德影响。这些帮助功能将自动化，使他们能够扩展到任何规模的教室，即使在教师不可用时也能为学生提供支持。该项目将在五年内通过在三个不同的教室进行研究来评估和改进这些技术，可能会改善1,500多名学生的成果。此外，通过分析日志数据、调查、访谈和学生成果，该项目将促进研究人员对学生如何学习ML的理解，并为支持他们的干预措施的设计提供信息。该项目将通过开发和分发"ML for Social Good"课程，并通过多种渠道传播其开发的技术，包括为16名教师举办的专业发展研讨会，进一步发挥影响力。该项目将探索支持学生学习设计和实施开放式ML管道的方法。它将通过新颖的数据驱动算法来实现这一目标，这些算法可以生成个性化的反馈、提示和示例来支持学生。这些算法将使用以前学生的跟踪数据为未来的学生提供支持，有效地利用ML的进步来支持ML教育。虽然以前的数据驱动支持主要用于早期计算机科学课程，以支持新手，但拟议的研究将推进这些技术，以支持从事复杂和开放式ML项目的更高级程序员。该提案的三个重点将回答ML教育研究中的基本问题：重点1将调查明确的ML设计过程的价值，该过程将计划与实施分开，并将探索数字接口如何支撑这一过程并促进教师反馈。第二部分将研究如何设计自动化提示和反馈，以有效地支持ML管道设计，以及即时算法帮助与延迟教师帮助的比较启示。Thrust 3将探索学生在实现ML管道代码时如何使用，集成和学习编程示例，并将开发使用学生数据个性化这些示例的方法。该项目将评估自动化支持在实验室和课堂研究中的影响，每学期将有数百名学生参加多门课程和教师。这项研究将通过对学生行为和工件的系统分析，为计算和ML教育研究领域做出贡献，以进一步了解学习者在构建ML管道时获取和展示知识的过程。该项目还将推进自动编程和设计反馈的技术，这将为未来支持其他高级计算课程的研究奠定基础。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine Learning (ML) is a powerful computational method that allows computers to automatically learn patterns, or models, from data to complete complex tasks such as diagnosing diseases from medical images, translating languages, or holding human-level chatbot conversations. These technologies have incredible potential to improve lives but can also cause harm if they are not designed thoughtfully. Learning how to create ML models, and how to do so in an ethical way, is therefore an increasingly important skill across STEM disciplines. However, ML is also a challenging subject for university students to learn, and researchers have only just begun to explore effective practices for ML education. Therefore, the goal of this project is to discover new technologies that can improve student learning in ML classrooms, and to advance our understanding of how students learn ML. Specifically, the project will develop new algorithms that generate personalized feedback, hints, and examples to support students in designing effective ML models, implementing them as code, and reflecting on the ethical implications of their design choices. These help features will be automated, allowing them to scale to classrooms of any size, supporting students even when the instructor is unavailable. This project will evaluate and improve these technologies through studies in three different classrooms over five years, potentially improving outcomes for over 1,500 students. Additionally, by analyzing log data, surveys, interviews, and student outcomes, this project will advance researchers' understanding of how students learn ML and inform the design of interventions to support them. The project will have further impact by developing and distributing an "ML for Social Good" curriculum and by disseminating the technologies it develops through multiple channels, including professional development workshops for 16 instructors.This project will discover methods for supporting students in learning to design and implement open-ended ML pipelines. It will do so through novel, data-driven algorithms that generate personalized feedback, hints, and examples to support students. These algorithms will use trace data from prior students to generate support for future students, effectively using advances in ML to support ML education. While prior data-driven support is primarily used in early computer science courses to support novices, the proposed research will advance these techniques to support more advanced programmers working on complex and open-ended ML projects. The three thrusts of this proposal will answer fundamental questions in ML education research: Thrust 1 will investigate the value of an explicit ML design process that separates planning from implementation and will explore how a digital interface can scaffold this process and facilitate instructor feedback. Thrust 2 will investigate how to design automated hints and feedback to effectively support ML pipeline design and the comparative affordances of immediate algorithmic help vs delayed instructor help. Thrust 3 will explore how students use, integrate, and learn from programming examples when implementing ML pipeline code and will develop methods to personalize these examples using student data. This project will evaluate the impact of automated support in laboratory and classroom studies, reaching hundreds of students each semester across multiple courses and instructors. This research will make contributions to the fields of computing and ML education research through systematic analysis of students' behavior and artifacts to further our understanding of the processes by which learners acquire and demonstrate knowledge when constructing ML pipelines. The project will also advance techniques for automated programming and design feedback, which will build a foundation for future research supporting other advanced computing courses.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.