IUSE: EHR: Improving Undergraduate Algorithms Instructions with Online Feedback

IUSE：EHR：通过在线反馈改进本科生算法说明

• 批准号: 1903304
• 负责人: George Heineman
• 金额: $ 29.91万
• 依托单位: Worcester Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-07-01 至 2023-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1903304&HistoricalAwards=false
• 关键词: IUSE; EHR; Improving; Undergraduate; Algorithms

With support from the NSF Improving Undergraduate STEM Education Program: Education and Human Resources (IUSE: EHR), this project aims to serve the national interest by leveraging artificial intelligence (AI) techniques to improve student learning and instructor productivity in computing courses. An important aspect of an instructor's job is to provide feedback on student work. Research in cognitive science demonstrates that students more effectively learn concepts if they explain answers to cognitively demanding questions. Learning is further improved when instructor feedback is provided. In large classes, it is difficult for instructors provide all students with the feedback and help they need. This project will develop a tool that uses machine learning to automatically generate feedback on student solutions. The tool is designed to reduce the grading burden of the instructor while ensuring that students receive frequent and worthwhile feedback. The project aims to generate knowledge about how AI can help to engage students without requiring significant instructor time. This tool will impact thousands of students as it is incorporated into ASSISTments, which is used by 50,000 students and is a free service of Worcester Polytechnic Institute (WPI). The goal of this project is to implement and study a tool that uses state-of-the-art machine learning techniques and natural language processing methods to automate instructor feedback. In the first phase of the project, students in the Algorithms course at WPI will be assigned open-ended questions to complete prior to each lecture. Teaching staff for the course will manually grade student responses. The data set generated in this phase will be used to develop and train the algorithms for the automated tool. The tool will apply methods of varying complexities including regression and tree-based modeling methods, and deep learning methods including long short-term memory (LSTM). It will analyze student responses and suggest feedback and a grade. Instructors may choose to accept the tool's suggestions or override the suggestions to provide different feedback. A randomized control trial will assign students to receive manual feedback or feedback generated with the help of the tool. The usability of the tool will be evaluated through interviews with the instructors. Data on how often the tool's suggestions are overridden and how long it takes the instructor to grade each solution will also be collected. Student experience with the tool will be evaluated through online surveys. Student learning will be evaluated through posttests given each week. The project is expected to improve student learning and instructor productivity in the WPI Algorithms course. It is also has potential to contribute to the broader fields of machine learning, natural language processing, and education through the study, generation, and deployment of effective feedback. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在NSF改善本科STEM教育计划：教育和人力资源（IUSE：EHR）的支持下，该项目旨在通过利用人工智能（AI）技术来改善学生学习和教师在计算课程中的生产力，从而为国家利益服务。教师工作的一个重要方面是对学生的作业提供反馈。认知科学的研究表明，如果学生解释认知要求高的问题的答案，他们会更有效地学习概念。当教师提供反馈时，学习会得到进一步改善。在大班中，教师很难为所有学生提供他们需要的反馈和帮助。该项目将开发一种工具，使用机器学习自动生成对学生解决方案的反馈。该工具旨在减轻教师的评分负担，同时确保学生经常收到有价值的反馈。该项目旨在生成有关人工智能如何在不需要大量教师时间的情况下帮助学生参与的知识。该工具将影响成千上万的学生，因为它被纳入ASSISTments，这是由50,000名学生使用，是伍斯特理工学院（WPI）的免费服务。 该项目的目标是实现和研究一种工具，该工具使用最先进的机器学习技术和自然语言处理方法来自动化教师反馈。在项目的第一阶段，WPI算法课程的学生将在每次讲座之前完成开放式问题。该课程的教学人员将手动对学生的回答进行评分。在此阶段生成的数据集将用于开发和培训自动化工具的算法。该工具将应用不同复杂性的方法，包括回归和基于树的建模方法，以及包括长短期记忆（LSTM）的深度学习方法。它将分析学生的反应，并建议反馈和评分。教师可以选择接受该工具的建议或忽略这些建议以提供不同的反馈。随机对照试验将分配学生接受手动反馈或在工具帮助下生成的反馈。该工具的可用性将通过与教员的访谈进行评估。还将收集有关工具建议被覆盖的频率以及教师对每个解决方案进行评分所需的时间的数据。学生使用该工具的经验将通过在线调查进行评估。学生的学习将通过每周的后测进行评估。该项目有望提高WPI算法课程的学生学习和教师生产力。它也有潜力通过研究、生成和部署有效的反馈，为机器学习、自然语言处理和教育等更广泛的领域做出贡献。NSF IUSE：EHR计划支持研究和开发项目，以提高所有学生STEM教育的有效性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Leveraging natural language processing to support automated assessment and feedback for student open responses in mathematics

利用自然语言处理支持学生数学开放式回答的自动评估和反馈

• 发表时间: 2023
• 期刊: Journal of computer assisted learning
• 影响因子: 5
• 作者: Botelho, Anthony;Baral, Sami;Erickson, John;Benachamardi, Priyanka;Heffernan, Neil
• 通讯作者: Heffernan, Neil

The automated grading of student open responses in mathematics

学生数学开放式回答的自动评分

• DOI: 10.1145/3375462.3375523
• 发表时间: 2020
• 期刊: Tenth International Conference on Learning Analytics & Knowledge
• 作者: Erickson, John A.;Botelho, Anthony F.;McAteer, Steven;Varatharaj, Ashvini;Heffernan, Neil T.
• 通讯作者: Heffernan, Neil T.

Enhancing Auto-scoring of Student Open Responses in the Presence of Mathematical Terms and Expressions

在存在数学术语和表达式的情况下增强学生开放式回答的自动评分

• 发表时间: 2022
• 期刊: AIED 2022: The 23rd International Conference on Artificial Intelligence in Education
• 作者: Baral, Sami;Seetharaman, Karthik;Botelho, Anthony;Wang, Anzhou
• 通讯作者: Wang, Anzhou

Deep Learning or Deep Ignorance? Comparing Untrained Recurrent Models in Educational Contexts.

深度学习还是深度无知？

• DOI: 10.1007/978-3-031-11644-5_23
• 发表时间: 2022
• 期刊: AIED 2022
• 作者: Botelho, Anthony;Prihar, Ethan;Heffernan, Neil
• 通讯作者: Heffernan, Neil

Using Past Data to Warm Start Active Machine Learning: Does Context Matter?

使用过去的数据来热启动主动机器学习：上下文重要吗？

• DOI: 10.1145/3448139.3448154
• 发表时间: 2021
• 期刊: LAK21: 11th International Learning Analytics and Knowledge Conference
• 作者: Karumbaiah, Shamya;Lan, Andrew;Nagpal, Sachit;Baker, Ryan S.;Botelho, Anthony;Heffernan, Neil
• 通讯作者: Heffernan, Neil