Preliminary Algorithmic Foundations for Ranking Quizzes and Students from Student-sourced Quizzes

该项目解决了一系列新的问题，这些问题将随着逐渐转向大规模开放式在线课程和大型课程而越来越多地出现，在这些课程中，测试概念主动学习的新评估设计成本很高。该项目的背景是一个基于云计算的测验管理系统原型，该系统通过脚手架活动引导在线学习者，例如创建，回答和改进学生来源的测验项目。然后，系统根据这些活动的结果自动评估学习者的能力和问题的质量。该项目解决了在有效地确定学生和提交项目的质量的系统中的关键算法问题。新算法的初步想法将被开发和部署在测试系统中，以了解它们的优点和缺点。算法的开发和测试将为培养两名研究生提供机会。这也将导致一个具有重大社会影响的可扩展在线评估系统的最终算法设计的改进。特别是，该项目为学生联合排名和基于同行表现和评审形式的反馈的学生来源测验问题开发了新的算法框架。一个关键的贡献是制定这些问题中的抽象问题服从组合和连续优化，这反过来又允许一个完全无关的子领域的人（优化）在未来的工作对这些问题。与此同时，该项目将承担开发第一个这样的算法的基础工作，用当前的方法解决这些问题。

The project addresses a new set of problems that will increasinglyarise with a gradual shift to massively open online courses andlarge classes where new assessments to test active learning of theconcepts are costly to design. The context of the project is aprototype cloud-based quiz management system that leads the onlinelearners through scaffolded activities, such as creating,answering, and improving of student-sourced quiz items. The systemthen automatically evaluates the competence of learners and thequality of questions from outcomes of these activities. Thisproject addresses the key algorithmic issue in such systems ofefficiently determining the quality of the students and thesubmitted items. Preliminary ideas for new algorithms will bedeveloped and deployed in the test system to understand theiradvantages and shortcomings. The development and testing ofalgorithms will provide opportunities for training two graduatestudents. This will also lead to a refinement of the eventualalgorithmic design of a system for scalable online assessments withsignificant societal impacts.In particular, the project develops new algorithmic frameworks forproblems in the joint ranking of students and student-sourced quizquestions based on feedback in the form of peer performance andreview. A key contribution is the formulation of these problems interms of abstract problems amenable to both combinatorial andcontinuous optimization, which in turn allows a completelyunrelated subfield of people (those of optimization) to work onthese problems in the future. At the same time, the project willundertake the groundwork of developing the first such algorithmsthat solve these problems with current approaches.