Preliminary Algorithmic Foundations for Ranking Quizzes and Students from Student-sourced Quizzes

该项目解决了一系列新的问题，这些问题将随着大规模开放在线课程和大班的逐渐转变而日益出现，在这些课程中，用于测试主动学习概念的新评估的设计成本很高。该项目的背景是一个基于云的测试管理系统原型，该系统引导在线学习者完成脚手架活动，例如创建、回答和改进来自学生的测试项目。然后，系统会根据这些活动的结果自动评估学习者的能力和问题的质量。该项目解决了有效确定学生和提交项目质量的关键算法问题。新算法的初步想法将在测试系统中开发和部署，以了解其优点和缺点。算法的开发和测试将为培养两名研究生提供机会。这也将导致对具有重大社会影响的可扩展在线评估系统的最终算法设计的改进。特别是，该项目开发了新的算法框架，用于解决学生联合排名问题和基于同伴表现和审查形式的反馈的学生来源的测验问题。一个关键的贡献是将这些问题以适合组合优化和连续优化的抽象问题的形式表述出来，这反过来又允许一个完全不相关的子领域的人（优化的人）在未来研究这些问题。与此同时，该项目将承担开发第一个这样的算法的基础工作，用现有的方法解决这些问题。

The project addresses a new set of problems that will increasinglyarise with a gradual shift to massively open online courses andlarge classes where new assessments to test active learning of theconcepts are costly to design. The context of the project is aprototype cloud-based quiz management system that leads the onlinelearners through scaffolded activities, such as creating,answering, and improving of student-sourced quiz items. The systemthen automatically evaluates the competence of learners and thequality of questions from outcomes of these activities. Thisproject addresses the key algorithmic issue in such systems ofefficiently determining the quality of the students and thesubmitted items. Preliminary ideas for new algorithms will bedeveloped and deployed in the test system to understand theiradvantages and shortcomings. The development and testing ofalgorithms will provide opportunities for training two graduatestudents. This will also lead to a refinement of the eventualalgorithmic design of a system for scalable online assessments withsignificant societal impacts.In particular, the project develops new algorithmic frameworks forproblems in the joint ranking of students and student-sourced quizquestions based on feedback in the form of peer performance andreview. A key contribution is the formulation of these problems interms of abstract problems amenable to both combinatorial andcontinuous optimization, which in turn allows a completelyunrelated subfield of people (those of optimization) to work onthese problems in the future. At the same time, the project willundertake the groundwork of developing the first such algorithmsthat solve these problems with current approaches.