AF: Small: Mechanism Design for the Classroom

机制设计研究如何设计一个系统的规则，以便当参与系统的个体具有战略意义时获得良好的结果。该项目开发并启动了一系列课堂机制设计问题的理论研究。具体来说，它将课堂视为一个计算系统，其中一些参与者可能操纵系统以获得更好的个人结果（即学生），而一些参与者可能不可靠（即评分者）。指导员的目标是制定一些具有自然目标的政策，例如，优化学习成果，评分政策的公平性，以及参与者（包括学生和评分者）努力的效率。通过将课堂理解为机制设计的应用领域，可以改善课堂成果。为基于实践的机构设计奠定了基础。这个基础可能会对机制设计的其他应用领域产生影响，例如在线市场。本项目探讨了三个主要重点：异质性评分的公平性，评分优化学习激励，以及学生反馈机制的设计。要点1：在在线考试中，从大量题库中随机抽取问题是一种流行的作弊手段。然而，当学生被分配具有不同难度的问题时，公平的评估并不是直截了当的。有些学生可能会被分配比其他人更容易的问题，简单的平均分将有利于这些更幸运的学生。公平和准确评估的表现基准是学生在完整题库中的平均成绩。该项目旨在(a)开发评分算法，使其在任何分配给学生的问题上都能在这个基准上表现良好，(b)了解分配给学生的问题结构对成绩的影响，即考试设计。要点2：当学生被分配任务时，他们的努力程度取决于他们的努力程度的评分。努力可以带来学习；然而，它不是直接观察到的。例如，一篇文章的阅读任务可能会通过阅读理解问题来评估，这些问题的答案可以被评估，但阅读的努力程度无法被观察到。知识的评分可以用评分规则的范式来理解。评分规则是激励预测者报告对未知状态的预测的经典范例。在课堂环境中，学生对课程材料问题的回答可以被解释为对正确答案的预测。在此背景下，本项目旨在开发一种优化评分规则的理论，即确定激励学生付出努力（从而学习）的评分规则。推力3：对学生的反馈使他们能够评估他们的努力如何导致与他们相关的结果，例如学习或成绩。例如，不太精确的分数反馈可能会导致更一致的努力，因为它可以避免学生对虚假的高分或低分反应过度（参见机器学习中的过拟合）。信息设计是一个经典的范例，在这个范例中，委托人向代理人发出关于未知状态的信号，以诱使代理人为委托人采取更有利的行动。该项目旨在了解作为信息设计问题的连续提供反馈给学生，并设计良好的反馈机制。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Mechanism design studies how the rules of a system can be designed so that good outcomes are obtained when individuals participating in the system are strategic. This project develops -- and initiates the theoretical study of -- a collection of mechanism design problems for the classroom. Specifically, it views the classroom as a computational system where some participants may manipulate the system to obtain better individual outcomes (i.e., the students) and some participants may be unreliable (i.e., the graders). The instructor aims to put in place policies with a number of natural objectives, e.g., optimizing learning outcomes, fairness of grading policies, and efficiency with respect to effort from participants (both students and graders). By understanding the classroom as an application domain for mechanism design, classroom outcomes can be improved. Moreover, a foundation for mechanism design that is grounded in practice can be established. This foundation may have an impact on other application domains for mechanism design, such as online markets.This project explores three main thrusts: fairness in heterogeneous grading, grading to optimize study incentives, and the design of student feedback mechanisms. Thrust 1: Randomizing questions from a large bank of questions is a popular cheating deterrent in online exams. When students are assigned questions with heterogeneous difficulties, however, fair assessment is not straightforward. Some students may be assigned easier questions than others and the simple averaging of scores will favor these more fortunate students. A performance benchmark for fair and accurate assessment is a student’s average grade on the full question bank. This project aims to (a) develop grading algorithms that perform well with respect to this benchmark for any assignment of questions to students, and (b) understand the impact of the structure of the assignment of questions to students on performance, i.e., exam design. Thrust 2: When students are assigned tasks, their level of effort depends on how their effort is graded. Effort can result in learning; however, it is not directly observed. For example, an article reading task might be assessed via reading comprehension questions where answers to these questions can be assessed, but the amount of effort of reading cannot be observed. The grading of knowledge can be understood in the paradigm of scoring rules. Scoring rules are a classical paradigm for incentivizing a forecaster to report a prediction about an unknown state. In the classroom context, student answers to questions about course material can be interpreted as a prediction of the correct answer. In this context, this project aims to develop a theory for the optimization of scoring rules, i.e., identifying scoring rules that incentivize the students to exert effort (that results in learning). Thrust 3: Feedback to students enables them to assess how their effort leads to outcomes that are relevant to them, such as learning or grades. For example, less precise feedback on grades could lead to more consistent effort because it would avoid students overreacting to spurious high or low grades (cf. overfitting in machine learning). Information design is a classical paradigm where a principal signals an agent about an unknown state to entice the agent to take a more favorable action for the principal. The project aims to understand the sequential provision of feedback to students as a problem of information design and to design good feedback mechanisms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.