AF: Small: Mechanism Design for the Classroom

机制设计研究如何设计系统的规则，以便在参与系统的个人具有战略意义时获得良好的结果。这个项目开发--并启动--一系列课堂机械设计问题的理论研究。具体地说，它将课堂视为一个计算系统，在这个系统中，一些参与者可能操纵系统以获得更好的个人结果(即学生)，而一些参与者可能不可靠(即评分员)。教师的目标是制定具有许多自然目标的政策，例如，优化学习结果，评分政策的公平性，以及参与者(学生和评分者)努力的效率。通过将课堂理解为机制设计的应用领域，课堂结果可以得到改善。此外，还可以为扎根于实践的机制设计奠定基础。这个基础可能会对其他机制设计的应用领域产生影响，如在线市场。本项目探索了三个主要推动力：异质评分的公平性、评分优化学习激励和学生反馈机制的设计。推力1：在线考试中，从大量题库中随机抽取问题是一种常见的作弊威慑手段。然而，当学生被分配有不同难度的问题时，公平的评估并不是一帆风顺的。一些学生可能会被分配到比其他学生更容易的问题，简单的平均分数将有利于这些更幸运的学生。公平和准确评估的一个表现基准是学生在完整题库中的平均成绩。这个项目的目的是(A)开发在任何向学生布置的问题的基准方面表现良好的评分算法，以及(B)了解向学生布置的问题的结构对表现的影响，即考试设计。主旨二：当学生被分配任务时，他们的努力程度取决于他们的努力如何评分。努力可以导致学习；然而，它不是直接观察到的。例如，一篇文章的阅读任务可以通过阅读理解问题来评估，其中可以评估这些问题的答案，但不能观察阅读的努力程度。知识的分级可以在评分规则的范式中得到理解。评分规则是激励预报员报告对未知状态的预测的经典范例。在课堂环境中，学生对有关课程材料的问题的回答可以解释为对正确答案的预测。在此背景下，本项目旨在开发一种优化评分规则的理论，即确定激励学生努力学习(从而导致学习)的评分规则。主旨3：对学生的反馈使他们能够评估他们的努力是如何导致与他们相关的结果的，比如学习或成绩。例如，不那么精确的分数反馈可能会导致更一致的努力，因为这将避免学生对虚假的高分或低分反应过度(参见。机器学习中的过度适应)。信息设计是一种经典的范式，其中委托人向代理人发出关于未知状态的信号，以诱使代理人采取更有利于委托人的行动。该项目旨在将连续向学生提供反馈理解为信息设计的问题，并设计良好的反馈机制。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Mechanism design studies how the rules of a system can be designed so that good outcomes are obtained when individuals participating in the system are strategic. This project develops -- and initiates the theoretical study of -- a collection of mechanism design problems for the classroom. Specifically, it views the classroom as a computational system where some participants may manipulate the system to obtain better individual outcomes (i.e., the students) and some participants may be unreliable (i.e., the graders). The instructor aims to put in place policies with a number of natural objectives, e.g., optimizing learning outcomes, fairness of grading policies, and efficiency with respect to effort from participants (both students and graders). By understanding the classroom as an application domain for mechanism design, classroom outcomes can be improved. Moreover, a foundation for mechanism design that is grounded in practice can be established. This foundation may have an impact on other application domains for mechanism design, such as online markets.This project explores three main thrusts: fairness in heterogeneous grading, grading to optimize study incentives, and the design of student feedback mechanisms. Thrust 1: Randomizing questions from a large bank of questions is a popular cheating deterrent in online exams. When students are assigned questions with heterogeneous difficulties, however, fair assessment is not straightforward. Some students may be assigned easier questions than others and the simple averaging of scores will favor these more fortunate students. A performance benchmark for fair and accurate assessment is a student’s average grade on the full question bank. This project aims to (a) develop grading algorithms that perform well with respect to this benchmark for any assignment of questions to students, and (b) understand the impact of the structure of the assignment of questions to students on performance, i.e., exam design. Thrust 2: When students are assigned tasks, their level of effort depends on how their effort is graded. Effort can result in learning; however, it is not directly observed. For example, an article reading task might be assessed via reading comprehension questions where answers to these questions can be assessed, but the amount of effort of reading cannot be observed. The grading of knowledge can be understood in the paradigm of scoring rules. Scoring rules are a classical paradigm for incentivizing a forecaster to report a prediction about an unknown state. In the classroom context, student answers to questions about course material can be interpreted as a prediction of the correct answer. In this context, this project aims to develop a theory for the optimization of scoring rules, i.e., identifying scoring rules that incentivize the students to exert effort (that results in learning). Thrust 3: Feedback to students enables them to assess how their effort leads to outcomes that are relevant to them, such as learning or grades. For example, less precise feedback on grades could lead to more consistent effort because it would avoid students overreacting to spurious high or low grades (cf. overfitting in machine learning). Information design is a classical paradigm where a principal signals an agent about an unknown state to entice the agent to take a more favorable action for the principal. The project aims to understand the sequential provision of feedback to students as a problem of information design and to design good feedback mechanisms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.