Creating a Scalable and Reliable Peer Assessment System for Mathematical Proofs

本项目开发并评估了一个系统，用于通过同行评估来评估本科数学和计算机科学课程中的证明。它将这个问题作为人类计算的一个实例来处理，使用计算机技术来利用大量人的集体能力来做有用的工作。这就需要分解评估证明的任务，使多个非专家的人能够为评估做出贡献。它要求建立机制，以确保评估过程的质量、一致性和完整性。这项工作建立在以前的经验与原型系统，已被用于两年的计算机科学和数学课程，有针对性地处理大型班级使用团队的本科年级。经验表明，基本战略是可行的，而该项目则致力于确保评估质量和评估员的学习经验。此外，该项目采取了初步措施，对评估系统，可以扩展到支持网络规模的courses.This工作创建结构化的框架，在其中分析的证据，使评估可以被分解为一些任务，适合非专家。它探讨了如何最大限度地提高学习经验的学生通过批判性地评估对方的证据。它创建了一个平台，用于进行定量实验的最有效的方法，教学生如何阅读和评估证明。

This project develops and evaluates a system for assessing proofs in undergraduate mathematics and computer science classes via peer evaluation. It approaches the problem as an instance of human computation, using computer technology to harness the collective capability of large numbers of people to do useful work. This requires breaking down the task of assessing a proof to make it possible for multiple, nonexpert people to contribute to the assessment. It requires instituting mechanisms to ensure the quality, uniformity, and integrity of the assessment process. The work builds on previous experience with a prototype system that had been used for two years in computer science and mathematics courses that targeted handling large classes using teams of undergraduate graders. Experience indicated the feasibility of the basic strategy, while this project works to ensure the quality of the assessments and of the learning experience for the assessors. In addition, this project takes initial steps toward an assessment system that can scale to support web-scale courses.This work creates structured frameworks in which to analyze proofs, so that assessment can be decomposed into a number of tasks, suitable for nonexperts. It explores ways to maximize the learning experience students gain by critically evaluating each other's proofs. It creates a platform for performing quantitative experiments on the most effective methods for teaching students how to read and evaluate proofs.