Collaborative Research: SaTC: CORE: Medium: Bubble Aid: Assistive AI to Improve the Robustness and Security of Reading Hand-Marked Ballots

在扫描和处理手写选票的选举系统领域，最复杂的系统只是查看气泡目标上标记的平均暗度。这使得传统的纸质选票扫描仪错过了未充分填充的标记，或者将杂散标记和扫描仪噪声误分类为填充气泡。此外，一个固有的安全问题是，这些系统不是设计来识别潜在的欺诈性投票情况下，一个单一的作者已经填写了多个选民的选票。针对这些问题，该项目的创新之处在于构建Bubble Aid，这是一种人工智能（AI）系统，由数百万张真实的选票数据辅助，比现有系统更有效地识别手工标记的气泡目标。该项目更广泛的意义和重要性在于提高使用手写选票的选举系统的效率和安全性，并对手写表格的其他应用产生更广泛的影响，例如标准化测试，该项目正在开发Bubble Aid，作为选举后选票审计系统的一部分，可在选举最后确定之前的“拉票”期间使用。我们的目标是确定选票的泡沫援助不同意与官方选票制表，并提出这些选票选举工作人员为他们的最终消除歧义。通过应用图像处理和深度学习领域的最先进技术，并在大量实际选票上进行训练，Bubble-Aid的目标是实现比任何现有系统更高的手写纸质选票扫描准确性，并检测同一只手填写多张选票的潜在欺诈案件。在设计和测试Bubble Aid的有效性时，研究团队正在进行创新的人类参与实验，以完成与这项工作相关的各种任务。这包括要求参与者标记测试选票，故意向其中注入噪音，或填写多张选票以模拟欺诈场景，以获得额外的培训数据。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

In the realm of election systems that scan and process hand-marked paper ballots, the most sophisticated ones simply look at the average darkness of marks across a bubble target. This allows traditional paper ballot scanners to miss marks that are insufficiently filled in, or to miscategorize stray marks and scanner noise as filled bubbles. In addition, an inherent security problem is that these systems are not designed to identify potentially fraudulent voting cases where a singular author has filled out ballots for multiple voters. The project’s novelties, responding to these problems, is building Bubble Aid, an Artificial Intelligence (AI) system, aided by data from millions of real ballots that recognizes hand-marked bubble targets more effectively than existing systems. The project's broader significance and importance lies in improving the efficiency and security of election systems that use hand-marked ballots, with wider implications to other applications of hand-marked forms, such as standardized testing.The project is building Bubble Aid as an assistive tool which works as part of a post-election ballot auditing system, which can be used during the "canvass" period before an election is finalized. The goal is to identify the ballots where Bubble Aid disagrees with the official ballot tabulator, and present these ballots to election workers for their ultimate disambiguation. By applying state of the art techniques from the domain of image processing and deep learning, trained on a large corpus of actual ballot marks, Bubble-Aid’s goal is achieving significantly higher accuracy at hand-marked paper ballot scanning than any existing system and also detecting potential fraudulent cases of multiple ballots filled in the same hand. In designing and testing the effectiveness of Bubble Aid, the research team is conducting innovative human-participant experiments for a variety of tasks related to this work. This includes requesting participants to mark test ballots, deliberately injecting noise into them, or filling multiple ballots to emulate the fraudulent scenario, in order to gain additional training data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.