Collaborative Research: SaTC: CORE: Medium: Bubble Aid: Assistive AI to Improve the Robustness and Security of Reading Hand-Marked Ballots

在扫描和处理手工标记的纸质选票的选举系统领域，最复杂的系统只是看气泡目标上标记的平均暗度。这使得传统的纸质选票扫描器遗漏了没有充分填写的标记，或者将偏离的标记和扫描器噪声错误地分类为填充的气泡。此外，一个固有的安全问题是，这些系统的设计不是为了识别潜在的欺诈投票情况，即单个作者为多个选民填写选票。针对这些问题，该项目的新颖之处在于建立一个人工智能（AI）系统Bubble Aid，该系统借助数百万张真实选票的数据，比现有系统更有效地识别手工标记的气泡目标。该项目更广泛的意义和重要性在于提高使用手写选票的选举系统的效率和安全性，并对手写表格的其他应用产生更广泛的影响，例如标准化测试。该项目正在将Bubble Aid打造为一种辅助工具，作为选举后选票审计系统的一部分，可以在选举最终确定之前的“拉票”期间使用。目的是找出“泡泡援助”与官方选票制表器不一致的选票，并将这些选票交给选举工作人员，以便最终消除歧义。通过应用图像处理和深度学习领域的最新技术，在大量实际选票标记的语料库上进行训练，Bubble-Aid的目标是在手工标记的纸质选票扫描方面取得比任何现有系统都高得多的准确性，并检测到同一手填写多张选票的潜在欺诈情况。在设计和测试Bubble Aid的有效性时，研究团队正在为与这项工作相关的各种任务进行创新的人类参与者实验。这包括要求参与者标记测试选票，故意向其中注入噪声，或填充多个选票以模拟欺诈场景，以获得额外的训练数据。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In the realm of election systems that scan and process hand-marked paper ballots, the most sophisticated ones simply look at the average darkness of marks across a bubble target. This allows traditional paper ballot scanners to miss marks that are insufficiently filled in, or to miscategorize stray marks and scanner noise as filled bubbles. In addition, an inherent security problem is that these systems are not designed to identify potentially fraudulent voting cases where a singular author has filled out ballots for multiple voters. The project’s novelties, responding to these problems, is building Bubble Aid, an Artificial Intelligence (AI) system, aided by data from millions of real ballots that recognizes hand-marked bubble targets more effectively than existing systems. The project's broader significance and importance lies in improving the efficiency and security of election systems that use hand-marked ballots, with wider implications to other applications of hand-marked forms, such as standardized testing.The project is building Bubble Aid as an assistive tool which works as part of a post-election ballot auditing system, which can be used during the "canvass" period before an election is finalized. The goal is to identify the ballots where Bubble Aid disagrees with the official ballot tabulator, and present these ballots to election workers for their ultimate disambiguation. By applying state of the art techniques from the domain of image processing and deep learning, trained on a large corpus of actual ballot marks, Bubble-Aid’s goal is achieving significantly higher accuracy at hand-marked paper ballot scanning than any existing system and also detecting potential fraudulent cases of multiple ballots filled in the same hand. In designing and testing the effectiveness of Bubble Aid, the research team is conducting innovative human-participant experiments for a variety of tasks related to this work. This includes requesting participants to mark test ballots, deliberately injecting noise into them, or filling multiple ballots to emulate the fraudulent scenario, in order to gain additional training data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.