Beyond Bot Detection: Combating Fraudulent Online Survey Takers
Beyond Bot Detection: Combating Fraudulent Online Survey Takers
复制标题
超越机器人检测：打击欺诈性在线调查者
DOI：
10.1145/3485447.3512230
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Wang, Gang
中科院分区：
文献类型：
--
作者：
Zhang, Ziyi;Zhu, Shuofei;Mink, Jaron;Xiong, Aiping;Song, Linhai;Wang, Gang
人们建议使用不同的技术来检测在线调查中的欺诈性回复，但很少有研究来系统地测试它们在实践中的实际作用程度。在本文中，我们对两项互补的在线调查中的 22 项反欺诈测试进行了实证评估。第一项调查在公共在线论坛和社交媒体网络上招募 Rust 程序员。我们发现欺诈性受访者同时涉及机器人和人类特征。在不同的反欺诈测试中，基于领域知识设计的测试是最有效的。通过结合单独的测试，我们可以实现与商业技术一样好的检测性能，同时使结果更易于解释。为了在更广泛的背景下探索这些测试，我们在 Amazon Mechanical Turk (MTurk) 上进行了一项不同的调查。结果表明，对于不需要用户具备任何领域知识的通用调查，区分欺诈性回复更加困难。然而，一部分测试仍然有效。
Different techniques have been recommended to detect fraudulent responses in online surveys, but little research has been taken to systematically test the extent to which they actually work in practice. In this paper, we conduct an empirical evaluation of 22 anti-fraud tests in two complementary online surveys. The first survey recruits Rust programmers on public online forums and social media networks. We find that fraudulent respondents involve both bot and human characteristics. Among different anti-fraud tests, those designed based on domain knowledge are the most effective. By combining individual tests, we can achieve a detection performance as good as commercial techniques while making the results more explainable. To explore these tests under a broader context, we ran a different survey on Amazon Mechanical Turk (MTurk). The results show that for a generic survey without requiring users to have any domain knowledge, it is more difficult to distinguish fraudulent responses. However, a subset of tests still remain effective.