Can Humans Correct Errors From System? Investigating Error Tendencies in Speaker Identification Using Crowdsourcing
Can Humans Correct Errors From System? Investigating Error Tendencies in Speaker Identification Using Crowdsourcing
复制标题
DOI：
10.21437/interspeech.2022-10580
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Yuta Ide;Susumu Saito;Teppei Nakano;Tetsuji Ogawa
中科院分区：
文献类型：
--
作者：
Yuta Ide;Susumu Saito;Teppei Nakano;Tetsuji Ogawa
有人试图澄清众包在减少自动说话人辨认错误方面的有效性。可以通过手动重新验证ASID系统给出的不可靠结果来有效地减少错误。理想情况下，错误应该得到适当的纠正，正确的答案不应该被错误纠正。此外，在认证中期望低的错误接受率，但是从可用性的观点来看，应该避免高的错误拒绝率。然而，并不确定人类是否可以实现这样一个理想的SID，并且在众包的情况下，恶意工作者的存在不容忽视。因此，本研究探讨人工验证容易出错的人群工作者的输入是否可以减少ASID错误，以及由此产生的纠正是否是理想的。在Amazon Mechanical Turk上进行的实验调查中，426名合格的工作人员从VoxCeleb数据中识别了256个语音对，结果表明，与ASID系统的结果相比，众包验证可以显着减少错误接受的数量，而不会增加错误拒绝的数量。
An attempt was made to clarify the effectiveness of crowd-sourcing on reducing errors in automatic speaker identification (ASID). It is possible to efficiently reduce errors by manually revalidating the unreliable results given by ASID systems. Ideally, errors should be corrected appropriately, and correct answers should not be miscorrected. In addition, a low false acceptance rate is desirable in authentication, but a high false rejection rate should be avoided from a usability viewpoint. It, however, is not certain that humans can achieve such an ideal SID, and in the case of crowdsourcing, the existence of malicious workers cannot be ignored. This study, therefore, investigates whether manual verification of error-prone inputs by crowd workers can reduce ASID errors and whether the resulting corrections are ideal. Experimental investigations on Amazon Mechanical Turk, in which 426 qualified workers identified 256 speech pairs from VoxCeleb data, demonstrated that crowd-sourced verification can significantly reduce the number of false acceptances without increasing the number of false rejections compared to the results from the ASID system.