Person-specific automatic speaker recognition: understanding the behaviour of individual speakers for applications of ASR

自动说话人识别（ASR）软件对语音进行处理和分析，以判断两个声音是属于同一个人还是不同的人。这种技术正在成为我们生活中越来越重要的一部分；用于访问个人账户（如银行）时的安全措施，或作为在智能设备上为特定人员定制内容的手段。在世界各地，ASR系统通常用于调查和法医目的，以分析身份不明的犯罪声音记录。然而，系统在处理某些声音时表现得或好或坏。因此，一个基本问题仍然存在：是什么让ASR容易或难以识别特定的声音？与旧方法相比，使用人工智能（AI）技术的最先进系统在性能上有了显著提高。然而，仍然存在一些问题。首先，ASR研究的重点是尽量减少众所周知的技术因素的影响，例如渠道（例如移动电话与固定电话）、录音质量和麦克风。在解决这些技术挑战的过程中，系统已经取得了很大的改进。然而，人们对说话者自身如何影响ASR表现知之甚少。其次，ASR研究一直致力于降低总体错误率。然而，在现实世界中（无辜和有罪可能处于危险之中），关键问题是：系统在这个特定实例中犯错误的几率有多大？最后，虽然人工智能方法无疑带来了整体性能的提高，但这种算法使得了解信息系统依赖于哪些信息来做出决策变得更加困难。这对法医专家来说是个问题，他们必须向法官、陪审团、律师和警察等非专家最终用户解释他们的方法。该项目是第一个系统地评估个体说话者在ASR系统内和跨系统的表现，并将说话者的效果（根据声音的语言特性或说话者的人口统计数据（例如口音、种族、性别））与经过充分研究的技术效果进行比较的项目。目的是利用这些知识，通过标记潜在的问题说话者来改进ASR系统，并开发处理这些问题说话者的方法。我们将使用新颖的跨学科方法，汇集语音技术，语言学和法医语音科学的专业知识。我们与商业ASR供应商Oxford Wave Research的合作使我们能够适应和改变系统，以评估对单个扬声器结果的影响。我们还将使用高度控制的小规模实验来单独评估说话者的效果，以及使用由我们的项目合作伙伴英国国防部和荷兰法医研究所提供的更大的更具法医真实性的录音数据集。各种数据集的可用性也使我们能够评估结果在一系列声音中的普遍性。该项目完全由现实问题驱动，因此其结果将对广泛的利益相关者产生相当大的影响。通过更多地了解个体，我们的研究结果有能力提高整体的ASR表现。这将有利于ASR系统的用户和开发人员。研究结果还将对法医和调查应用产生具体影响，指导验证方法的数据收集（专家们正面临越来越大的监管压力），并提供一个将ASR和语言分析结合起来的框架。在这样做的过程中，通过与法律界的合作，我们的目标是影响英格兰和威尔士ASR地位的变化，使其成为可接受的专家证据。我们将通过与由法医语音科学、执法和法律界代表组成的法医咨询小组进行知识交流来发挥作用。

Automatic speaker recognition (ASR) software processes and analyses speech to make decisions about whether two voices belong to the same or different individuals. Such technology is becoming an increasingly important part of our lives; used as a security measure when gaining access to personal accounts (e.g. banks), or as a means of tailoring content to a specific person on smart devices. Around the world, ASR systems are commonly used for investigative and forensic purposes, to analyse recordings of criminal voices where identity is unknown. Yet systems perform better or worse with certain voices. Therefore, a fundamental question remains: what makes a particular voice easy or difficult for ASR to recognise?State-of-the-art systems, using techniques from artificial intelligence (AI), have shown marked improvements in performance compared with older approaches. However, there remain issues. Firstly, ASR research has focused on minimising the effects of well-known technical factors, such as channel (e.g. mobile vs. landline telephone), recording quality and microphones. In resolving these technical challenges, large improvements in systems have been achieved. Yet little is known about how speakers themselves affect ASR performance. Secondly, ASR research has been interested in reducing overall error rates. Yet, in the real-world (where innocence and guilt may be at stake), the key question is: what is the chance the system has made an error in this specific instance? Finally, while AI approaches have undoubtedly brought improvements in overall performance, such algorithms make it more difficult to know what information systems are relying on to make decisions. This is problematic for forensic experts, who must explain their methods to non-expert end users, such as judges, juries, lawyers and police.This project is the first to systematically assess how individual speakers perform within and across ASR systems and to compare speaker effects, in terms of linguistic properties of voices or speaker demographics (e.g. accent, ethnicity, gender), with well-studied technical effects. The aim is to use this knowledge to improve ASR systems by flagging potentially problematic speakers and to develop methods to handle these problematic speakers. We will use novel, interdisciplinary methods, bringing together expertise from speech technology, linguistics, and forensic speech science. Our collaboration with commercial ASR vendor Oxford Wave Research allows us to adapt and change systems to assess the effects on results for individual speakers. We will also use highly controlled, small-scale experiments to assess speaker effects in isolation, as well as using much larger datasets of more forensically realistic recordings, provided by our project partners, the UK Ministry of Defence and the Netherlands Forensic Institute. The availability of a variety of datasets also allows us to assess the generalisability of results across a range of voices. This project is entirely driven by real-world issues and so the results will deliver considerable impact to a wide range of stakeholders. By understanding more about individuals, our results have the capability to improve overall ASR performance. This will be of benefit to users and developers of ASR systems. The results will also have specific implications for forensic and investigative applications, guiding data collection for validating methods (something which experts are under increasing regulatory pressure to do) and provide a framework for combining ASR and linguistic analysis. In doing so, through engagement with the legal community, we aim to affect a change in the status of ASR in England and Wales, such that it is admissible as expert evidence. We will deliver impact via knowledge exchange with a Forensic Advisory Panel consisting of representatives from forensic speech science, law enforcement, and the legal community.