Speech privacy protection by high-quality, invertible, and extendable speech anonymization and de-anonymization

第二年的工作包括三个部分：第一部分)在前一年工作的基础上，我们和其他大学共同组织了第二届语音隐私挑战。我们制定了新的评估框架，开展了扎实的评估。除了许多发现之外，我们还发现，作为前一年的研究成果的新基线的表现优于传统基线。第二部分，基于语音隐私挑战的框架，我们深入分析了产生匿名化说话人身份表征的常见方法(即伪说话人嵌入)。通过大规模实验，我们确定了选择和分配伪说话人的好策略，包括随机性别选择和语音级匿名化。我们还发现，简单的基于百分位数的音调转换降低了对抗最强(半知情)攻击者的风险。这些发现发表在IEEE顶级期刊上。第三部分)我们遵循研究计划，扩展了与语言无关的说话人匿名框架。虽然该框架是独立于语言的，但在处理未知语言时，其性能会下降。我们发现，对波形发生器使用多语言训练数据是有帮助的。我们还提出了一种基于相关性对齐的策略来缓解信道失配。此外，我们还扩展了框架以隐藏性别信息。这两部作品都发表在顶级会议上。

The second year's work consists of three parts: Part 1) Based on the previous year's work, the second VoicePrivacy challenge was organized by us and other universities. We defined new evaluation frameworks and conducted solid evaluations. In addition to many findings, we found that the new baseline, which was the research outcome of the previous year, outperformed the legacy baseline. We also saw submissions that outperformed the new baseline, which indicates the advancement of the research field brought by the VoicePrivacy challenge.Part 2) Based on the framework of the voice privacy challenge, we did a deep analysis of the common approaches to generating anonymized speaker identity representation (i.e., pseudo speaker embedding). Through a large-scale experiment, we identified good strategies to choose and assign the pseudo-speaker, including random gender selection and utterance-level anonymization. We also found that a simple percentile-based pitch conversion reduced the risk against the strongest (Semi-Informed) attacker. These findings were published in a top IEEE journal.Part 3) We followed the research plan and extended the language-independent speaker anonymization framework. Although the framework is language-independent, its performance degrades when processing unseen languages. We found that using multilingual training data for the waveform generator was helpful. We also proposed a correlation-alignment-based strategy to alleviate channel mismatch. Additionally, we extended the framework to hide gender information. Both works were published in top conferences.