Meta-learning with Latent Space Clustering in Generative Adversarial Network for Speaker Diarization.
Meta-learning with Latent Space Clustering in Generative Adversarial Network for Speaker Diarization.
复制标题
DOI：
10.1109/taslp.2021.3061885
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Narayanan S
中科院分区：
文献类型：
--
作者：
Pal M;Kumar M;Peri R;Park TJ;Kim SH;Lord C;Bishop S;Narayanan S
大多数使用x向量嵌入的说话人日志化系统的性能都容易受到噪声环境的影响，并且缺乏域鲁棒性。早期使用生成对抗网络（GAN）与编码器网络（GAN）将输入x向量投影到潜在空间中的扬声器日志化工作在会议数据上表现出了良好的性能。在本文中，我们扩展了QuerterGAN网络，以提高日志化的鲁棒性，并在各种具有挑战性的领域中实现快速泛化。为此，我们从MeterGAN中获取预训练的编码器，并在元学习范式下使用原型损失（元MeterGAN或MCGAN）对其进行微调。实验在CALLHOME电话会话、AMI会议数据、DIHARD-II（开发集）（包括具有挑战性的多领域语料库）和两个与自闭症谱系障碍领域相关的儿童-临床医生交互语料库（ADOS，BOSCC）上进行。对实验数据进行了广泛的分析，以研究所提出的X-GAN和MCGAN嵌入在x向量上的有效性。结果表明，所提出的嵌入与归一化最大特征间隙谱聚类（NME-SC）后端始终优于Kaldi最先进的x-向量日记系统。最后，我们采用嵌入融合与x向量，以提供进一步的改善，在日记化的性能。我们在上述数据集上使用所提出的x向量融合嵌入实现了6.67%至53.93%的相对日志化错误率（DER）改善。此外，MCGAN嵌入提供了更好的性能，在说话人的数量估计和短的语音段日记相比，x向量和ChesterGAN在电话交谈。
The performance of most speaker diarization systems with x-vector embeddings is both vulnerable to noisy environments and lacks domain robustness. Earlier work on speaker diarization using generative adversarial network (GAN) with an encoder network (ClusterGAN) to project input x-vectors into a latent space has shown promising performance on meeting data. In this paper, we extend the ClusterGAN network to improve diarization robustness and enable rapid generalization across various challenging domains. To this end, we fetch the pre-trained encoder from the ClusterGAN and fine tune it by using prototypical loss (meta-ClusterGAN or MCGAN) under the meta-learning paradigm. Experiments are conducted on CALLHOME telephonic conversations, AMI meeting data, DIHARD-II (dev set) which includes challenging multi-domain corpus, and two child-clinician interaction corpora (ADOS, BOSCC) related to the autism spectrum disorder domain. Extensive analyses of the experimental data are done to investigate the effectiveness of the proposed ClusterGAN and MCGAN embeddings over x-vectors. The results show that the proposed embeddings with normalized maximum eigengap spectral clustering (NME-SC) back-end consistently outperform the Kaldi state-of-the-art x-vector diarization system. Finally, we employ embedding fusion with x-vectors to provide further improvement in diarization performance. We achieve a relative diarization error rate (DER) improvement of 6.67% to 53.93% on the aforementioned datasets using the proposed fused embeddings over x-vectors. Besides, the MCGAN embeddings provide better performance in the number of speakers estimation and short speech segment diarization compared to x-vectors and ClusterGAN on telephonic conversations.
登录
查看更多内容
DOI：
10.1109/taslp.2019.2955293
发表时间：
2020-01-01
影响因子：
5.4
作者：
Diez, Mireia;Burget, Lukas;Cernocky, Jan
通讯作者：
Cernocky, Jan
影响因子：
3.9
作者：
Grzadzinski, Rebecca;Carr, Themba;Lord, Catherine
通讯作者：
Lord, Catherine
DOI：
10.1109/tnnls.2018.2875194
发表时间：
2019-07-01
影响因子：
10.4
作者：
Creswell, Antonia;Bharath, Anil Anthony
通讯作者：
Bharath, Anil Anthony
影响因子：
--
作者：
Chen, Ke;Salman, Ahmad
通讯作者：
Salman, Ahmad
DOI：
10.1109/tasl.2008.2002085
发表时间：
2008-11-01
影响因子：
--
作者：
Han, Kyu J.;Kim, Samuel;Narayanan, Shrikanth S.
通讯作者：
Narayanan, Shrikanth S.