Consistency Regularization for GAN-based Neural Vocoders
Consistency Regularization for GAN-based Neural Vocoders
复制标题
DOI：
10.23919/apsipaasc55919.2022.9980309
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Kotaro Onishi;Toru Nakashika
中科院分区：
文献类型：
--
作者：
Kotaro Onishi;Toru Nakashika
基于生成对抗网络（GAN）的神经声码器对于实现高质量和快速推理速度非常有用。然而，它们通常比自回归模型的神经声码器质量更低或需要更长的学习时间。在这项工作中，我们提出了一种通过数据增强的一致性正则化来改进基于GAN的神经声码器的方法。具体来说，我们将数据增强应用于自然语音，并通过这种增强来惩罚语音输出的差异。一致性正则化在半监督学习和图像生成GAN中表现出良好的效果。我们研究了用于一致性正则化的语音波形的各种数据增强，包括噪声添加和有损压缩。由于所提出的方法只对卷积增加了正则化，因此它可以应用于各种基于GAN的神经声码器。客观评估实验的结果表明，MelGAN与我们的方法优于基线。此外，主观评价实验的结果表明，MelGAN的自然度的平均意见分数从3.23提高到3.85。
Neural vocoders based on generative adversarial networks (GANs) are very useful for enabling both high quality and fast inference speed. However, they are typically lower quality or take longer to learn than the neural vocoders of autoregressive models. In this work, we propose a method for improving GAN-based neural vocoders by means of consistency regularization with data augmentation. Specifically, we apply data augmentation to a natural speech and penalize the difference of the discriminator output with this augmentations. Consistency regularization demonstrated good results in semi-supervised learning and image-generation GANs. We investigated various data augmentations for speech waveforms used in the consistency regularization, including noise addition and lossy compression. Since the proposed method only adds a regularization to the discriminator, it can be applied to a wide range of GAN-based neural vocoders. The results of an objective evaluation experiment showed that MelGAN with our method outperformed the baseline. Moreover, the results of a subjective evaluation experiment showed that the mean opinion score for naturalness was improved from 3.23 to 3.85 for MelGAN.