Parameterized Control of Voicing & Accent in Speech Generated by Deep Networks

Deep Mind最近开发了一个卷积网络（WaveNet），除其他外，可用于从文本生成语音（Oord et. al., 2016）。与传统系统相比，这种新模型在感知音频质量方面有了显著改善。随后，引入了一个并行版本模型，其生成速度快于实时（Oord et. al., 2017）。在这两篇论文中，主要关注的是文本生成语音的质量，但作者简要地尝试了修改生成语音的声音。他们通过将扬声器ID作为模型的参数来实现这一点；在几个不同的说话者的录音上训练网络，这允许从同一个模型生成多个离散的“声音”。然而，我相信，有了更先进的发声模型，就有可能开发出一种允许用户不断改变发声的系统，从而更好地控制声音。一个足够先进的系统也可能允许建模和参数化的口音和韵律的声音。我还希望探讨是否有可能将这种“语音”控制模型与TTS模型解耦。这可能使建立一个语音到语音转换网络成为可能，它可以操纵现有录音的声音。

Deep Mind have recently developed a convolutional network (WaveNet) which can be used, among other things, for generating speech from text (Oord et. al., 2016). This new model shows a significant improvement in perceived audio quality compared to conventional systems. Later, a parallelised version model is introduced, with faster-than-real-time generation (Oord et. al., 2017). In both papers, the main focus is on the quality of speech generated from text, but the authors briefly experiment with modifying the voicing of generated speech. They do this by introducing a speaker ID as a parameter to the model; training the network on recordings from several different speakers, this allows multiple discrete 'voices' to be generated from the same model.However, I believe that with a more advanced model of voicing it would be possible to develop a system which allows users to alter the voicing continuously, giving greater control over the sound. A sufficiently advanced system could potentially also allow modelling and parameterization of accent and prosody of voice. I also wish to explore whether it is possible to decouple this 'voicing' control model from the TTS model. This could potentially enable the building of a speech-to-speech transformation network, which can manipulate the voicing of existing audio recordings.