Parameterized Control of Voicing & Accent in Speech Generated by Deep Networks

Deep Mind最近开发了一种卷积网络(WaveNet)，该网络可用于从文本生成语音(Oord et.Al.，2016)。这一新模型显示，与传统系统相比，感知的音频质量有了显著改善。后来，引入了并行化版本模型，具有比实时生成更快的速度(Oord et.Al.，2017)。在这两篇论文中，主要关注的是从文本生成的语音质量，但作者简要地实验了修改生成语音的发音。他们通过将说话人ID作为参数引入模型来做到这一点；根据来自几个不同说话者的录音来训练网络，这允许从同一模型产生多个离散的声音。然而，我相信通过更高级的发声模型，将有可能开发出一种系统，允许用户连续改变发声，从而对声音进行更好的控制。一个足够先进的系统还可能允许对声音的重音和韵律进行建模和参数化。我还想探讨一下是否有可能将这种“发声”控制模型从TTS模型中分离出来。这可能会使语音到语音转换网络的建立成为可能，该网络可以操纵现有音频记录的发声。

Deep Mind have recently developed a convolutional network (WaveNet) which can be used, among other things, for generating speech from text (Oord et. al., 2016). This new model shows a significant improvement in perceived audio quality compared to conventional systems. Later, a parallelised version model is introduced, with faster-than-real-time generation (Oord et. al., 2017). In both papers, the main focus is on the quality of speech generated from text, but the authors briefly experiment with modifying the voicing of generated speech. They do this by introducing a speaker ID as a parameter to the model; training the network on recordings from several different speakers, this allows multiple discrete 'voices' to be generated from the same model.However, I believe that with a more advanced model of voicing it would be possible to develop a system which allows users to alter the voicing continuously, giving greater control over the sound. A sufficiently advanced system could potentially also allow modelling and parameterization of accent and prosody of voice. I also wish to explore whether it is possible to decouple this 'voicing' control model from the TTS model. This could potentially enable the building of a speech-to-speech transformation network, which can manipulate the voicing of existing audio recordings.