Adapting and controlling DNN-based speech synthesis using input codes
Adapting and controlling DNN-based speech synthesis using input codes
复制标题
DOI：
10.1109/icassp.2017.7953089
复制
发表时间：
2017-06
期刊：
影响因子：
--
通讯作者：
Hieu-Thi Luong;Shinji Takaki;G. Henter;J. Yamagishi
中科院分区：
文献类型：
--
作者：
Hieu-Thi Luong;Shinji Takaki;G. Henter;J. Yamagishi
适应和控制输出语音的特性的方法是语音合成中的重要主题。在这项工作中，我们研究了基于 DNN 的文本转语音系统的性能，该系统与传统文本输入并行，也将说话人、性别和年龄代码作为输入，以便 1）执行多说话人合成，2）使用少量目标说话人适应数据执行说话人适应，以及 3）根据输入代码修改合成语音特征。我们使用包含 135 名不同性别、年龄在 0 到 80 岁之间的说话者的大规模录音室质量语音语料库，进行了三个实验：1）首先，我们使用说话者子集构建一个基于 DNN 的、带有说话者代码的多说话者声学模型。 2）接下来，我们通过少量适应材料的反向传播估计新说话人的码向量来执行说话人适应。 3）最后，我们尝试手动操作输入代码向量来改变合成语音的性别和/或年龄特征。实验结果表明，可以使用所提出的具有多种编码方案的码向量来构建高性能多说话者模型，并且可以使用这些码有效地执行自适应和操作。
Methods for adapting and controlling the characteristics of output speech are important topics in speech synthesis. In this work, we investigated the performance of DNN-based text-to-speech systems that in parallel to conventional text input also take speaker, gender, and age codes as inputs, in order to 1) perform multi-speaker synthesis, 2) perform speaker adaptation using small amounts of target-speaker adaptation data, and 3) modify synthetic speech characteristics based on the input codes. Using a large-scale, studio-quality speech corpus with 135 speakers of both genders and ages between tens and eighties, we performed three experiments: 1) First, we used a subset of speakers to construct a DNN-based, multi-speaker acoustic model with speaker codes. 2) Next, we performed speaker adaptation by estimating code vectors for new speakers via backpropagation from a small amount of adaptation material. 3) Finally, we experimented with manually manipulating input code vectors to alter the gender and/or age characteristics of the synthesised speech. Experimental results show that high-performance multi-speaker models can be constructed using the proposed code vectors with a variety of encoding schemes, and that adaptation and manipulation can be performed effectively using the codes.