On Prosody Modeling for ASR+TTS Based Voice Conversion
On Prosody Modeling for ASR+TTS Based Voice Conversion
复制标题
DOI：
10.1109/asru51503.2021.9688010
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Wen-Chin Huang;Tomoki Hayashi;Xinjian Li;Shinji Watanabe;T. Toda
中科院分区：
文献类型：
--
作者：
Wen-Chin Huang;Tomoki Hayashi;Xinjian Li;Shinji Watanabe;T. Toda
在语音转换（VC）方面，在最新的语音转换挑战赛（VCC 2020）中显示出良好结果的方法是首先使用自动语音识别（ASR）模型将源语音转录为底层语言内容；然后，这些数据被文本转语音 (TTS) 系统用作输入，以生成转换后的语音。这种被称为 ASR+TTS 的范式忽略了韵律的建模，而韵律在语音自然度和转换相似度方面起着重要作用。尽管一些研究人员考虑过从源语音中转移韵律线索，但在训练和转换过程中会出现说话人不匹配的情况。为了解决这个问题，在这项工作中，我们建议以目标说话人相关的方式从语言表示直接预测韵律，称为目标文本预测（TTP）。我们在 VCC2020 基准上评估这两种方法并考虑不同的语言表示。结果证明了 TTP 在客观和主观评估方面的有效性。
In voice conversion (VC), an approach showing promising results in the latest voice conversion challenge (VCC) 2020 is to first use an automatic speech recognition (ASR) model to transcribe the source speech into the underlying linguistic contents; these are then used as input by a text-to-speech (TTS) system to generate the converted speech. Such a paradigm, referred to as ASR+TTS, overlooks the modeling of prosody, which plays an important role in speech naturalness and conversion similarity. Although some researchers have considered transferring prosodic clues from the source speech, there arises a speaker mismatch during training and conversion. To address this issue, in this work, we propose to directly predict prosody from the linguistic representation in a target-speaker-dependent manner, referred to as target text prediction (TTP). We evaluate both methods on the VCC2020 benchmark and consider different linguistic representations. The results demonstrate the effectiveness of TTP in both objective and subjective evaluations.