Intelligibility Enhancement Via Normal-to-Lombard Speech Conversion With Long Short-Term Memory Network and Bayesian Gaussian Mixture Model
Intelligibility Enhancement Via Normal-to-Lombard Speech Conversion With Long Short-Term Memory Network and Bayesian Gaussian Mixture Model
复制标题
通过使用长短期记忆网络和贝叶斯高斯混合模型的普通到伦巴底语音转换来增强清晰度
DOI：
10.1109/tmm.2021.3068565
复制
发表时间：
2021
影响因子：
7.3
通讯作者：
Ke Shanfa
中科院分区：
文献类型：
--
作者：
Li Gang;Wang Xiaochen;Hu Ruimin;Zhang Huyin;Ke Shanfa
语音通信和交互经常发生在各种环境中。环境中的噪声显著降低了说话和倾听时的语音清晰度。特别是在收听阶段，即使多媒体终端输出干净的语音，收听者仍然很难获得信息。语音的可懂度增强（IENH）是一种在听音阶段克服环境噪声的技术。它实现了非噪声语音的感知增强。本研究的重点是通过正常语音到伦巴第语语音转换的IENH，其灵感来自于一种名为伦巴第效应的众所周知的声学机制。我们的方法结合了长短期记忆（LSTM）网络和贝叶斯高斯混合模型（BGMM）来构建转换架构。与基线相比，它具有三个主要优点：1）利用LSTM网络进行频谱倾斜映射，充分考虑了短期相关性和高维表达能力; 2）利用BGMM将非周期性（AP）与基频（$F_0$）一起映射，考虑了它们的相关性约束和AP的重要性; 3）对F_0和AP采用性别相关映射，考虑性别间的分布差异。实验结果表明，该方法在客观和主观测试中均取得了较好的效果。
Speech communications and interactions frequently occur in a variety of environments. Noise in the environment significantly degrades speech intelligibility when speaking and listening. Especially in the listening stage, even if the multimedia terminal outputs clean speech, it is still difficult for listeners to obtain information. Intelligibility enhancement (IENH) of speech is a technique for overcoming the environmental noise in the listening stage. It implements a perceptual enhancement of non-noisy speech. This study focuses on IENH via normal-to-Lombard speech conversion, inspired by a well known acoustic mechanism named the Lombard effect. Our method combines the long short-term memory (LSTM) network and Bayesian Gaussian mixture model (BGMM) to build a conversion architecture. Compared with baselines, it has three main advantages: 1) an LSTM network is used for spectral tilt mapping with fully considering short-term correlations and high-dimensional expression abilities; 2) the aperiodicity (AP) is mapped together with the fundamental frequency ($F_0$) by a BGMM, which considers their relevance constraints and the importance of APs; 3) the gender-dependent mapping is used for $F_0$ and APs to consider distribution differences between genders. Experiments indicate that our method gets better performance in both objective and subjective tests.