Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
复制标题
DOI：
10.1109/taslp.2019.2915167
复制
发表时间：
2019-08-01
影响因子：
5.4
通讯作者：
Mesgarani, Nima
中科院分区：
文献类型：
--
作者：
Luo, Yi;Mesgarani, Nima
单渠道，独立的语音分离方法最近取得了巨大进展。但是，此类方法的准确性，延迟和计算成本仍然不足。以前的大多数方法通过混合信号的时频表示提出了分离问题，这有几个缺点，包括信号的相位和幅度的解耦，语音分离的时间频率表示的次频，以及计算频谱图的长延迟。为了解决这些缺点，我们提出了一个完全卷积的时间域音频分离网络（Conv-Tasnet），这是一个深度学习框架，用于端到端时间域语音分离。 Conv-Tasnet使用线性编码器来生成针对分离单个扬声器进行优化的语音波形的表示。扬声器分离是通过将一组加权功能（蒙版）应用于编码器输出来实现的。然后，使用线性解码器将修改的编码器表示倒回波形。使用的掩模使用时间卷积网络，该网络由堆叠的一维扩张卷积块组成，该网络使网络能够对语音信号的长期依赖性进行建模，同时保持较小的模型大小。所提出的Conv-TASNET系统在分离两扬声器和三扬声器混合物中的先前时间频率掩盖方法明显优于先前的时频掩蔽方法。此外，Conv-Tasnet超过了两种理想的时频幅度掩码，以两种扬声器的语音分离，可以通过人类听众的客观失真度量和主观质量评估来评估。最后，Conv-Tasnet的模型大小明显较小，最小延迟较短，这使其成为离线和实时语音分离应用程序的合适解决方案。因此，这项研究代表了实现真实世界语音处理技术的语音分离系统的重要一步。
Single-channel, speaker-independent speech separation methods have recently seen great progress. However, the accuracy, latency, and computational cost of such methods remain insufficient. The majority of the previous methods have formulated the separation problem through the time-frequency representation of the mixed signal, which has several drawbacks, including the decoupling of the phase and magnitude of the signal, the suboptimality of time-frequency representation for speech separation, and the long latency in calculating the spectrograms. To address these shortcomings, we propose a fully convolutional time-domain audio separation network (Conv-TasNet), a deep learning framework for end-to-end time-domain speech separation. Conv-TasNet uses a linear encoder to generate a representation of the speech waveform optimized for separating individual speakers. Speaker separation is achieved by applying a set of weighting functions (masks) to the encoder output. The modified encoder representations are then inverted back to the waveforms using a linear decoder. The masks are found using a temporal convolutional network consisting of stacked one-dimensional dilated convolutional blocks, which allows the network to model the long-term dependencies of the speech signal while maintaining a small model size. The proposed Conv-TasNet system significantly outperforms previous time-frequency masking methods in separating two- and three-speaker mixtures. Additionally, Conv-TasNet surpasses several ideal time-frequency magnitude masks in two-speaker speech separation as evaluated by both objective distortion measures and subjective quality assessment by human listeners. Finally, Conv-TasNet has a significantly smaller model size and a shorter minimum latency, making it a suitable solution for both offline and real-time speech separation applications. This study, therefore, represents a major step toward the realization of speech separation systems for real-world speech processing technologies.