RI: Extension of the APP detector for multipitch tracking and speaker separation

在许多现实场景中，语音识别和说话人识别系统必须处理来自多个说话人的同时语音，即自然环境中代表对话的语音混合。由于精细颞叶结构的丧失，人工耳蜗使用者在多扬声器环境中会遇到分离扬声器的问题。因此，这种系统的一个关键预处理步骤是根据语音的组成来源对语音进行分离。这个项目是这个过程的第一部分，它涉及到识别说话者的数量，并根据语音信号的周期性部分（即浊音区域）分离他们的音高轨道。由于不同的说话者具有声带生理学的特征音高范围，音高轨道可以用来帮助将组合信号分离成不同的语音流。当前流行的多音高跟踪方法容易受到不同语音信号周期区域之间相互作用所引起的伪影的影响。因此，组合信号的周期性可能不同于单个分量的周期性。主要的新思想是将现有的周期和基音估计过程扩展到更高的维度，得到一个不受谐波相互作用伪像影响的多维周期函数。初步结果表明，即使在一个说话者比另一个说话者更占优势的情况下，所获得的多重音轨也是准确的。该方法易于推广到非语音音频，并且在噪声信道中具有较强的鲁棒性。这个项目的结果将在未来的项目中使用，在这个项目中，实际的语音流将基于多音高信息彼此分离。

In many real world scenarios, speech recognition and speaker identification systems must deal with simultaneous speech from several talkers, i.e., speech mixtures representing conversations in natural environments. Users of cochlear implants encounter problems in separating speakers in multi-speaker environments, because of the loss of fine temporal structure. Thus, a crucial preprocessing step for such systems is the segregation of speech according to its constituent sources. The project is the first part of this process which involves the recognition of the number of speakers and the separation of their pitch tracks based on the periodic portions of the speech signal (i.e., voiced regions). Since different speakers have characteristic pitch ranges as a consequence of vocal cord physiology, pitch tracks can be used to help separate the combined signal into different speech streams. Current popular multi-pitch tracking approaches are susceptible to artifacts caused by the interaction between the periodic regions of the different speech signals. Consequently, the periodicity of the combined signal can be different from that of the individual components. The major new idea is the extension of an existent periodicity and pitch estimation process to higher dimensions, arriving at a multi-dimensional periodicity function which is not susceptible to the harmonic interaction artifacts. Preliminary results show that the multiple pitch tracks obtained are accurate even when one speaker is considerably more dominant than the other speaker. The approach is easily generalized to non-speech audio and it should be robust in noisy channels. The outcome of this project will be used in a future project where the actual speech streams will be separated from each other based on the multi-pitch information.