Automatic Transcription of Conversations

多说话者对话语音识别涉及到使用远程麦克风将正式会议或非正式聚会的录音转录成机器可读的形式。目前的解决方案远远达不到人类的表现。这项任务的困难可以归结为三个因素。首先，录音条件是具有挑战性的：麦克风从远处捕获的语音信号是嘈杂和混响的，并且通常包含非平稳的声学失真，这使得它很难解码。其次，有很大比例的时间是重叠的，即多个说话者同时说话。最后，该场景的交互动态是具有挑战性的，因为说话者以断断续续的方式表达自己，并交替出现不活动、单个和多个说话者的讲话。我们的目标是开发一种转录系统，能够在任意长度的输入上操作，正确处理重叠和非重叠的语音片段，并将不同说话者的语音一致地转录到单独的输出流中。虽然现有的方法使用单独训练的子系统进行分类、分离和识别，到目前为止还无法达到人类的表现，但我们认为，缺失的部分是一个公式，它封装了满足转录的所有方面，并允许在单一优化标准下设计一个联合方法。这个项目的目标就是这样一个连贯的方案。我们将开发一个集成的解决方案，用于使用远距离麦克风的多说话者会话语音识别，其中一次活动的说话者数量和说话者重叠的程度是未知的和随时间变化的。将在一个共同的目标函数下处理记录、增强和识别的任务，最终接近端到端的训练。此外，我们打算避免过早的决定，以实现端到端识别的目标。我们将推导并考虑不同的架构（全神经的、混合的、级联的、集成的），并评估它们的编码和转录性能，以及中间结果的可解释性和相应配方的易用性。

Multi-talker conversational speech recognition is concerned with the transcription of audio recordings of formal meetings or informal get-togethers in machine-readable form using distant microphones. Current solutions are far from reaching human performance. The difficulty of the task can be attributed to three factors. First, the recording conditions are challenging: The speech signal captured by microphones from a distance is noisy and reverberated and often contains nonstationary acoustic distortions, which makes it hard to decode. Second, there is a significant percentage of time with overlapped speech, where multiple speakers talk at the same time. Finally, the interaction dynamics of the scenario are challenging because speakers articulate themselves in an intermittent manner with alternating segments of speech inactivity, single-, and multi-talker speech. We aim to develop a transcription system that is able to operate on arbitrary length input, correctly handles segments of overlapped as well as non-overlapped speech, and transcribes the speech of different speakers consistently into separate output streams. While existing approaches using separately trained subsystems for diarization, separation, and recognition are by far not able to reach human performance, we believe that the missing piece is a formulation which encapsulates all aspects of meeting transcription and which allows to design a joint approach under a single optimization criterion. This project is aimed at such a coherent formulation. We are going to develop an integrated solution for multi-talker conversational speech recognition using distant microphones, where the number of speakers who are active at a time and the extent of speaker overlap is unknown and time-varying. The tasks of diarization, enhancement and recognition will be addressed under a common objective function, ultimately approaching end-to-end training. Complementarily, we intend to avoid premature decisions with the goal of end-to-end recognition. We will derive and consider different architectures (all-neural, hybrid, cascaded, integrated), and assess them w.r.t. diarization and transcription performance, as well as w.r.t. the interpretability of intermediate results and the ease of use of the corresponding recipes.