Iterative Information Fusion in Automatis Speech Recognition According to the Turbo Principle

根据Turbo原理的自动语音识别中的迭代信息融合

• 批准号: 414091002
• 负责人: Professor Dr.-Ing. Tim Fingscheidt
• 金额: --
• 依托单位: Institut für Nachrichtentechnik
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2019
• 资助国家: 德国
• 起止时间: 2018-12-31 至 2022-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/414091002?language=en
• 关键词: Iterative; Information; Fusion; Automatis; Speech

The intelligent fusion of information plays a major role in two opposing megatrends of information technology: (1) Decentralization (internet, internet of things, decentralized network control, sensor networks, industry 4.0, ...), and since recently in the field of automatic speech recognition also (2) centralization (Siri, Google Home, Amazon Alexa, YouTube). Both trends have in common that multiple information sources are being used: It may be multimodal approaches (e.g., audiovisual speech recognition: microphone, camera), or uni-modal (speech recognition only with microphone signals). The uni-modal approach may operate multi-channel or single-channel, in the latter case using, e.g., information of two different feature representations.In prior works of the applicant the turbo principle known from Digital Communications for iterative fusion of information has been successfully transferred to automatic speech recognition (ASR). One objective of this project is to further explore the still widely uncovered potential of turbo information fusion in the field of automatic speech recognition. It is not only capable of fusing feature representations very well, but can also perform fusion of acoustic models. Since modelling in ASR is meanwhile performed with deep neural networks, and a variety of network model topologies are subject to research nowadays, fusion is a hot topic, but high-performance fusion approaches rarely come with modularity. However, since modularity in information fusion in both trends (1) and (2) is almost indispensable, in this project turbo information fusion shall be further developed to become completely modular, thereby proving high flexibility and relevance for a wide range of applications.A further objective is to acquire a deeper knowledge of the iteratively operating turbo information fusion. Why is it performing so well? And how about the relation between its performance and statistical dependence of the information sources? Controlled experiments with synthetic data allowing perfect modelling shall provide answers. Also the both useful and theoretically demanding so-called EXIT charts known from Digital Communications shall be developed further with the ultimate goal to be able to predict the performance of turbo information fusion. Even more, using the EXIT analysis tool, it shall become possible that the fusion can be designed in a way such that after a few iterations indeed a high quality recognition result is obtained.Finally, we plan to explore ASR with turbo information fusion with more than two information sources or recognizers, respectively. Besides the fusion of a couple of complementary models the scenario of spatially distributed microphones and ASR systems is of interest: Is turbo information fusion capable of obtaining a performance gain from spatially distributed microphones in, e.g., a reverberant environment?

信息的智能融合在信息技术的两个相反的大趋势中发挥着重要作用：(1) 去中心化（互联网、物联网、去中心化网络控制、传感器网络、工业 4.0 等），以及最近在自动语音识别领域的发展（2）集中化（Siri、Google Home、Amazon Alexa、YouTube）。这两种趋势的共同点是使用多个信息源：可能是多模态方法（例如视听语音识别：麦克风、摄像头），也可能是单模态方法（仅使用麦克风信号进行语音识别）。单模方法可以操作多通道或单通道，在后一种情况下使用例如两种不同特征表示的信息。 在申请人的先前工作中，从数字通信已知的用于信息迭代融合的turbo原理已经成功地转移到自动语音识别(ASR)。该项目的目标之一是进一步探索涡轮信息融合在自动语音识别领域中尚未广泛发现的潜力。它不仅能够很好地融合特征表示，而且还可以进行声学模型的融合。由于ASR中的建模同时使用深度神经网络进行，并且当今有多种网络模型拓扑正在研究，融合是一个热门话题，但高性能融合方法很少具有模块化。然而，由于趋势（1）和（2）中信息融合的模块化几乎是不可或缺的，因此在本项目中，Turbo信息融合应进一步发展为完全模块化，从而为广泛的应用提供高度的灵活性和相关性。进一步的目标是获得对迭代操作Turbo信息融合的更深入的了解。为什么它的表现这么好？其性能与信息源的统计依赖性之间的关系如何？使用允许完美建模的合成数据进行的对照实验将提供答案。此外，数字通信中已知的有用且理论上要求较高的所谓退出图也应进一步开发，最终目标是能够预测涡轮信息融合的性能。更重要的是，使用 EXIT 分析工具，可以将融合设计为经过几次迭代后确实获得高质量的识别结果。最后，我们计划分别探索具有两个以上信息源或识别器的 Turbo 信息融合的 ASR。除了几个互补模型的融合之外，空间分布式麦克风和 ASR 系统的场景也很有趣：Turbo 信息融合是否能够在混响环境等情况下从空间分布式麦克风获得性能增益？