A multi-modal sensor fusion architecture for audio-visual speech understanding

用于视听语音理解的多模态传感器融合架构

• 批准号: 184129-2007
• 负责人: Basir, Otman
• 金额: $ 1.97万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2010
• 资助国家: 加拿大
• 起止时间: 2010-01-01 至 2011-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=452534
• 关键词: multi; modal; sensor; fusion; architecture

Modern speech recognition systems remain challenged in noisy environments despite all the technical advances that have been made to enhance their capabilities. There has been significant research work to address the effect of noise and word ambiguities on recognition performance. For speech recognition systems to be of practical use in noisy environments, such as in automobile, crowded areas and/or simultaneous human computer discourse applications, the issue of robustness must be addressed. The applicant maintains that this can possibly be accomplished by utilizing other sensing modalities to complement the acoustic signal of the speech. An example in line with this strategy is to fuse visual lip movements and expressions with the acoustic signal of the speech so as to maximize information gathering about the words uttered and to minimize the impact of acoustic noise.

现代语音识别系统仍然面临着挑战，在嘈杂的环境中，尽管所有的技术进步，已经取得了提高他们的能力。已经有大量的研究工作来解决噪声和单词歧义对识别性能的影响。对于语音识别系统在嘈杂环境中的实际应用，例如在汽车，拥挤的区域和/或同时的人机对话应用中，必须解决鲁棒性的问题。申请人坚持认为，这可以通过利用其他感测模态来补充语音的声学信号来实现。符合这种策略的一个例子是将视觉嘴唇运动和表达与语音的声学信号融合，以便最大限度地收集有关所说出的单词的信息，并最小化声学噪声的影响。