Fusion of Face and Speech Characteristics for Real-Time Recognition of Behaviors in Human-Centric E-Learning

融合面部和语音特征，实时识别以人为本的电子学习中的行为

• 批准号: 485329-2015
• 负责人: Granger, Eric
• 金额: $ 1.82万
• 依托单位: École de technologie supérieure
• 依托单位国家: 加拿大
• 项目类别: Engage Grants Program
• 财政年份: 2015
• 资助国家: 加拿大
• 起止时间: 2015-01-01 至 2016-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=584872
• 关键词: Fusion; Face; Speech; Characteristics; Real

Over the past few decades, several new technologies have been adopted to enhance online e-learning programs. NVS Learning Inc. seeks to deliver interactive training programs based on virtual and immersive environments that develop positive behavioral skills, as needed in effective leadership, management, customer service, sales, etc. Although still underexploited, recent developments webcam and microphone technologies, as well as signal processing and pattern recognition techniques would allow for more natural and unobtrusive interactions with such environments. However, commercial systems that exploit these technologies are limited to offline post-analysis of face and speech information and recognition of a small set of expressions. Real-time recognition of behavioral skills based on faces and speech captured in videos is a challenging large-scale e-learning application. This project investigates state-of-the-art techniques that are suitable for the real-time recognition of behavioral skills in e-learning based on face and speech modalities. Given a video stream that captures the variations in the learner's face and speech during interactions with an e-learning environment, an automated system is required to learn and interpret his/her behavioral skills, e.g., the flow of the learner's speech verbalizations, and provide the learner with personalised feedback. Several advanced signal processing and pattern techniques will be reviewed and compared. In particular, patch-based sparse representation, multiple instance learning, and ensemble learning techniques will be considered to design a pool of behavior detectors. This project aims to provide NVS Learning with the expertise required to deploy robust interactive e-learning systems for customer applications. Benchmarking of promising techniques will be performed with videos from the RECOLA public dataset, and performance will be assessed in terms of accuracy and resource requirements.

在过去的几十年里，已经采用了几种新技术来增强在线电子学习计划。NVS Learning Inc.致力于提供基于虚拟和沉浸式环境的互动培训课程，培养有效领导、管理、客户服务、销售等方面所需的积极行为技能。尽管仍未充分开发，但最近开发的网络摄像头和麦克风技术，以及信号处理和模式识别技术，将允许与此类环境进行更自然、更低调的交互。然而，利用这些技术的商业系统仅限于对面部和语音信息的离线后分析以及一小部分表情的识别。基于视频中捕获的面部和语音的行为技能的实时识别是一项具有挑战性的大规模电子学习应用。