Learning representations of players' emotions and state for next generation gaming

点击翻译按钮获取中文摘要

Computer and console gaming platforms are increasingly equiped with sensors, such as cameras and depth sensors, that can watch and track players while they are playing a game. It however remains a scientific and technical challenge to analyse this data in real time, beyond mere parts of body tracking, to reliably recognize and track a user's precise emotional state during gameplay. With our Montreal-based partner Ubisoft, we propose to leverage recent advances in machine learning techniques to achieve this. Specifically we will employ recent approaches for training deep neural networks on big datasets, to learn a mapping between a player's captured images (especially the face) and emotional state. The research will focus on improving these techniques to allow fast and transparent adaptation to each player's specific modes of facial expression. The Canadian computer gaming industry would gain a major competitive advantage thanks to such technology, since the ability of a gaming system to accurately perceive a player's emotional state in real time will enable the game to adapt to it, and is anticipated to be the key for crafting the next generation of truly immersive gaming experience.