Intelligent Multimedia Entertainment: Online monitoring of affective and cognitive states

点击翻译按钮获取中文摘要

The entertainment industry is faced with major changes such as the diversification of electronic platforms as well as the rapid increase of people's consumption of games and digital media, and their fluctuating interests and moods. To address these challenges, media entertainment industries must now develop next-generation adaptive systems; that is, in order to tailor content and provide the most positive user-centered experience, they must include online monitoring of entertainees' cognitive and affective state based on biometric and behavioral markers. Key innovations are needed in terms of identifying reliable markers and their integration within the interactivity of the media. Our approach combines experimental psychology and cognitive ergonomics to investigate physiological and behavioural metrics as online markers of workload, information seeking patterns, engagement, and mood of the entertainee interacting with the Liveshout social platform. First, the proposed project will allow the initial development of an entertainee's functional state (EFS) based on the integration of electrocardiography (e.g., heart rate), eye-tracking (e.g., fixation duration and pupil dilation) and face-reading (e.g., joy and sadness) features. Second, the project aims to contribute to the development of IAS-based interactive entertainment through the conceptualization of an architecture that enables the real-time adaptation of the Liveshout platform content to the EFS. Results of this proposed project will define guidelines and recommendations for designing adaptive and interactive social media, and will positively impact the Canadian technological, financial and education sectors. The outcomes of the project will also have beneficial impacts on other areas such as intelligent tutoring, transportation and public security.