EAR: Eye-based Activity Recognition

点击翻译按钮获取中文摘要

Activity recognition is concerned with inference of human activity from observation of the user's actions, to facilitate systems that proactively support users in everyday activity. Application areas include health, safety, cognitive assistance, lifestyle monitoring, entertainment, and energy management. In this project, we propose to advance activity recognition by investigating patterns of eye movement as a novel contextual cue. What makes eye movement a very distinct source of activity-related information is their direct link to visual cognition. This means that eye movement data has the potential to provide an online indication not only of what activity occurs but also of underlying cognitive processes, for example attention. Eye movements, including their measurement and analysis, have been studied extensively in clinical ophthalmology and cognitive science, but their use for on-line activity recognition in everyday settings is a novel proposition. Significant challenges include the adaptation of measurement techniques fit for everyday settings, the quality of information that can be mined under realistic conditions, and the demonstration of practical utility of eye-based context in real-world application.This project is designed as an initial investigation of the feasibility and potential of eye-based activity recognition. The research will include an assessment of eye tracking techniques, specifically of optical methods that track features on the eye, and of electrooculography (EOG) measuring eye movement with skin electrodes placed near the eyes. Based on the assessment, we will conduct explorative studies of eye patterns in daily life. The aim here is to gain insight into what eye movement reveals about everyday activity, and to develop methods for extraction of features from eye movement patterns that may be useful for activity and context recognition. A third part of the work will investigate application of eye-based context information in interactive systems. The plan is to develop two practical demonstrators, a first one exploring how longitudinal eye movement analysis may contribute to health and lifestyle monitoring, and a second one showing how eye movements might be used for task assistance (for example, search assistance when the eyes indicate visual search activity).