基于生成图像的人眼检测及视线估计方法研究

基于图像的人眼检测及视线估计具有巨大潜在应用市场，比如人机交互、游戏娱乐、生物特征识别、辅助驾驶系统等。尽管目前已有大量相关研究工作，但是由于个体差异、光照变化、姿态变换以及数据标注困难等，人眼检测及视线估计仍然是一个具有挑战性的问题。本项目拟研究一种基于生成对抗网络的图像生成模型，并提出一个基于深度级联回归的统一框架来同时实现人眼检测及视线估计。具体地，我们显式地对大规模仿真人眼图像、对应标注信息、及无标注真实图像分布进行建模，利用对抗学习来得到有效图像生成模型，该模型能有效融合仿真图像、标注信息及真实人眼图像外观纹理。通过生成模型产生大量训练数据后，我们进一步学习基于深度级联回归的检测识别模型。在每一级级联迭代步骤中，基于眼部关键点，我们利用深度网络对眼部全局结构形状信息、局部区域图像特征、人眼瞳孔中心、三维视线方向的关联性进行建模，从而实现交替优化人眼检测及视线方向估计两个任务。

Image-based eye detection and gaze estimation have a wide range of potential applications, such as human-computer interaction, gaming, biometrics recognition, and driver assistant systems, etc. Though a large number of researches have been don for these two tasks, they are still challenging due to the variation of appearance, illumination, head pose and lack of annotated images. In this project, we are proposing a Generative Adversarial Networks (GANs) based model for image generation and a united deep cascade regression framework for simultaneous eye detection and gaze estimation. More specifically, we explicitly model the distribution of large scale of synthetic eyes with labels and real images without annotations based on adversarial learning methods. After generating the large scale of eye images with annotations, we further learn the deep regression models. At each cascade level, we learn the deep models to explore the coupled relations of global structure, local features, pupil position and 3D gaze direction based on the eye related landmarks. As a result, alternative optimization of the eye detection and gaze estimation is achieved.