CRII: CHS: Scalable Webcam Eyetracking by Learning from User Interactions

眼动跟踪技术在很多应用中都非常有用，包括人机交互研究、可用性测试、医学研究和心理学实验等。 但这些设备是高度专业化和昂贵的设备，难以校准和使用，因此它们大部分只能在实验室中使用。 在这个项目中，PI将建立在他之前的工作基础上，建立一个研究计划，以研究一种基于当今笔记本电脑和移动的设备中常见的网络摄像头的眼动跟踪新方法，目标是使该技术成为日常用户自然体验的一部分，从而不再局限于实验室和高度受控的研究。 当然，网络摄像头在估计用户在屏幕上的位置方面不如专门的眼球跟踪设备准确。 PI克服这一缺点的方法是通过利用用户交互来提高准确性，以在常规使用期间持续校准网络摄像头即眼动仪，并且在线完成此操作而无需安装额外的软件。 项目成果最终将包括一个实时在线眼动跟踪系统，该系统使用笔记本电脑和移动的设备中的典型网络摄像头，并对其性能进行沿着评估。 PI还将研究如何使用光标点击、文本输入和触摸等用户交互来自动训练眼动跟踪算法。 这项新技术将使眼动追踪大众化，将其从实验室的限制中解放出来; PI将把源代码沿着眼动追踪演示，让其他研究人员和开发人员在他们的工作中应用他的技术。PI之前的工作表明，当用户点击网页时，他们会首先看他们想点击的地方。 此外，心理学研究表明，眼睛很可能在屏幕上最后一个键入字符的右侧2-4个字符。 在这些用户交互期间的网络摄像头图像可以由网站收集，以用作当她/他与特定位置交互时给定用户的瞳孔看起来像什么的线索。 当系统收集瞳孔特征到页面上眼睛注视位置的映射时，瞳孔的未来观察结果可以与具有相似瞳孔的过去实例相匹配，从而允许模型即使在用户没有交互时也能推断眼睛注视位置。 可以在用户与网站交互的整个时间期间并且在不中断用户体验的情况下收集瞳孔数据，包括在计算机使用会话开始时，以提供在环境照明、用户坐姿和背景环境方面更好地匹配本地环境的模型训练数据。 通过使眼动跟踪能够从典型的web浏览器访问，并且通过在用户访问网站时不断提高跟踪准确性，眼动跟踪对于许多潜在的应用成为现实，例如大规模自然用户研究、在线游戏或使人们能够执行网站的免提导航。 当浏览器请求访问网络摄像头时，这种眼睛跟踪程序是选择性的，如果用户同意，网站就能够捕获这些数据。

Eye tracking technology is very useful for a variety of applications including human-computer interaction studies, usability testing, medical research, and experiments in psychology, to name just a few. But the devices are highly specialized and costly equipment that is difficult to calibrate and use, so they remain available for the most part in the lab only. In this project the PI will build on his prior work to establish a research program to investigate a new approach to eye tracking based on the webcams commonly present in today's laptops and mobile devices, with the goal of making the technology viable for a broader range of applications as part of the natural experience of everyday users and so no longer restricted to laboratories and highly controlled studies. Of course, webcams are less accurate than specialized eye tracking equipment for estimating where a user is looking on the screen. The PI's approach to overcoming this drawback is to improve the accuracy by exploiting user interactions to continuously calibrate the webcam-as-eye tracker during regular usage, and to do this online without the need to install additional software. Project outcomes will ultimately include a real-time online eye tracking system using the typical webcam available in laptops and mobile devices, along with an evaluation of its performance. The PI will also conduct research into how user interactions such as cursor clicks and text entry and touches can be used to automatically train the eye tracking algorithms. The new technology will democratize eye tracking, releasing it from the confines of the lab; the PI will disseminate source code along with eye tracking demos to allow other researchers and developers to apply his technology in their work.The PI's prior work has shown that when a user clicks on a web page, they will first look where they intend to click. Furthermore, psychology studies have shown that the eye is likely to be 2-4 characters to the right of the last typed character on the screen. Webcam images during these user interactions can be collected by the website to use as cues to what a given user's pupil looks like when s/he is interacting with a particular location. Future observations of the pupil can then be matched to past instances with similar-looking pupils as the system collects mappings of pupil features to eye-gaze locations on the page, allowing the model to infer the eye-gaze location even when the user is not interacting. The pupil data can be collected during the entire time that a user interacts with a website and without disrupting the user experience, including at the beginning of a computer usage session to provide model training data that better matches the local environment in terms of ambient lighting, user sitting position, and background environment. By enabling eye tracking to be accessible from a typical web browser and by continuously improving the tracking accuracy as a user visits a website, eye tracking becomes a reality for many potential applications such as large-scale naturalistic user studies, online gaming, or enabling people to perform hands-free navigation of websites. This eye tracking procedure is opt-in as browsers request access to the webcam, and the website is able to capture this data if the user agrees.