GazeGraph: graph-based few-shot cognitive context sensing from human visual behavior
GazeGraph: graph-based few-shot cognitive context sensing from human visual behavior
复制标题
DOI：
10.1145/3384419.3430774
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Guohao Lan;Bailey Heit;T. Scargill;M. Gorlatova
中科院分区：
文献类型：
--
作者：
Guohao Lan;Bailey Heit;T. Scargill;M. Gorlatova
在这项工作中，我们提出了GazeGraph，一个系统，利用人类的目光作为感知模态的认知上下文传感。GazeGraph是一个通用的框架，与不同的眼动仪兼容，并支持各种基于凝视的传感应用。它确保了在人类视觉行为存在异质性的情况下的高感测性能，并使系统能够快速适应具有少数镜头实例的不可见感测场景。为了实现这些功能，我们引入了时空凝视图和基于深度学习的表示学习方法，从眼动中提取强大的和广义的特征，用于上下文感知。此外，我们开发了一个少数镜头凝视图学习模块，适应“学习学习”的概念，从元学习，使快速系统适应数据高效的方式。我们的评估表明，在三个数据集上，GazeGraph的识别准确率平均比现有解决方案高出45%。此外，在少量学习场景中，GazeGraph的性能比基于迁移学习的方法高出19%到30%，同时将系统适应时间减少了80%。
In this work, we present GazeGraph, a system that leverages human gazes as the sensing modality for cognitive context sensing. GazeGraph is a generalized framework that is compatible with different eye trackers and supports various gaze-based sensing applications. It ensures high sensing performance in the presence of heterogeneity of human visual behavior, and enables quick system adaptation to unseen sensing scenarios with few-shot instances. To achieve these capabilities, we introduce the spatial-temporal gaze graphs and the deep learning-based representation learning method to extract powerful and generalized features from the eye movements for context sensing. Furthermore, we develop a few-shot gaze graph learning module that adapts the `learning to learn' concept from meta-learning to enable quick system adaptation in a data-efficient manner. Our evaluation demonstrates that GazeGraph outperforms the existing solutions in recognition accuracy by 45% on average over three datasets. Moreover, in few-shot learning scenarios, GazeGraph outperforms the transfer learning-based approach by 19% to 30%, while reducing the system adaptation time by 80%.