Human Hands as Probes for Interactive Object Understanding
Human Hands as Probes for Interactive Object Understanding
复制标题
DOI：
10.1109/cvpr52688.2022.00329
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Mohit Goyal;Sahil Modi;Rishabh Goyal;Saurabh Gupta
中科院分区：
文献类型：
--
作者：
Mohit Goyal;Sahil Modi;Rishabh Goyal;Saurabh Gupta
交互式对象理解，或者我们可以对对象做什么以及如何做，是计算机视觉的一个长期目标。在本文中，我们通过观察野外自我中心视频中的人手来解决这个问题。我们证明，观察人类的手与什么以及如何相互作用，可以提供相关数据和必要的监督。注意手部，容易定位和稳定学习的活动物体，并揭示与物体发生互动的地方。分析手表明我们可以对物体做什么以及如何做。我们将这些基本原则应用于EPIC-KITCHENS数据集，并成功地学习状态敏感特征和对象可视性（交互区域和提供的抓取），纯粹通过观察以自我为中心的视频中的手。
Interactive object understanding, or what we can do to objects and how is a long-standing goal of computer vision. In this paper, we tackle this problem through observation of human hands in in-the-wild egocentric videos. We demonstrate that observation of what human hands interact with and how can provide both the relevant data and the necessary supervision. Attending to hands, readily localizes and stabilizes active objects for learning and reveals places where interactions with objects occur. Analyzing the hands shows what we can do to objects and how. We apply these basic principles on the EPIC-KITCHENS dataset, and successfully learn state-sensitive features, and object affordances (regions of interaction and afforded grasps), purely by observing hands in egocentric videos.