Learning Visual Actions Using Multiple Verb-Only Labels
Learning Visual Actions Using Multiple Verb-Only Labels
复制标题
DOI：
--
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
Michael Wray;D. Damen
中科院分区：
文献类型：
--
作者：
Michael Wray;D. Damen
这项工作介绍了动词只表示识别和检索的视觉动作，在视频中。当前的方法忽视了动词之间合理的语义歧义，而是选择动词的明确子集沿着对象来消除动作的歧义。相反，我们提出了多个仅限动词的标签，我们通过硬或软分配作为回归来学习。这使得学习更大的动词词汇，包括这些动词的上下文重叠。我们收集了三个动作视频数据集的多动词注释，并评估了动作识别和跨模态检索（视频到文本和文本到视频）的动词标签表示。我们证明了多标签动词表示优于传统的单动词标签。我们还探索了多动词表示的其他好处，包括跨数据集检索和动词类型（方式和结果动词类型）检索。
This work introduces verb-only representations for both recognition and retrieval of visual actions, in video. Current methods neglect legitimate semantic ambiguities between verbs, instead choosing unambiguous subsets of verbs along with objects to disambiguate the actions. We instead propose multiple verb-only labels, which we learn through hard or soft assignment as a regression. This enables learning a much larger vocabulary of verbs, including contextual overlaps of these verbs. We collect multi-verb annotations for three action video datasets and evaluate the verb-only labelling representations for action recognition and cross-modal retrieval (video-to-text and text-to-video). We demonstrate that multi-label verb-only representations outperform conventional single verb labels. We also explore other benefits of a multi-verb representation including cross-dataset retrieval and verb type manner and result verb types) retrieval.