Compound Prototype Matching for Few-shot Action Recognition
Compound Prototype Matching for Few-shot Action Recognition
复制标题
DOI：
10.48550/arxiv.2207.05515
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Lijin Yang;Yifei Huang;Y. Sato
中科院分区：
文献类型：
--
作者：
Lijin Yang;Yifei Huang;Y. Sato
少镜头动作识别的目标是识别新的动作类，只使用少量的标记训练样本。在这项工作中，我们提出了一种新的方法，首先总结每个视频到复合原型组成的一组全球原型和一组重点原型，然后比较视频相似性的基础上的原型。鼓励每个全局原型从整个视频中总结一个特定方面，例如，动作的开始/演变。由于没有为全局原型提供明确的注释，因此我们使用一组聚焦原型来关注视频中的某些时间戳。我们通过匹配支持和查询视频之间的复合原型来比较视频相似性。全局原型被直接匹配以从相同视角比较视频，例如，以比较两个动作是否相似地开始。对于集中的原型，由于动作在视频中有各种时间变化，我们应用二分匹配来比较具有不同时间位置和移位的动作。实验表明，我们提出的方法在多个基准测试中取得了最先进的结果。
Few-shot action recognition aims to recognize novel action classes using only a small number of labeled training samples. In this work, we propose a novel approach that first summarizes each video into compound prototypes consisting of a group of global prototypes and a group of focused prototypes, and then compares video similarity based on the prototypes. Each global prototype is encouraged to summarize a specific aspect from the entire video, for example, the start/evolution of the action. Since no clear annotation is provided for the global prototypes, we use a group of focused prototypes to focus on certain timestamps in the video. We compare video similarity by matching the compound prototypes between the support and query videos. The global prototypes are directly matched to compare videos from the same perspective, for example, to compare whether two actions start similarly. For the focused prototypes, since actions have various temporal variations in the videos, we apply bipartite matching to allow the comparison of actions with different temporal positions and shifts. Experiments demonstrate that our proposed method achieves state-of-the-art results on multiple benchmarks.