Self-supervised Face-Grouping on Graphs
Self-supervised Face-Grouping on Graphs
复制标题
DOI：
10.1145/3343031.3351071
复制
发表时间：
2019-10
期刊：
影响因子：
--
通讯作者：
Veith Röthlingshöfer;Vivek Sharma;R. Stiefelhagen
中科院分区：
文献类型：
--
作者：
Veith Röthlingshöfer;Vivek Sharma;R. Stiefelhagen
我们提出了一种新的自监督方法，用于微调深度人脸表示，称为Face-Duplex on Graphs。我们将我们的方法应用于自动人脸分组，其中字符将根据其身份进行分离。针对这一问题，提出了一种基于人脸轨迹时间重叠和相似性约束的正负边图结构，该结构不需要人工操作。我们计算每个轨道（子轨道）的子序列上的特征表示，以便获得鲁棒的特征，同时能够利用包含在面部方差中的信息。每个子轨道被赋予与相邻子轨道交换信息的能力，通过一个运行在诱导图上的类型图神经网络。这允许我们在特征空间中将每个表示推向一个方向，该方向将同一字符的所有表示分组在一起，并将不同字符的表示分开。我们表明，我们的方法是能够提高流行的视频人脸聚类数据集的大爆炸理论和吸血鬼杀手巴菲的聚类准确率分别为4.9%和17.0%相比，基线性能，和0.52%，分别为5.55%的com-kind的最先进的方法。此外，与其他最先进的无监督方法相比，我们在《哈利·波特1》（ACCIO）上实现了B3 F-Score绝对提高19.0%。我们提供了《生活大爆炸》和《吸血鬼猎人巴菲》所有剧集的表现指标，以便将来进一步比较。
We propose a novel self-supervised method for fine-tuning deep face representations called Face-Grouping on Graphs. We apply our method to automatic face grouping, where characters are to be separated based on their identity. To solve this problem, a graph structure with positive and negative edges over a set of face-tracks based on their temporal overlap and similarity constraints is in- duced, which requires no manual labor. We compute feature repre- sentations over sub-sequences of each track (sub-tracks) in order to obtain robust features whilst being able to utilize information contained in face variance. Each sub-track is given the ability to exchange information with adjacent sub-tracks via a typed graph neural network running over the induced graph. This allows us to push each representation in a direction in feature space that groups all representations of the same character together and separates representations of different characters. We show that our method is capable of improving clustering accuracy on popular video face clustering datasets The Big Bang Theory and Buffy the Vampire Slayer by 4.9% and 17.0% respectively compared to baseline performance, and 0.52% respective 5.55% com- pared to state-of-the-art methods. Additionally, we achieve 19.0% absolute increase in B3 F-Score on Harry Potter 1 (ACCIO) over other state-of-the-art unsupervised methods. We provide perfor- mance metrics on all episodes of The Big Bang Theory and Buffy the Vampire Slayer to enable further comparison in the future.