Multiview Supervision By Registration
Multiview Supervision By Registration
复制标题
DOI：
10.1109/wacv45572.2020.9093591
复制
发表时间：
2018-11
期刊：
影响因子：
--
通讯作者：
Yilun Zhang;H. Park
中科院分区：
文献类型：
--
作者：
Yilun Zhang;H. Park
本文提出了一种半监督学习框架，用于在给定有限数量的标记实例（通常<4%）的情况下，使用多视图图像流来训练关键点检测器。我们利用多视图跟踪中的三个自我监督信号来利用未标记数据：（1）一个视图中的关键点可以通过对极几何被其他视图监督;（2）关键点检测必须在时间上一致;（3）一个视图中的可见关键点可能在相邻视图中可见。我们设计了一个新的端到端网络，可以以可区分的方式将这些自我监督信号传播到未标记的数据和标记的数据中。我们表明，我们的方法优于现有的检测器，包括针对非人类物种（如猴子、狗和小鼠）的关键点检测而定制的DeepLabCut。
This paper presents a semi-supervised learning framework to train a keypoint detector using multiview image streams given the limited number of labeled instances (typically <4%). We leverage three self-supervisionary signals in multiview tracking to utilize the unlabeled data: (1) a keypoint in one view can be supervised by other views via epipolar geometry; (2) a keypoint detection must be consistent across time; (3) a visible keypoint in one view is likely to be visible in the adjacent view. We design a new end-to-end network that can propagate these self-supervisionary signals across the unlabeled data from the labeled data in a differentiable manner. We show that our approach outperforms existing detectors including DeepLabCut tailored to the keypoint detection of non-human species such as monkeys, dogs, and mice.