CRII: RI: Large-Scale Discovery and Organization of Subcategories and Parts from Image and Video Segments

这个项目开发了一个系统，用于理解给定有限注释的视觉类别。系统自动检测仅具有类别级注释的子类别和对象部件。这种详细的理解对于自主系统执行与对象的交互或在遮挡下识别它们是重要的。为此，目前的深度神经网络将被扩展，以更好地支持不规则形状的对象和部件。一旦在这样的水平上的理解已经实现，它有助于构建新的类别，从几个样本，这在自治systems.This研究推广以前成功的方法在语义分割和无监督视频分割的有效方法来学习子类别和部分，具有广泛的应用。该框架从重叠的图形-背景段建议开始，计算输入段对段重叠的最小二乘回归，并利用谢尔曼-莫里森-伍德伯里公式和二次损失函数的结构，同时有效地优化数千到数十万的子类别和部分。然后，该项目探索了深度卷积网络的训练，并从这些子类别和自由形式段上定义的部分进行初始化。这需要神经网络架构的泛化，以处理可以通过视频序列变形的自由形式片段。建议使用测地距离变换的空间-时间段定义定制的过滤器，为不同的地方，以提高性能和更好的可解释性。

This project develops a system for understanding of visual categories given limited annotations. The system automatically detects subcategories and object parts with only category-level annotations. Such detailed understanding is important for autonomous systems to perform interactions with objects or to recognize them under occlusion. For this purpose, current deep neural networks will be extended to better support objects and parts of irregular shape. Once understandings at such level have been achieved, it helps to construct new categories from just a few exemplars, which has broad applications in autonomous systems.This research generalizes previously successful approaches in semantic segmentation and unsupervised video segmentation for an efficient approach to learn subcategories and parts. The framework starts from overlapping figure-ground segment proposals, computes least squares regressors from input segments against segment overlaps, and utilizes the Sherman-Morrison-Woodbury formula and structures from the quadratic loss function for efficient optimization of thousands to hundreds of thousands of subcategories and parts simultaneously. This project then explores the training of deep convolutional networks with initializations from these subcategories and parts defined on free-form segments. This requires generalization of the neural network architecture to handle free-form segments that can deform through a video sequence. It is proposed to use the geodesic distance transform on spatial-temporal segments to define customized filters for different localities for improved performance and better interpretability.