Developing Visual Data Understanding through Self-Similarity

EPSRC研究领域：图像和视觉计算，人工智能技术研究背景：相同类型的数据（例如向量，文本文件，图像）之间的比较是信息处理中的关键任务。然而，在视觉数据（图像和视频）的背景下，计算机视觉方法仍然缺乏相对于人类视觉感知的有意义的比较工具。例如，如果一个孩子在字母表的最后看到一只斑马的图片，他通常就能认出动物园里的动物，尽管在姿势、外貌、规模和其他数量上不可避免地存在差异。在这种情况下，孩子已经学会了两个非常不同的原始视觉数据之间的相似性。目前深度神经网络的大部分进展都依赖于大型人工注释数据集，而这些数据集的收集既昂贵又耗费人力。回到这个例子，目前的方法很难从一个例子中理解斑马的特征，然后可以立即应用，就像孩子一样。目的和目标：本研究旨在利用自然图像中的自相似性来解决监督学习框架中的多个任务，并开发可广泛应用于多个计算机视觉任务的视觉数据表示。短期目标围绕着使用自相似性来解决类别不可知论计数和显著对象主体化问题，即预测图像中显著对象的存在和数量。提供任何感兴趣的对象的示例，我们的目标是在查询图像集合中提供准确的计数，而不需要/很少的微调。这可以应用于微生物学中，在不同初始条件下计算细胞生长的情况，也可以应用于动物学中，在一天的不同时间计算特定物种的数量。长期研究目标是利用自监督框架发展当前的特征学习工作，重点关注依赖于自相似性的技术。研究方法的新颖性：参考类别不可知论计数的短期目标和显著客体的主观化。在给定域上强调无/最小微调与计数任务中的零/几次学习有关。以前没有一个模型能够在不需要对感兴趣的领域进行大量训练的情况下有效地做到这一点。

EPSRC Research Areas: Image and Vision Computing, Artificial Intelligence TechnologiesResearch Context: Comparison between data of the same type (e.g. vectors, text files, images) is a critical task in information processing. In the context of visual data however (images and videos), computer vision methods still lack meaningful tools for comparison relative to human visual perception. For example, a child which sees a single picture of a zebra at the end an alphabet book, will often be able to recognise the animal in a zoo, despite inevitable differences in pose, appearance, scale and other quantities. In this case the child has learnt the similarity between two very different pieces of raw visual data.Much of the current progress of deep neural networks has relied on large human-annotated datasets, which are expensive and labour-intensive to collect. Returning to the example, current methods will struggle to understand the character of a zebra from a single example which can then be applied immediately, as a child could.Aims and Objectives: This research aims to leverage self-similarity present in natural images to solve multiple tasks in the supervised-learning framework and to develop representations of visual data which can be widely applied to multiple computer vision tasks. Short-term objectives revolve around using self-similarity to solve the problem of class-agnostic counting and salient object subitizing, i.e. predicting the existence and the number of salient objects in an image. Provided an example of any object of interest, we aim to provide accurate counts in a collection of query images, with no/minimal fine-tuning. This could be applied in microbiology in the context of counting cell growth given different initial conditions or in zoology to count the number of a certain species at different times of the day.Long-term research aims are to develop the current work of feature-learning using the self-supervised framework with a focus on techniques relying on self-similarity.Novelty of the Research Methodology: With reference to the short-term objective of class-agnostic counting and salient object subitizing. The emphasis on no/minimal fine-tuning on a given domain is connected to zero/few-shot learning in the counting task. No previous model has been able to do this effectively without requiring significant training on a domain of interest.