Developing Visual Data Understanding through Self-Similarity

EPSRC研究领域：图像和视觉计算、人工智能技术研究背景：同类数据(如矢量、文本文件、图像)之间的比较是信息处理中的一项关键任务。然而，在视觉数据(图像和视频)的背景下，计算机视觉方法仍然缺乏与人类视觉感知进行比较的有意义的工具。例如，一个孩子在字母表的末尾看到一张斑马的图片，通常能够识别动物园里的动物，尽管在姿势、外表、体型和其他数量上不可避免地会有所不同。在这种情况下，孩子已经学会了两个非常不同的原始视觉数据之间的相似性。目前深度神经网络的大部分进展依赖于大量人类注释的数据集，收集这些数据集既昂贵又劳动密集型。回到这个例子，现有的方法很难从单个例子中理解斑马的特征，然后像孩子一样立即应用。目的和目标：本研究旨在利用自然图像中存在的自相似性来解决监督学习框架中的多个任务，并开发可广泛应用于多个计算机视觉任务的视觉数据表示。短期目标围绕着使用自相似来解决与类别无关的计数和显著对象细分的问题，即预测图像中显著对象的存在和数量。提供任何感兴趣的对象的示例，我们的目标是在没有/最少微调的情况下，在查询图像集合中提供准确的计数。这可以应用于微生物学，在给定不同初始条件下对细胞生长进行计数，或在动物学中对某一物种在一天中不同时间的数量进行计数。长期研究目标是利用自我监督框架发展当前的特征学习工作，重点是依赖于自相似的技术。研究方法的新奇之处：参考类不可知计数和显著对象细分的短期目标。在计数任务中，对给定域上的无/最小微调的强调与零/少射击学习有关。以前的任何模型都不需要在感兴趣的领域进行大量培训就能够有效地做到这一点。

EPSRC Research Areas: Image and Vision Computing, Artificial Intelligence TechnologiesResearch Context: Comparison between data of the same type (e.g. vectors, text files, images) is a critical task in information processing. In the context of visual data however (images and videos), computer vision methods still lack meaningful tools for comparison relative to human visual perception. For example, a child which sees a single picture of a zebra at the end an alphabet book, will often be able to recognise the animal in a zoo, despite inevitable differences in pose, appearance, scale and other quantities. In this case the child has learnt the similarity between two very different pieces of raw visual data.Much of the current progress of deep neural networks has relied on large human-annotated datasets, which are expensive and labour-intensive to collect. Returning to the example, current methods will struggle to understand the character of a zebra from a single example which can then be applied immediately, as a child could.Aims and Objectives: This research aims to leverage self-similarity present in natural images to solve multiple tasks in the supervised-learning framework and to develop representations of visual data which can be widely applied to multiple computer vision tasks. Short-term objectives revolve around using self-similarity to solve the problem of class-agnostic counting and salient object subitizing, i.e. predicting the existence and the number of salient objects in an image. Provided an example of any object of interest, we aim to provide accurate counts in a collection of query images, with no/minimal fine-tuning. This could be applied in microbiology in the context of counting cell growth given different initial conditions or in zoology to count the number of a certain species at different times of the day.Long-term research aims are to develop the current work of feature-learning using the self-supervised framework with a focus on techniques relying on self-similarity.Novelty of the Research Methodology: With reference to the short-term objective of class-agnostic counting and salient object subitizing. The emphasis on no/minimal fine-tuning on a given domain is connected to zero/few-shot learning in the counting task. No previous model has been able to do this effectively without requiring significant training on a domain of interest.