SCK - Sparse Coding Based Key-point Detectors for Knowledge Discovery

在图像匹配、图像拼接和目标检测/识别等众多基于视觉的应用中，图像关键点或特征点的检测是非常重要的一步。在该领域的研究中，已经成功地引入了许多关键点检测器，如Harris、SIFT、SURF、MSER、SFOP。然而，基于人类预定义的结构(角、斑点或结点等)进行开发的本质使这些经典的(手工制作的)探测器在意外情况下使用时不灵活。近年来，深度学习被应用于关键点检测。然而，主流的基于学习的方法仍然使用手工制作的检测器来为他们的训练算法提供输入，基本上也是手工制作的。这些事实清楚地表明，新经典探测器的研制仍然是一项不可或缺的任务。1.我们将解析地验证SCK所具有的标度和转动不变性。预计，通过结合这一特性，SCK的应用范围将显著扩大，以处理非常不同比例和旋转角度的图像。2.我们将研究如何使用SCK来增强现有的检测器，无论是经典检测器还是基于学习的检测器。通过将SCK与手工制作的探测器融合在一起，预计SCK对不均匀的光照、比例和旋转变化的稳健性将提高经典探测器的质量。还将研究一种基于学习的检测器，其中SCK提供可靠的输入。3.提出并开发了一种新的特征描述子，以取代目前流行的信息识别力有限的SIFT描述子。新的描述符应与SCK检测器一致地工作。4.将前面三个任务中开发的算法应用到多个实际的视觉任务中，特别是在两个我们拥有高水平专业知识和广泛经验的领域：非侵入性生物特征识别和多媒体信息处理。拟议研究的预期结果是产生一个新颖和完整的框架，以及用于关键点检测和特征描述符提取的相关工具。所获得的知识将为加拿大带来显著的社会效益，并为图像匹配、图像拼接、人机交互、多媒体信息检索等应用研究提供使能技术，并解决广泛的现实世界问题，包括自动驾驶汽车、高质量摄影、新媒体制作/交付、文化遗产保护、电子健康、智能家居、游戏和电子娱乐，以及在沉浸式多媒体环境中创造逼真的体验。

Detection of key-points or distinctive points in images is a very important step in numerous vision-based applications i.e. image matching, image stitching and object detection/recognition. The research in the field has seen successful introduction of many key-point detectors such as Harris, SIFT, SURF, MSER, SFOP. However, the very nature of being developed based on a human pre-defined structure (a corner, blob or junction, etc.) makes these classical (hand-crafted) detectors inflexible when being used in unintended situations. In recent years, deep learning has been applied to key-point detection. Yet, the main stream of learning based methods remains using hand-crafted detectors to provide input to their training algorithms, essentially making them hand-crafted as well. These facts clearly indicate that the development of new classical detectors is still an indispensable task. 1.We will analytically verify the property of scale and rotational invariance possessed by SCK. It is expected that, by incorporating this property, the application range of SCK will be significantly broadened to handle images of very different scales and rotational angles. 2.We will investigate how SCK can be used to enhance the available detectors, classical or learning based alike. By fusing SCK and the hand-crafted detectors, it is expected that the robustness of SCK against non-uniform lighting, scale and rotational changes will improve the quality of classical detectors. A learning based detector with SCK providing reliable input will also be studied. 3.We will propose and develop a new feature descriptor, to replace the popular SIFT descriptor which possesses limited information discrimination power. The new descriptor is expected to work coherently with the SCK detector. 4.We will apply the developed algorithms in the previous three tasks to multiple practical vision tasks, especially focusing on two areas in which we have high level of expertise and broad experience: non-invasive biometrics and multimedia information processing. The expected outcome of the proposed research is the generation of a novel and complete framework with associated tools for key-point detection and feature descriptor extraction. The gained knowledge will generate significant societal benefits to Canada and provide enabling technology for applied research such as image matching, image stitching, human-computer interaction, multimedia information retrieval, and solution to a broad range of real world problems, including autonomous vehicles, high quality photography, new media production/delivery, preservation of cultural heritage, e-health, smart homes, gaming and e-entertainment, and creating life-like experience in immersive multimedia environment.