SCK - Sparse Coding Based Key-point Detectors for Knowledge Discovery

在图像匹配、图像拼接和目标检测/识别等众多基于视觉的应用中，关键点或特征点的检测是非常重要的一步。该领域的研究已经成功地引入了许多关键点探测器，如Harris、SIFT、SURF、MSER、SFOP等。然而，基于人类预先定义的结构（角落，斑点或连接处等）开发的本质使得这些经典（手工制作）探测器在意外情况下使用时缺乏灵活性。近年来，深度学习已被应用于关键点检测。然而，主流的基于学习的方法仍然使用手工制作的检测器为他们的训练算法提供输入，本质上使它们也是手工制作的。这些事实清楚地表明，开发新的经典探测器仍然是一项不可或缺的任务。1.我们将解析地验证SCK所具有的尺度不变性和旋转不变性。预计通过结合这一特性，SCK的应用范围将大大扩大，可以处理非常不同尺度和旋转角度的图像。2.我们将研究如何使用SCK来增强可用的检测器，无论是经典的还是基于学习的。通过融合SCK和手工制作的探测器，期望SCK对非均匀光照、尺度和旋转变化的鲁棒性将提高经典探测器的质量。一个基于学习的检测器与SCK提供可靠的输入也将被研究。3.我们将提出并开发一种新的特征描述符，以取代目前流行的SIFT特征描述符，该特征描述符具有有限的信息识别能力。新的描述符有望与SCK探测器协同工作。4.我们将在前三个任务中开发的算法应用于多个实际的视觉任务，特别是关注我们具有高水平专业知识和丰富经验的两个领域：非侵入性生物识别和多媒体信息处理。所提出的研究的预期结果是生成一个具有关键点检测和特征描述符提取相关工具的新颖而完整的框架。获得的知识将为加拿大带来显著的社会效益，并为应用研究提供支持技术，如图像匹配、图像拼接、人机交互、多媒体信息检索，以及解决广泛的现实世界问题，包括自动驾驶汽车、高质量摄影、新媒体制作/交付、文化遗产保护、电子医疗、智能家居、游戏和电子娱乐。并在沉浸式多媒体环境中创造栩栩如生的体验。

Detection of key-points or distinctive points in images is a very important step in numerous vision-based applications i.e. image matching, image stitching and object detection/recognition. The research in the field has seen successful introduction of many key-point detectors such as Harris, SIFT, SURF, MSER, SFOP. However, the very nature of being developed based on a human pre-defined structure (a corner, blob or junction, etc.) makes these classical (hand-crafted) detectors inflexible when being used in unintended situations. In recent years, deep learning has been applied to key-point detection. Yet, the main stream of learning based methods remains using hand-crafted detectors to provide input to their training algorithms, essentially making them hand-crafted as well. These facts clearly indicate that the development of new classical detectors is still an indispensable task. 1.We will analytically verify the property of scale and rotational invariance possessed by SCK. It is expected that, by incorporating this property, the application range of SCK will be significantly broadened to handle images of very different scales and rotational angles. 2.We will investigate how SCK can be used to enhance the available detectors, classical or learning based alike. By fusing SCK and the hand-crafted detectors, it is expected that the robustness of SCK against non-uniform lighting, scale and rotational changes will improve the quality of classical detectors. A learning based detector with SCK providing reliable input will also be studied. 3.We will propose and develop a new feature descriptor, to replace the popular SIFT descriptor which possesses limited information discrimination power. The new descriptor is expected to work coherently with the SCK detector. 4.We will apply the developed algorithms in the previous three tasks to multiple practical vision tasks, especially focusing on two areas in which we have high level of expertise and broad experience: non-invasive biometrics and multimedia information processing. The expected outcome of the proposed research is the generation of a novel and complete framework with associated tools for key-point detection and feature descriptor extraction. The gained knowledge will generate significant societal benefits to Canada and provide enabling technology for applied research such as image matching, image stitching, human-computer interaction, multimedia information retrieval, and solution to a broad range of real world problems, including autonomous vehicles, high quality photography, new media production/delivery, preservation of cultural heritage, e-health, smart homes, gaming and e-entertainment, and creating life-like experience in immersive multimedia environment.