基于深度学习的特征点检测与描述方法研究

本项目旨在探索基于深度学习的图像特征点检测和局部图像特征描述方法，主要研究内容包括以下三个方面：（1）图像特征点检测：建立新的基于深度神经网络的特征点响应计算方法和端到端的特征点检测方案，充分利用深度神经网络强大的函数拟合与表达能力来提升特征点检测算法的性能；（2）局部图像特征描述：建立面向特征描述的光谱特征变换理论和相应的深度神经网络，根据局部图像内容自适应地确定相应的光谱变换关系，将对应点的局部图像映射到相似的光谱特征空间，以提升特征描述子的匹配性能；（3）特征点检测和描述的一体化：面向特征点检测与描述的多任务特性，建立相应的深度神经网络结构和学习算法，提出特征点检测与描述一体化的深度神经网络方法，同时建立基于弱监督信息的网络学习算法。本项目不仅有重要学术意义更具有广泛的应用价值，特别适用于三维重建、SLAM、图像融合、图像风格迁移等以图像内容匹配为基础的高层视觉应用任务。

This proposal aims to research on new methods for keypoint detection and local feature description based on deep learning. The main research contents include: (1) For keypoint detection, in order to fully utilize the powerful ability of function fitting and feature representation of CNNs, we research on how to learn keypoint’s response function by CNNs to improve the detector’s repeatability. We also propose an end-to-end CNN solution for keypoint detection. (2) For local feature description, we research on a spectral transform network and related learning methods to transform local images of corresponding keypoints into similar spectral space, so as to extract local features with high matching performance. (3) The previous two items are incorporated together by researching on multi-task CNNs for simultaneously keypoint detection and local feature description. Meanwhile, we research on how to leverage on weakly supervised information for simultaneously keypoint detection and local feature description. This proposal not only has important scientific merits, but also is extremely useful for many computer vision applications which are based on local image matching, such as 3D reconstruction, SLAM, image fusion, image style transferring, etc.