Multiwavelet Evaluation for Stereo Correspondence

立体视觉，从立体图像中提取深度信息的过程，在其离散形式中有多种实际应用，范围从娱乐，广播，3D建模和可视化到机器人和自动化工业检测，仅举几例。计算左右立体图像之间的差异，即立体对应匹配，是立体视觉应用中最具挑战性的阶段。从生物学上讲，人类大脑通过采用多通道、多分辨率机制来完成这项任务，识别自然视觉场景中包含多种分辨率的细节信息。离散立体视觉可以在空间域和频率域进行。空间图像处理技术在空间上提供完美的定位，但在频率上没有定位。传统的图像变换在频率上实现了很好的定位，但在空间上没有（或很差）定位。小波是第一个提供分层多分辨率方法的现代变换，它结合了两个世界的优点：原始图像的空间频率分层多分辨率分解，具有局部和全局空间支持（第2.3节）。它们迅速成为立体声通信的流行选择。为了在立体对应的背景下实现最佳性能，小波在理想情况下应该同时具有以下特性：正交性、局部性（即紧凑支持、平滑滤波器和大量消失矩）和对称性。不幸的是，小波不可能同时拥有所有这些特征。幸运的是，这不是多小波的情况，多小波是（标量）小波的一个相对较新的多通道竞争者。尽管多小波在应用方面的研究还不够充分，但在许多应用中，它们已经被证明比标量小波表现得更好。此外，还有不同类型的多小波可用：可分离的与不可分离的，平衡的与不平衡的，对称对称的与对称反对称的，它们以不同的方式分解图像。本文主要研究了多小波在立体对应中的应用，并对不同类型的多小波进行了客观评价。该方案的最终目标是实现一种混合的、基于多分辨率的视频立体算法，该算法利用了多小波的特性。这项研究的一个直接应用领域将是立体到3D （2D+深度）的转换，以确保我们融合媒体实验室中各种3D设备之间的互操作性。这项研究的另一个同样重要的目标是与我们的工业合作者合作评估其可能的商业化路线，以及在更广泛的研究界传播和促进多小波的使用。

Stereopsis, the process of extracting depth information from stereoscopic images, has multiple practical applications in its discrete form, ranging from entertainment, broadcasting, 3D modelling and visualisation to robotics and automated industrial inspection, just to name a few. Calculating the disparity between the left and right stereo images - aka stereo correspondence matching - is the most challenging stage in stereopsis applications. Biologically, the human brain achieves this task by employing a multichannel, multiresolution mechanism, recognising the fact that natural visual scenes contain detail information in several resolutions. Discrete stereopsis can be carried out both in the spatial domain and the frequency domain. Spatial image processing techniques offer perfect localisation in space but no localisation in frequency. Traditional image transforms achieve perfect localisation in frequency but no (or very poor) spatial localisation. Wavelets were the first modern transform to offer a hierarchical, multiresolution approach, which combines the best of both worlds: a space-frequency hierarchical multiresolution decomposition of the original image, with both local and global spatial support (section 2.3). They rapidly became a popular choice for stereo correspondence. To achieve optimal performance in the context of stereo correspondence, a wavelet should ideally simultaneously posses the following properties: orthogonality, locality (i.e. compact support, smooth filters and a large number of vanishing moments) and symmetry. Unfortunately a wavelet can never possess all these features simultaneously. Fortunately, this is not the case for multiwavelets, a relatively new, multichannel competitor of the (scalar) wavelets. In spite of the fact that application-wise multiwavelets are a rather under-researched topic, they have been already proven to perform better than scalar wavelets in many applications. Furthermore, there are different types of multiwavelets available: separable vs nonseparable, balanced vs unbalanced, symmetric-symmetric vs symmetric-antisymmetric which decompose an image in different ways.This proposal focuses on investigating the use of, and objectively evaluating the different types and families of multiwavelets in the context of stereo correspondence. The end objective of this proposal is to implement a hybrid, multiresolution-based stereo algorithm for video, which takes advantage of the properties exhibited by multiwavelets. An immediate application area of this research would be stereo to 3D (2D+depth) conversion in order to ensure interoperability between the various 3D facilities in our Fused Media Lab. Another equally important objective of this research is to evaluate its possible commercialisation routes in partnership with our industrial collaborators as well as to disseminate and promote the use of multiwavelets in the wider research community.