Generalized Assorted Camera Arrays: Robust Cross-Channel Registration and Applications
Generalized Assorted Camera Arrays: Robust Cross-Channel Registration and Applications
复制标题
DOI：
10.1109/tip.2015.2413291
复制
发表时间：
2015-03-01
影响因子：
10.6
通讯作者：
Veeraraghavan, Ashok Narayanan
中科院分区：
文献类型：
--
作者：
Holloway, Jason;Mitra, Kaushik;Veeraraghavan, Ashok Narayanan
用于多模态成像的一种流行技术是广义分类像素（GAP），其中图像传感器上的分类像素阵列允许多模态捕获。不幸的是，GAP在其适用性方面受到限制，因为需要多模态滤波器，该多模态滤波器适合于半导体制造工艺并导致固定的多模态成像配置。在本文中，我们提倡用于多模态成像的广义组合相机（GAC）阵列，即，具有放置在每个相机光圈前面的不同特性的滤波器的相机阵列。与GAP相比，GAC为我们提供了三个明显的优势：易于实现，灵活的应用程序相关成像，因为过滤器是外部的并且可以改变，以及可以用于启用新应用程序的深度信息（例如，捕获后重新聚焦）。GAC阵列中的主要挑战是，由于不同的模态是从不同的视角获得的，因此需要准确且有效的跨通道配准。传统的方法，如平方差和，绝对差和，互信息都导致多模态配准误差。在这里，我们提出了一个强大的跨通道匹配成本函数，对齐归一化梯度的基础上，这使我们能够计算跨通道子像素对应的场景表现出非平凡的几何形状。我们强调了GAC阵列的承诺，我们的跨通道归一化梯度成本的几个应用，如低光成像，捕获后重新聚焦，皮肤灌注成像，使用彩色+近红外，和高光谱成像。
One popular technique for multimodal imaging is generalized assorted pixels (GAP), where an assorted pixel array on the image sensor allows for multimodal capture. Unfortunately, GAP is limited in its applicability because of the need for multimodal filters that are amenable with semiconductor fabrication processes and results in a fixed multimodal imaging configuration. In this paper, we advocate for generalized assorted camera (GAC) arrays for multimodal imaging-i.e., a camera array with filters of different characteristics placed in front of each camera aperture. The GAC provides us with three distinct advantages over GAP: ease of implementation, flexible application-dependent imaging since filters are external and can be changed and depth information that can be used for enabling novel applications (e.g., postcapture refocusing). The primary challenge in GAC arrays is that since the different modalities are obtained from different viewpoints, there is a need for accurate and efficient cross-channel registration. Traditional approaches such as sum-of-squared differences, sum-of-absolute differences, and mutual information all result in multimodal registration errors. Here, we propose a robust cross-channel matching cost function, based on aligning normalized gradients, which allows us to compute cross-channel subpixel correspondences for scenes exhibiting nontrivial geometry. We highlight the promise of GAC arrays with our cross-channel normalized gradient cost for several applications such as low-light imaging, postcapture refocusing, skin perfusion imaging using color + near infrared, and hyperspectral imaging.