Large-vocabulary Semantic Image Processing: Theory and Algorithms

经典的图像处理大多忽略了语义图像表示，有利于更数学上易于处理的表示基于低？]电平信号特性（频率分解、均方误差等）。这与图像处理问题的生物解决方案不同，后者广泛依赖于对场景内容的理解。例如，人脸区域通常比背景中的灌木丛处理得更仔细。无法将图像处理调整到图像内容的语义相关性经常导致子？]最佳的资源分配，如带宽，错误保护，或观看时间，以图像领域的感知无关。部署语义图像处理系统的主要障碍之一是训练内容的困难。理解具有大规模词汇表的系统。这在很大程度上是由于传统的词汇学习方法需要大量的训练数据和密集的人工监督。本研究的目的是建立一个语义图像处理系统，可以学习大规模的词汇非正式标注的数据和没有额外的人的监督的基础。它建立在语义图像标记的最新进展之上，这使得从嘈杂的训练数据中学习词汇成为可能，例如在网络上大量（且廉价）可用的数据。这项研究既研究了词汇学习的理论问题，也研究了图像处理算法的设计，这些算法根据被处理图像的内容来调整它们的行为。语义图像处理可能会导致图像压缩、增强、加密、去重等领域的变革性进步。噪声、或分割等，这些都是医学成像、图像搜索和检索、或安全和监视等不同应用所关注的。

Classical image processing has mostly disregarded semantic image representations, in favor of more mathematically tractable representations based on low?]level signal properties (frequency decompositions, mean squared error, etc.). This is unlike biological solutions to image processingproblems, which rely extensively on understanding of scene content. For example, regions of faces are usually processed more carefully than the bushes in the background. The inability to tune image processing to the semantic relevance of image content frequently leads to the sub?]optimal allocation ofresources, such as bandwidth, error protection, or viewing time, to image areas that are perceptually irrelevant. One of the main obstacles to the deployment of semantic image processing systems has been the difficulty of training content?]understanding systems with large scale vocabularies. This is, in great part, due to the requirement for large amounts of training data and intensive human supervision associated with the classical methods for vocabulary learning. This research aims to establish a foundation for semantic image processing systems that can learn large scale vocabularies frominformally annotated data and no additional human supervision. It builds on recent advances in semantic image labeling, which have made it possible to learn vocabularies from noisy training data, such as that massively (and inexpensively) available on the web. The research studies both theoreticalissues in vocabulary learning, and the design of image processing algorithms that tune their behavior according to the content of the images being processed. Semantic image processing could lead to transformative advances in areas such as image compression, enhancement, encryption, de?]noising, orsegmentation, among others, which are of interest for applications as diverse as medical imaging, image search and retrieval, or security and surveillance.