权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Multimodal sparsity models for segmentation of visual objects

用于视觉对象分割的多模态稀疏模型

基本信息

批准号：
263894508
负责人：
Professor Dr. Daniel Cremers
金额：
--
依托单位：
Lehrstuhl für Datenverarbeitung (LDV)
依托单位国家：
德国
项目类别：
Research Grants
财政年份：
2015
资助国家：
德国
起止时间：
2014-12-31 至 2020-12-31
项目状态：
已结题

来源：
https://gepris.dfg.de/gepris/projekt/263894508?language=en
关键词：
Multimodal sparsity models segmentation visual

项目摘要

Automatic segmentation of objects in a 3D scene is a fundamental problem in Computer Vision. It is a core aspect for many high-level algorithms including object recognition, semantic scene analysis and 3D object reconstruction. To date unsupervised segmentation faces many challenges such as complex texture and lighting variations in real-world scenes. The proposed project is based on the reasonable assumption that an object is determined by characteristic properties of signals captured through different modalities. We intend to demonstrate that a rigorous mathematical derivation of multimodal segmentation approaches will lead to drastic improvements in unsupervised segmentation ultimately allowing for a robust and precise segmentation of physical objects in the scene. More specifically we will focus on the following challenges: - Inspired by biological systems which perceive their environment through many different signal modalities at once, we intend to devise algorithms to fuse sensory information from different modalities in order to drastically enhance the performance of unsupervised segmentation methods. - Unlike existing fusion schemes that combine information on higher processing levels, we will focus on fusion across several modalities on the signal level.- We will generalize existing techniques for sparse representation and inference from the unimodal setting to the multimodal setting and demonstrate that an accurate modeling of sparsity and interdependency of the multiple channels allows for much better discrimination of objects of interest. Our methods will be general enough to work on different types of visual data and their cross-modal dependencies, without the need of hand-crafted signal-specific features. We investigate sparse signal representations in general and focus on the so-called co-sparse analysis model in particular. - We will derive variational segmentation algorithms which exploit multimodal sparsity for unsupervised object segmentation, thereby combining multimodal sparsity with powerful convex regularization methods.- We will develop a demonstrator which generates semantic 3D segmentations from multimodal data taken from multiple views of a scene. We hope to demonstrate that the proposed method is robust to noise and challenging lighting conditions of real-world environments.

三维场景中物体的自动分割是计算机视觉中的一个基本问题。它是物体识别、语义场景分析、三维物体重建等高级算法的核心。迄今为止，无监督分割面临着许多挑战，如复杂的纹理和光照变化的现实世界的场景。拟议的项目是基于一个合理的假设，即一个对象是由通过不同的方式捕获的信号的特征属性确定。我们打算证明，多模态分割方法的严格数学推导将导致无监督分割的大幅改善，最终允许在场景中的物理对象的鲁棒性和精确的分割。更具体地说，我们将专注于以下挑战：-受生物系统的启发，通过许多不同的信号模态一次感知它们的环境，我们打算设计算法来融合来自不同模态的感官信息，以大幅提高无监督分割方法的性能。- 与现有的融合方案不同，联合收割机在更高的处理水平上结合信息，我们将专注于信号水平上的几种模态的融合。我们将推广现有的技术，从单峰设置的多模态设置的稀疏表示和推理，并证明了一个精确的建模的稀疏性和相互依赖的多个通道允许更好地区分感兴趣的对象。我们的方法足够通用，可以处理不同类型的视觉数据及其跨模态依赖关系，而不需要手工制作特定于信号的功能。我们调查稀疏信号表示一般集中在所谓的共同稀疏分析模型。- 我们将推导出变分分割算法，该算法利用多模态稀疏性进行无监督对象分割，从而将多模态稀疏性与强大的凸正则化方法相结合。-我们将开发一个演示生成语义3D分割从多模态数据从一个场景的多个视图。我们希望证明所提出的方法对真实世界环境中的噪声和具有挑战性的光照条件具有鲁棒性。