Reconstruction of Complex Deformations in 3D Scenes from Color and Depth Images

从彩色和深度图像重建 3D 场景中的复杂变形

• 批准号: 257332386
• 负责人: Professor Dr.-Ing. Reinhard Koch
• 金额: --
• 依托单位: Institut für Informatik
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2014
• 资助国家: 德国
• 起止时间: 2013-12-31 至 2020-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/257332386?language=en
• 关键词: Reconstruction; Complex; Deformations; 3D; Scenes

While the 3D reconstruction of rigid environments has been investigated very thoroughly in the last two decades, the analysis of deforming scenes is still subject to intense research. Most reconstruction algorithms for deformable objects utilize feature point tracking in a sequence of input images, however, feature tracking requires a sufficient amount of texture or structure in the image as well as a certain level of unambiguity of these features. This does not only apply to color images but also to depth images, since depth based reconstruction requires structural information for a stable tracking result.Since the input data of feature based tracking methods is already the result of an interpretation, a number of disadvantages are inherent to this class of algorithms:1. The reduction of complex image data to a set of feature coordinates does discard a lot of information that can potentially be used to solve the reconstruction problem.2. Optimization does not employ the measurement data directly, which is inaccurate and entails error prone propagation of image noise.3. The existence of outliers in the feature correspondences requires the algorithm to identify wrong information and discard it. This leads to a weaker convergence behavior of the overall optimization problem.This issues can be circumvented by using methods operating directly on the input data: 'Analysis by Synthesis' (AbS) based methods generate a 3D model of the tracked object and use it to synthesize a virtual camera image. The difference between the synthesized images and the real input images is then used to analyze the object parameters. A disadvantage of this system is the need for an extensive global optimization process, since the problem can not be linearized in its object parameters. However, AbS makes use of the complete image data available.The main goal of this research project is to combine the advantages of existing AbS methods and feature based algorithms while maintaining the useful properties of direct optimization. This is going to be achieved by using information about feature movement to provide 'guidance' to the direct optimization process without contaminating the direct optimization goal. In addition to this, an adaptive deformation model will be constructed to ensure versatility while reducing the computational optimization costs.

虽然在过去的二十年里，刚性环境的三维重建已经得到了非常深入的研究，但对变形场景的分析仍然处于紧张的研究阶段。大多数用于可变形对象的重建算法利用输入图像序列中的特征点跟踪，然而，特征跟踪要求图像中具有足够量的纹理或结构以及这些特征具有一定程度的无二义性。这不仅适用于彩色图像，也适用于深度图像，因为基于深度的重建需要结构信息来获得稳定的跟踪结果。由于基于特征的跟踪方法的输入数据已经是解释的结果，因此这类算法存在许多固有的缺点：1.将复杂的图像数据缩减为一组特征坐标确实丢弃了大量可能用于解决重建问题的信息。优化算法没有直接利用测量数据，不准确，容易引起图像噪声的错误传播。特征对应关系中离群点的存在要求算法识别错误信息并将其丢弃。这导致全局优化问题的收敛行为较弱，可以通过使用直接对输入数据进行操作的方法来规避这个问题：基于合成分析(ABS)的方法生成被跟踪对象的3D模型，并使用它来合成虚拟相机图像。然后利用合成图像和实际输入图像之间的差异来分析对象参数。该系统的一个缺点是需要一个广泛的全局优化过程，因为该问题不能在其目标参数中线性化。然而，ABS利用了已有的完整图像数据，本研究的主要目标是结合现有ABS方法和基于特征的算法的优点，同时保持直接优化的有用性质。这将通过使用有关特征移动的信息来为直接优化过程提供“指导”来实现，而不会污染直接优化目标。此外，还将构建自适应变形模型，以确保通用性，同时降低计算优化成本。

项目成果

An Analysis by Synthesis Approach for Automatic Vertebral Shape Identification in Clinical QCT

临床QCT椎体形状自动识别的综合方法分析

• DOI: 10.1007/978-3-030-12939-2_6
• 发表时间: 2019
• 期刊: ArXiv
• 作者: S. Reinhold;T. Damm;L. Huber;R. Andresen;R. Barkmann;C.-C. Glüer;R. Koch
• 通讯作者: R. Koch

Tracking of object deformations in color and depth video: deformation models and applications

• DOI: 10.1117/12.2190780
• 发表时间: 2015-06
• 期刊: Discret. Math.
• 作者: Andreas Jordt;Stefan Reinhold;R. Koch

An Adaptable Robot Vision System Performing Manipulation Actions With Flexible Objects

• DOI: 10.1109/tase.2014.2320157
• 发表时间: 2014-05
• 期刊: IEEE Transactions on Automation Science and Engineering
• 影响因子: 5.6
• 作者: L. Bodenhagen;A. R. Fugl;Andreas Jordt;M. Willatzen;K. A. Andersen;Martin M. Olsen;R. Koch;H. G. Petersen;N. Krüger

Randomly Sparsified Synthesis for Model-Based Deformation Analysis

• DOI: 10.1007/978-3-319-45886-1_12
• 发表时间: 2016-09
• 作者: Stefan Reinhold;Andreas Jordt;R. Koch