Online Scene Reconstruction and Understanding

3D场景是现实世界环境数字化的结果。与图像和视频等二维视觉媒体相比，3D场景具有更丰富的自由视点信息，即使无法从同一Vantage位置看到物体，它们也可以捕获物体之间的空间关系。这使得3D场景表示在需要检索位置和姿态相关信息的广泛应用中是有用的，例如用于自主车辆或移动的增强现实。虽然3D测量技术已经取得了相当大的进步，高效的3D重建算法也取得了显著的进步，但使用当今消费级（便携式）设备捕获的3D场景的精度和质量仍然不能完全令人满意，特别是在场景信息需要不断更新的在线场景中。此外，在许多应用中，环境的低级几何表示（例如，点云）是不够的，使得需要分割和标记算法，其应该对噪声、失真和不完整数据具有鲁棒性。最终，我们希望让代理（人类或机器人）与他们的环境进行交互，这使得有必要分析和建模代理与3D场景中（分割和标记）对象的交互模式。我们的目标是：-通过使用概率公式，在捕获过程中仔细建模所有类型的不确定性，显着提高多传感器原始数据流的在线3D重建的精度和质量。通过利用动态变化的上下文信息来执行鲁棒的在线3D场景分割和标记。同样，概率公式，但此外也将应用机器学习方法。通过开发鲁棒的手部跟踪和手势分类算法，以及通过挖掘大型3D场景和交互记录库来分析交互模式，以进行数据驱动的交互建模。

3D scenes are the result of digitizing real-world environments. In comparison to two-dimensional visual media such as images and videos, 3D scenes carry much richer free view-point information and they can capture spatial relations between objects even if they cannot be seen from the same vantage point. This makes 3D scene representations useful in a wide range of applications where location- and pose-dependent information needs to be retrieved, e.g. for autonomous vehicles or mobile augmented reality. While there have been considerable advances in 3D measurement technology as well as significant progress in efficient 3D reconstruction algorithms, the precision and quality of 3D scenes captured with today s consumer-level (portable) equipment is still not fully satisfying, especially in online scenarios where the scene information needs to be continuously updated. Moreover low-level geometric representations (e.g. point clouds) of an environment are not sufficient in many applications such that segmentation and labeling algorithms are required which should be robust against noise, distortion, and incomplete data. Ultimately we want to let agents (humans or robots) interact with their environment which makes it necessary to analyse and model interaction patterns of the agents with (segmented and labeled) objects in a 3D scene. Our goals are:- To significantly improve the precision and quality of online 3D reconstructions from streams of multi-sensor raw data by using probabilistic formulations which carefully model all types of uncertainties in the capturing process.- To perform robust online 3D scene segmentation and labeling by exploiting dynamically changing context information. Again, probabilistic formulations but in addition also machine learning methods will be applied.- To analyze interaction patterns by developing algorithms for robust hand tracking and gesture classification and by mining large repositories of 3D scenes and interaction records for data driven interaction modeling.