Scene Dynamics Prediction using Physically based Simulation (P5)

该项目的第一阶段侧重于静态场景的实时重建和活动地图的创建。为此，使用了几何数据以及关于材料属性的数据。这两个数据源都为服务机器人导航和对象操作等应用程序提供了有价值的信息。第二阶段项目的主要目标是场景动态的预期。计划中的研究包括3D重建，基于物理的模拟，包括实时推断动态环境中的材料参数。为了集中我们的研究，我们首先将自己限制在预测纺织品的动态，这是机器人正确处理纺织品所需的。为此，我们将首先扩展在第一个项目阶段开发的静态场景的实时重建方法，以包括场景动力学和对象的变形。除了重建粗略的几何图形外，我们还计划捕捉纺织品的基本结构，如织物的编织或编织图案，以便得出与动态行为相关的物理材料参数。重点放在可区分的基于物理的织物模拟上。所得到的模拟将与用于有效的3D场景分割的新方法相结合，以便导出符合观测数据并允许预测对象的相应动态的参数化模型。后者使机器人能够将握持纺织品时的预期运动与实际观察进行比较，并在必要时进行校正。

The first phase of the project focused on the real-time reconstruction of static scenes and the creation of activity maps. For this purpose, geometric data as well as data on material properties were used. Both data sources provide valuable information for applications such as service robot navigation and object manipulation. The main goal of the second project phase is the anticipation of scene dynamics. The planned investigations cover 3D reconstruction, physical-based simulation including inference of material parameters in dynamic environments in real time. In order to focus our research, we first limit ourselves to anticipating the dynamics of textiles as required by robots to properly handle textiles. To this end, we will first extend the approaches to real-time reconstruction of static scenes developed in the first project phase to include scene dynamics and the deformation of objects. In addition to the reconstruction of the rough geometry, we also plan to capture the underlying structures of the textiles, such as weaving or knitting patterns of fabrics, in order to derive physical material parameters that are relevant for the dynamic behaviour. The focus is on a differentiable physically based fabric simulation. The resulting simulation is to be combined with new approaches for efficient 3D scene segmentation in order to derive a parametrized model that fits to the observed data and allow for the prediction of the corresponding dynamics of objects. The latter enables e.g. a robot to compare the anticipated movement of a textile when gripping it with real observations and to correct it if necessary.