Scene Dynamics Prediction using Physically based Simulation (P5)

点击翻译按钮获取中文摘要

The first phase of the project focused on the real-time reconstruction of static scenes and the creation of activity maps. For this purpose, geometric data as well as data on material properties were used. Both data sources provide valuable information for applications such as service robot navigation and object manipulation. The main goal of the second project phase is the anticipation of scene dynamics. The planned investigations cover 3D reconstruction, physical-based simulation including inference of material parameters in dynamic environments in real time. In order to focus our research, we first limit ourselves to anticipating the dynamics of textiles as required by robots to properly handle textiles. To this end, we will first extend the approaches to real-time reconstruction of static scenes developed in the first project phase to include scene dynamics and the deformation of objects. In addition to the reconstruction of the rough geometry, we also plan to capture the underlying structures of the textiles, such as weaving or knitting patterns of fabrics, in order to derive physical material parameters that are relevant for the dynamic behaviour. The focus is on a differentiable physically based fabric simulation. The resulting simulation is to be combined with new approaches for efficient 3D scene segmentation in order to derive a parametrized model that fits to the observed data and allow for the prediction of the corresponding dynamics of objects. The latter enables e.g. a robot to compare the anticipated movement of a textile when gripping it with real observations and to correct it if necessary.