Deep learning-enabled video-based motion tracking for human-robot interaction in assembly tasks

未来的制造环境将涉及人类与机器人的近距离互动。这些相互作用的优化将需要人类运动的连续记录，以输入控制机器人运动的控制算法。虽然商业运动跟踪系统是可用的，但这些不能实际应用于大多数制造环境，因为这些是昂贵的，并且需要传感器在精确指定的位置附着在身体上。用于从图像中提取模式的现代计算工具可能提供一种可行的替代方法。近年来，机器学习已被用于从人类或动物的身体图像中提取身体部位和身体姿势。当应用于视频（图像序列）时，这种方法有可能在不将传感器附加到身体上的情况下提取3D身体运动。然而，这还没有优化到能够提供上述应用，目前的博士项目将解决该领域的几个突出挑战。例如，它将专注于改进基于视频记录的训练数据，与最先进的全身运动跟踪同步，并利用算法来解释跨帧的运动模式。

Manufacturing environments of the future will involve humans interacting with robots at close distance. Optimisation of these interactions will require continuous recordings of human movements, to feed into the control algorithms governing the robot movements. Although commercial motion tracking systems are available, these cannot practically be applied in most manufacturing environments, since these are costly and require sensors to be attached to the body at precisely specified positions. Modern computational tools for extracting patterns from images may provide a viable alternative. In recent years, machine learning has been used to extract body parts and body postures from images of human or animal bodies. When applied to videos - sequences of images - this method potentially affords extracting 3D body movements without having to attach sensors to the body. This, however, is not yet optimized to afford the aforementioned applications and the current PhD project will address several outstanding challenges in this field. It will for instance focus on improving training data based on video recordings synchronised with state-of-the-art full body motion tracking and appropriating the algorithms to account for motion patterns across frames.