Mobile. Egocentric Human Body Motion Reconstruction Using Only Eyeglasses-mounted Cameras and a Few Body-worn Inertial Sensors
Mobile. Egocentric Human Body Motion Reconstruction Using Only Eyeglasses-mounted Cameras and a Few Body-worn Inertial Sensors
复制标题
DOI：
10.1109/vr50410.2021.00087
复制
发表时间：
2021-03
期刊：
影响因子：
--
通讯作者：
Young-Woon Cha;Husam Shaik;Qian Zhang;Fan Feng;A. State;A. Ilie;H. Fuchs
中科院分区：
文献类型：
--
作者：
Young-Woon Cha;Husam Shaik;Qian Zhang;Fan Feng;A. State;A. Ilie;H. Fuchs
我们设想一种方便的远程呈现系统，用户可以在任何地点、任何时间使用。这样一个系统需要将显示器和传感器嵌入到常见的佩戴物品中，比如眼镜、手表和鞋子。为此，我们提出了一种独立的实时系统，用于对人进行动态3D捕捉，该系统仅依靠嵌入头戴式设备中的摄像头以及佩戴在手腕和脚踝上的惯性测量单元（IMU）。我们的原型系统通过基于学习的姿态估计以自我为中心重建佩戴者的运动，这种姿态估计融合了相互补充的视觉和惯性传感器的输入，克服了诸如头戴式视角中肢体可见性不一致以及稀疏IMU导致的姿态模糊等挑战。估计出的姿态会持续重新定位到预先扫描的表面模型上，从而实现高保真的3D重建。我们通过重建各种人体运动来展示我们的系统，并表明我们这种基于视觉 - 惯性学习的实时运行方法，优于仅使用视觉和仅使用惯性的方法。我们采集了一个以自我为中心的视觉 - 惯性3D人体姿态数据集，可在https://sites.google.com/site/youngwooncha/egovip公开获取，用于训练和评估类似方法。
We envision a convenient telepresence system available to users anywhere, anytime. Such a system requires displays and sensors embedded in commonly worn items such as eyeglasses, wristwatches, and shoes. To that end, we present a standalone real-time system for the dynamic 3D capture of a person, relying only on cameras embedded into a head-worn device, and on Inertial Measurement Units (IMUs) worn on the wrists and ankles. Our prototype system egocentrically reconstructs the wearer's motion via learning-based pose estimation, which fuses inputs from visual and inertial sensors that complement each other, overcoming challenges such as inconsistent limb visibility in head-worn views, as well as pose ambiguity from sparse IMUs. The estimated pose is continuously re-targeted to a prescanned surface model, resulting in a high-fidelity 3D reconstruction. We demonstrate our system by reconstructing various human body movements and show that our visual-inertial learning-based method, which runs in real time, outperforms both visual-only and inertial-only approaches. We captured an egocentric visual-inertial 3D human pose dataset publicly available at https://sites.google.com/site/youngwooncha/egovip for training and evaluating similar methods.