HybridTrak: Adding Full-Body Tracking to VR Using an Off-the-Shelf Webcam
HybridTrak: Adding Full-Body Tracking to VR Using an Off-the-Shelf Webcam
复制标题
DOI：
10.1145/3491102.3502045
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
Jackie Yang;Tuochao Chen;Fang Qin;M. Lam;J. Landay
中科院分区：
文献类型：
--
作者：
Jackie Yang;Tuochao Chen;Fang Qin;M. Lam;J. Landay
虚拟现实中的全身跟踪提高了临场感，允许通过身体姿势进行交互，并促进了用户之间更好的社交表达。然而，今天的全身跟踪系统需要与环境固定的复杂设置(例如，多个灯塔/摄像头)和繁琐的校准过程，这与使VR系统更便携和集成的愿望背道而驰。我们推出了Hybridge Trak，它通过使用单个外部现成的RGB网络摄像头来增强由内到外的上半身VR跟踪系统，从而提供准确、实时的全身跟踪。Hybridge Trak使用全神经解决方案，利用由内向外的上半身跟踪数据，将用户的2D全身姿势从网络摄像头转换为3D姿势。我们在MPI-INF-3dHP数据集上显示了Hybridge Trak比RGB或基于深度的跟踪方法更准确。我们还在流行的VRChat应用程序中测试了HyBridge Trak，结果显示，HyBridge Trak呈现的身体姿势比使用RGBD摄像头的解决方案更容易辨别，也更自然。
Full-body tracking in virtual reality improves presence, allows interaction via body postures, and facilitates better social expression among users. However, full-body tracking systems today require a complex setup fixed to the environment (e.g., multiple lighthouses/cameras) and a laborious calibration process, which goes against the desire to make VR systems more portable and integrated. We present HybridTrak, which provides accurate, real-time full-body tracking by augmenting inside-out1 upper-body VR tracking systems with a single external off-the-shelf RGB web camera. HybridTrak uses a full-neural solution to convert and transform users’ 2D full-body poses from the webcam to 3D poses leveraging the inside-out upper-body tracking data. We showed HybridTrak is more accurate than RGB or depth-based tracking methods on the MPI-INF-3DHP dataset. We also tested HybridTrak in the popular VRChat app and showed that body postures presented by HybridTrak are more distinguishable and more natural than a solution using an RGBD camera.