A Survey of Facial Capture for Virtual Reality
A Survey of Facial Capture for Virtual Reality
复制标题
虚拟现实面部捕捉调查
作者：
Lihang Wen;Jianlong Zhou;Weidong Huang;Fang Chen
星星战争中的全息图激发了许多研究人员实时捕捉整个人体并将其作为虚拟现实（VR）的化身。面部捕捉对于实现这一点很重要，因为面部表情对于社交互动至关重要。然而，面部捕捉只有在面部没有被遮挡时才能很好地工作。虽然VR头戴式耳机已被广泛用作VR的显示设备，但它遮挡了一半的脸，成为障碍。本文介绍了一个系统的文献综述的面部捕捉VR。调查的目的是：回顾当前最先进的VR面部捕捉技术;确定技术类型和使用背景、方法和理论基础;确定VR面部捕捉的研究差距;并确定VR头显面部捕捉的解决方案。定义了一个真实性指数来评价和比较所收集的论文。结果显示了VR面部捕获的几个技术趋势：使用标记跟踪面部运动，使用相机或传感器在耳机中进行面部捕获，面部性能捕获以及全息图/体积捕获。结果表明，模块化编解码器头像是VR头显中最好的面部捕捉方法，而超人类具有最好的输出效果。本文还提出，开放式设计的VR头显是降低总拥有成本（TCO）的有效途径。
The holograms in Star Wars have inspired many researchers to capture the whole human body in real-time and present it as an avatar into Virtual Reality (VR). Facial capture is important to achieve this because facial expressions are essential for social interaction. However, facial capture works well only when the face is not occluded. While a VR headset has been widely used as a display device for VR, it occludes half of the face and becomes an obstacle. This paper presents a systematic literature review on facial capture for VR. The survey aims to: review the current state-of-the-art facial capture technologies for VR; identify the types of technologies and context of the use, the methodologies, and theoretical groundings; identify research gaps in facial capture for VR; and identify solutions of facial capture for VR headsets. A realism index is defined to evaluate and compare the collected papers. The results show several technology trends in the facial capture for VR: tracking facial motion with markers, facial capture in headsets using cameras or sensors, facial performance capture, and hologram/volumetric capture. It is shown that the Modular Codec Avatar is the best facial capture method in a VR headset, whereas Metahuman has the best output effects. This paper also proposes that an open-design VR headset is an effective approach to lower the Total Cost of Ownership (TCO).