GestOnHMD: Enabling Gesture-based Interaction on Low-cost VR Head-Mounted Display
GestOnHMD: Enabling Gesture-based Interaction on Low-cost VR Head-Mounted Display
复制标题
GestOnHMD：在低成本 VR 头戴式显示器上实现基于手势的交互
DOI：
10.1109/tvcg.2021.3067689
复制
发表时间：
2021-05-01
影响因子：
5.2
通讯作者：
Zhu, Kening
中科院分区：
文献类型：
--
作者：
Chen, Taizhou;Xu, Lantian;Zhu, Kening
低成本的虚拟现实（VR）头戴式显示器（hmd）与智能手机的融合，将沉浸式VR带给了大众，并增加了VR的普遍性。然而，这些系统往往受到交互性差的限制。在本文中，我们介绍了GestOnHMD，这是一种基于手势的交互技术和手势分类管道，它利用商品智能手机中的立体声麦克风来检测移动VR头戴式耳机的正面，左侧和右侧表面上的敲击和刮擦手势。以谷歌Cardboard作为我们的重点耳机，我们首先进行了手势激发研究，生成150个用户定义的手势，每个手势表面上有50个手势。然后，我们根据用户偏好和信号可检测性分别为前、左、右表面选择了15、9和9个手势。我们构建了一个包含18个用户执行这些地面手势的声学信号的数据集，并训练了用于手势检测和识别的深度学习分类管道。最后，通过实时演示GestOnHMD，我们进行了一系列在线参与式设计会议，以收集一组可能受益于GestOnHMD的用户定义手势参考映射。
Low-cost virtual-reality (VR) head-mounted displays (HMDs) with the integration of smartphones have brought the immersive VR to the masses, and increased the ubiquity of VR. However, these systems are often limited by their poor interactivity. In this paper, we present GestOnHMD, a gesture-based interaction technique and a gesture-classification pipeline that leverages the stereo microphones in a commodity smartphone to detect the tapping and the scratching gestures on the front, the left, and the right surfaces on a mobile VR headset. Taking the Google Cardboard as our focused headset, we first conducted a gesture-elicitation study to generate 150 user-defined gestures with 50 on each surface. We then selected 15, 9, and 9 gestures for the front, the left, and the right surfaces respectively based on user preferences and signal detectability. We constructed a data set containing the acoustic signals of 18 users performing these on-surface gestures, and trained the deep-learning classification pipeline for gesture detection and recognition. Lastly, with the real-time demonstration of GestOnHMD, we conducted a series of online participatory-design sessions to collect a set of user-defined gesture-referent mappings that could potentially benefit from GestOnHMD.