Derma: Silent Speech Interaction Using Transcutaneous Motion Sensing
Derma: Silent Speech Interaction Using Transcutaneous Motion Sensing
复制标题
DOI：
10.1145/3458709.3458941
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
J. Rekimoto;Yusuke Nishimura
中科院分区：
文献类型：
--
作者：
J. Rekimoto;Yusuke Nishimura
无声语音交互（SSI）使语音通信不发出实际的声音，并有可能使语音交互在公共场所可用。然而，通常研究的用于SSI的基于图像的唇读需要在面部前面的相机，因此它不适合于移动的使用。超声成像需要昂贵且复杂的设备。相比之下，我们提出了一个更简单的方法，皮肤运动传感。两个小的6-DOF加速度计/角速度传感器连接在下巴下获得由无声话语引起的皮肤运动的12维多维信息。利用神经网络，识别出35种不同的无声命令，识别率为94%。虽然以前的唇读研究通常从有声语音的视频图像推断语音，但本研究还提出了一种完全从非有声语音学习的方法。与以前的研究相比，我们认为所提出的解决方案是不太明显，重量轻，不受光照条件的影响。
Silent speech interaction (SSI) enables speech communication without uttering an actual voice and can have a potential to make speech interaction available in public places. However, commonly studied image-based lip-reading for SSI requires a camera in front of the face and thus it is not suitable for mobile use. Ultrasound imaging requires expensive and complex equipment. In contrast, we propose a much simpler method by skin motion sensing. Two small 6-DOF accelerometer/angular velocity sensors attached under the chin acquire 12-dimensional multidimensional information of skin motion caused by the silent utterance. With neural networks, 35 different silent commands are identified with a recognition rate of 94%. While previous lip-reading studies have normally inferred speech from images of video with vocal speech, this study also proposes a method of learning entirely from non-vocal speech only. Compared to previous studies, we consider the proposed solution is less visible, lightweight, and is not affected by lighting conditions.