MagTrack: A Wearable Tongue Motion Tracking System for Silent Speech Interfaces.
MagTrack: A Wearable Tongue Motion Tracking System for Silent Speech Interfaces.
复制标题
MagTrack：用于无声语音接口的可穿戴舌头运动跟踪系统。
DOI：
10.1044/2023_jslhr-22-00319
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Wang,Jun
中科院分区：
文献类型：
--
作者：
Cao,Beiming;Ravi,Shravan;Sebkhi,Nordine;Bhavsar,Arpan;Inan,OmerT;Xu,Wen;Wang,Jun
目的目前的电磁舌头跟踪设备不适合日常使用，因此不适合无声语音接口和其他应用。我们最近开发了MagTrack，这是一种新型的可穿戴电磁关节记录仪舌头跟踪设备。本研究旨在验证MagTrack在无声语音接口中的潜在应用。方法我们进行了两个实验：(A)对8个孤立的辅音-元音-辅音形式的元音进行分类；(B)连续无声语音识别。在这些实验中，我们使用了通过MagTrack收集的健康成年说话者的数据。元音分类的性能通过准确率来衡量。用音素错误率来衡量连续无声语音识别。结果利用所有的MagTrack信号(x，y，z坐标；方位和磁信号)，使用MagTrack进行孤立元音分类的平均准确率达到89.74%，超过了我们先前研究中使用商业电磁关节图数据(只有lyy，z坐标)的准确率。用MagTrack对两名受试者进行连续语音识别，音素错误率分别为73.92%和66.73%。商用电磁关节图对同一受试者的准确率为64.53%(使用MagTrack数据为66.73%)。结论在使用相同的定位信息时，MagTrack与商用电磁关节图的结果具有可比性。添加原始磁信号将改善MagTrack的性能。我们的初步测试证明了无声语音接口作为一种轻量级可穿戴设备的潜力。这项工作还为支持MagTrack在其他应用方面的潜力奠定了基础，包括基于视觉反馈的言语治疗和第二语言学习。
PurposeCurrent electromagnetic tongue tracking devices are not amenable for daily use and thus not suitable for silent speech interface and other applications. We have recently developed MagTrack, a novel wearable electromagnetic articulograph tongue tracking device. This study aimed to validate MagTrack for potential silent speech interface applications.MethodWe conducted two experiments: (a) classification of eight isolated vowels in consonant–vowel–consonant form and (b) continuous silent speech recognition. In these experiments, we used data from healthy adult speakers collected with MagTrack. The performance of vowel classification was measured by accuracies. The continuous silent speech recognition was measured by phoneme error rates. The performance was then compared with results using data collected with commercial electromagnetic articulograph in a prior study.ResultsThe isolated vowel classification using MagTrack achieved an average accuracy of 89.74% when leveraging all MagTrack signals (x,y,zcoordinates; orientation; and magnetic signals), which outperformed the accuracy using commercial electromagnetic articulograph data (onlyy,zcoordinates) in our previous study. The continuous speech recognition from two subjects using MagTrack achieved phoneme error rates of 73.92% and 66.73%, respectively. The commercial electromagnetic articulograph achieved 64.53% from the same subject (66.73% using MagTrack data).ConclusionsMagTrack showed comparable results with the commercial electromagnetic articulograph when using the same localized information. Adding raw magnetic signals would improve the performance of MagTrack. Our preliminary testing demonstrated the potential for silent speech interface as a lightweight wearable device. This work also lays the foundation to support MagTrack's potential for other applications including visual feedback–based speech therapy and second language learning.