权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

SPEECH RECOGNITION WITH SYNCHRONOUS INPUT OF HAND-WRITTEN GESTURES FOR MOBILE DEVICES

移动设备同步输入手写手势的语音识别

基本信息

批准号：
15300054
负责人：
SHINODA Koichi
金额：
$ 3.78万
依托单位：
Tokyo Institute of Technology
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (B)
财政年份：
2003
资助国家：
日本
起止时间：
2003 至 2004
项目状态：
已结题

项目摘要

Mobile devices have recently been often used in daily life. User-friendly interface with high accuracy has been strongly demanded. For this purpose, we propose an interface using simultaneous inputs of speech and hand-written gestures. This interface is more robust against environmental noise than speech-only interface, and its input speed is faster than the interface with only hand-written gestures. Our target application is e-mail making with the input of sentences.First year, we proposed an interface in which a sentence is input by speech while the "hiragana" character at the head of each phrase in the sentence is input by hand-written gestures. We implemented a recognition algorithm for hand-written gestures, designed a method for recognizing the simultaneous inputs of the two modes. The proposed method was evaluated by simulation experiments using speech data and hand-written gesture data, which are recorded independently, and was proved to be effective.Second year, we constructed a recording system for the input of the two modes, and recorded 530 sentences from ten subjects. For integrating the two modes, we employed a two-pass process in which a word graph generated by speech recognition in the first pass is utilized for the integration process of the two modes in the second pass. The proposed method improved the recognition accuracy by 2.6 point over the method only with speech recognition.For future work, a method for optimizing the weights among the two modes should be developed. We are going to develop a demonstration system which works in real time and evaluate it in noisy environment.

移动的设备近来已经经常用于日常生活中。用户友好的界面，高精度的强烈要求。为此，我们提出了一个接口，同时使用语音和手写手势输入。该接口比纯语音接口对环境噪声的鲁棒性更好，输入速度比纯手写手势接口更快。我们的目标应用程序是电子邮件制作与输入的句子。第一年，我们提出了一个接口，其中一个句子是通过语音输入，而在句子中的每个短语的头部的“平假名”字符是通过手写手势输入。实现了一种手写手势识别算法，设计了一种识别两种模式同时输入的方法。通过对语音和手写手势数据的模拟实验，验证了该方法的有效性。第二年，我们构建了一个语音和手写手势输入的记录系统，记录了10名受试者的530个句子。为了整合这两种模式，我们采用了两遍过程，其中在第一遍中由语音识别生成的词图用于第二遍中的两种模式的整合过程。该方法比单纯语音识别的识别率提高了2.6个百分点，需要进一步研究两种模式间权值的优化方法。我们将开发一个演示系统，它的工作在真实的时间和评估它在嘈杂的环境。