Experimental analysis of auditory and gestural interface

本研究的目的是探讨多模态界面的可用性，使用系统的音乐音调和手势。在实验1中，三个participantsmatched的声音刺激的三个要素的音乐音调;音高，节奏，和方向的声音。结果表明，音高匹配的准确率最高，节奏匹配的准确率次之，方向匹配的准确率最低。匹配的顺序是（1）节奏，（2）音高和（3）方向。还讨论了元素的组合。有了这个结果，我们可以制作听觉界面，将用户的注意力引向最紧急的信息，并使其能够以必要的准确性被接收。在实验2和实验3中，还考察了使用同步恒声保持呈现多维信息的听觉条件。在此条件下进行了两个实验，在迷宫导航任务（实验2）和一个移动的PC模拟器（实验3）。在此条件下，执行时间比无声条件（实验2）短。第二次使用系统时，被试的操作错误发生率降低（实验3），说明这种听觉条件促进了被试对系统的学习。实验4和实验5中，24名被试操作数据库模拟器。手势指令条件下的执行时间比指向和快捷键条件下的执行时间短。最后，讨论了这些结果对听觉和手势接口的影响。

The purpose of this research is to examine the usability of multimodal interface using systematic musical tones and gesture. In experiment 1, three participantsmatched the sound stimulus on three elements of musical tones ; pitch, rhythm, and the direction of the sound. The result showed that the accuracy of matching was the highest in the pitch, the second in rhythm, and the last in the direction. The order of matching is (1) rhythm, (2) pitch and (3) direction. The combinations of the elements are also discussed. With this result, we can make auditory interfaces that lead users' attention to the most urgent information, and enable it to be received with the necessary accuracy. In experiment 2 and 3, an auditory condition that uses syncbronic constant tones keep presented multi-dimensional information was also examined. Two experiments were conducted with this condition, in maze-navigation task (experiment 2) and a mobile PC simulator (experiment 3). Under this condition, the performing time was shorter than no-sound condition (experiment 2). The frequency of operating errors was less when participants used the system the second time (experiment 3).The result indicates that subjects' learning about the systems are promoted by this auditory condition. In experiment 4 and 5, 24 participants operated a simulator of data base. Under gestural command condition, the performing time was shorter than pointing and short-cut key conditions. Finally, the implications of these results for auditory and gestural interfaces were discussed.