Towards an evaluation standard for speech control concepts in real-world scenarios
Towards an evaluation standard for speech control concepts in real-world scenarios
复制标题
建立现实场景中语音控制概念的评估标准
DOI：
10.21437/eurospeech.2003-455
复制
发表时间：
2003
期刊：
影响因子：
--
通讯作者：
Jörg Helbig
中科院分区：
文献类型：
--
作者：
Jens Maase;D. Hirschfeld;U. Koloska;Timo Westfeld;Jörg Helbig
语音控制仍然主要通过统计性能指标（识别率、插入率等）进行评估考虑语音识别器在实验室或人工噪声条件下的性能。所有这些措施没有给出关于语音接口的实际可用性的想法，因为实际方面关注的不仅仅是产品内部语音识别器的操作方面。由于它被认为，没有评价标准，到目前为止，满足语音控制产品的实际要求，本文旨在建立一个开放的设计和评价标准，语音控制概念在现实世界的场景。首先，在可用性实验中评估用户的行为和正常环境条件（典型噪声）。数据记录进行，试图捕捉这些典型的使用要求在一个特殊的语料库（阿波罗语料库）。最后，一组标准的桌面，以及嵌入式语音识别器的性能进行了测试，在这些真实的世界条件下。
Speech control is still mainly evaluated through statistical performance measures (recognition rate, insertion rate, etc.) considering the performance of a speech recognizer under laboratory or artificial noise conditions. All these measures give no idea about the practical usability of a speech interface, since practical aspects concern more than operational aspects of the speech recognizer inside a product. Since it was felt, that no evaluation standard so far fulfills the practical requirements for speech controlled products, this paper aims in the establishment of an open design-and evaluation standard for speech control concepts in real-world scenarios. First, the behaviour of the users and the normal environmental conditions (typical noises) were evaluated in usability experiments. Data recordings were conducted, trying to capture these typical usage requirements in a special corpus (Apollo-corpus). Finally, a set of standard desktop, as well as embedded speech recognizers were tested for their performance under these real world conditions.