A study on the conkersion from a sentence speech to a kanji-kana string using phoneme recognition, syntax and semantics processings

我们开发了一个日语听写系统，它可以将未指定的说话者发出的连续语音转换为汉字-假名字符串。该系统由声学处理部分、类bunsetsu单元识别部分和语法语义处理部分组成。我们的目标是识别句法和语义结构在句法和语义上都合理的句子语音。在声学处理部分，输入语音由一个29通道带通滤波器组进行分析。利用时谱模式从短时谱中提取片段特征，并将其转换为音素字符串。在类bunsetsu单位的识别部分，使用语法驱动的连续DP从音素字符串中识别类bunsetsu单位，可以显著减少识别类bunsetsu单位所需的计算量和存储量。在句法和语义处理部分，对词汇项中的类bunsetsu单元候选者给出功能特征，其中功能特征包含句法和语义信息。根据功能特征检查相邻的两个类bunsetsu单元之间连接的可能性，如果满足句法和语义的合理性，则相邻的两个单元组成一个更大的单元，从而最终构成一个完整的句子。当说话人以类bunsetsu单元说出一个句子时，音素识别得分为85%，句子识别得分为60%。音素识别分数为95%，句子识别分数为80%。对句法和语义处理部分的规则进行细化，可以提高句子识别分数。

We have developed a Japanese dictation system which can convert a continuous speech uttered by unspecified speaker to a Kanji-Kana string. The system is composed of an acoustic processing part, a spotting part of Bunsetsu-like units and a syntax and semantic processing part. Our object is to recognize a sentence speech whose syntax and semantic structures are syntactically and semantically reasonable. In the acoustic processing part, an input speech is analized by a 29 channel band-pass filter bank. Segment features are extracted from short-time spectra using time-spectrum patterns and then converted to a phoneme string. In the spotting part of Bunsetsu-like units, Bunsetsu-like units are spotted from a phoneme string using a syntactic driven continuous DP can dominantly reduce the amounts of computation and storage necessary to spot Bunsetsu-like units. Inthe syntactic and semantic processing part, functional features are given to Bunsetsu-like unit candidates from lexical items, where functional features contains syntactic and semantic information. Possibility of concatenation between two adjacent Bunsetsu-like units is checked based on the functional features and then the two adjacent units make a larger unit if satisfying syntactic and semantic reasonability, and thus a complete sentence is finally made.When a speaker utterd a sentence by a Bunsetsu-like unit, 60% of sentence recognition score is obtained in case of 85% phoneme recognition score. 80% of sentence recognition score is obtained in case of 95% phoneme recognition score. Refining the rules in syntactic and semantic processing part can improve the sentence recognition score.