SGER: Exploring Universal Acoustic Characterization of Spoken Languages

探索通用的声学特性的口语abstractWe探索一种新的方法来模拟所有人类语言的假设，口语的声音特征可以被一组通用的声学单位，没有直接联系到传统的语音定义。它们对应的模型，称为声学段模型（ASMs），可用于将口语解码成这样的单元的串。这些单元的统计数据及其对应于特定语言的训练集中的话语的共现可以用于构造特征向量以构建用于自动口语识别（LID）的基于向量的语言分类器。对于口语查询，ASM派生的特征向量以类似的方式提取，然后用于区分个别口语。这个集合的ASM可以建立自下而上在一个无监督的方式，并将作为模型的声学字母，以构建声学词典的语音识别和语言识别。在该项目中，我们研究了与UAC相关的三个基本问题，即：（1）建模口语所需的声学单元的声学覆盖率和分辨率;（2）用于口语识别的UAC衍生特征的复杂性和区分能力;以及（3）语言线索与建模口语的UAC单元的关系。这项研究有助于更好地理解人类通过声学和语言线索识别口语，并提供数学建模和计算技术来构建LID系统。我们还打算利用我们的研究成果，在另一个美国国家科学基金会资助的自动语音属性转录（ASAT）模型突出的语音线索的语言特征及其相关性的听觉感知。可用的语言线索的整个集合，包括音素、音节、单词、韵律和词汇线索，也可以被并入这种协同方法中以进行口语建模和识别。

Exploring Universal Acoustic Characterization of Spoken LanguagesAbstractWe explore a novel approach to modeling all human languages by assuming that the sound characteristics of spoken languages can be covered by a universal set of acoustic units with no direct link to conventional phonetic definitions. Their corresponding models, called acoustic segment models (ASMs), can be used to decode spoken utterances into strings of such units. The statistics of these units and their co-occurrences corresponding to utterances in a training set of a particular language can be used to construct feature vectors to build vector-based language classifiers for automatic spoken language identification (LID). For spoken queries, ASM-derived feature vectors are extracted in a similar manner and then used to discriminate individual spoken languages. This collection of ASMs can be established from bottom up in an unsupervised manner, and will serve as models of acoustic alphabets to construct acoustic lexicons for speech recognition and language identification. In the project we study three fundamental issues related to UAC, namely: (1) acoustic coverage and resolution of acoustic units needed to model spoken languages; (2) complexity and discriminative power of UAC-derived features for spoken language identification; and (3) relationship of language cues with UAC units for modeling spoken languages. This research facilitates a better understanding of human identification of spoken languages through acoustic and linguistic cues, and provides mathematical modeling and computing techniques to build LID systems. We also intend to leverage our research results in another NSF grant on automatic speech attribute transcription (ASAT) to model salient speech cues for language characterization and their relevance to auditory perception. The entire collection of available language cues, including phones, syllables, words, prosody, and lexical cues, can also be incorporated into this synergistic approach to spoken language modeling and identification.