Auditory Grammar : Organization of Speech and Nonspeech

为了理解人类听觉系统在我们的语言行为中的关键作用，研究听觉系统如何选择和连接感知元素进行组织是很重要的。在本研究项目之前，还没有从这种观点进行过系统的实验。因此，我们利用了与我们发现的新听觉现象相关的语言和非语言材料，如间隙转移错觉和分裂效应。我们进行了听觉演示，进行了感性实验，并分析了语音和节奏产生的时间结构。我们利用音调序列对听觉注意进行了实验。此外，我们还分析了世界语言的音节结构与句法结构之间的关系。结果表明，听觉组织中的听觉事件与言语中的音节具有共性。因此，我们证实了我们关于听觉事件和听觉流感知的假设。在这个假设中，听觉流被认为是听觉事件和沉默的线性连接。更确切地说，听觉流由听觉子事件组成，即开始、偏移、填充和沉默，它们根据简单的语法和邻近原则（格式塔原则之一）线性地相互连接。使用合成的普通话语音，我们证明了属于物理上不同的声音的起始和偏移可以产生对单个普通话音节的错觉。这一发现将我们使用相对简单的声音获得的结果扩展到语言领域。最近，我们正在对听觉系统如何在时间连续变化的声信号中确定和组织离散语言元素的线索进行计算研究。

In order to understand the crucial role of the human auditory system in our linguistic behavior, it is important to investigate how the auditory system selects and connects perceptual elements to be organized. No systematic experiments from such a viewpoint had been conducted before the present research project. Thus, we utilized linguistic and nonlinguistic materials related to new auditory phenomena that we had discovered, such as the gap transfer illusion and the split-off effect. We made auditory demonstrations, conducted perceptual experiments, and analyzed the temporal structures of speech and rhythm production. We conducted experiments on auditory attention employing tone sequences. Furthermore, we analyzed the relationship between syllable structures and syntactic structures of languages of the world. It was revealed that auditory events in auditory organization and syllables in speech have common aspects. Thus, we confirmed our hypothesis on the perception of auditory events and auditory streams. In this hypothesis, an auditory stream is considered as a linear concatenation of auditory events and silences. More exactly, an auditory stream consists of auditory subevents, i.e.,onsets, offsets, fillings, and silences, connected to each other linearly according to simple grammar and the proximity principle, one of the Gestalt principles. Employing synthesized Mandarin speech, we showed that an onset and an offset belonging to physically different sounds can create an illusory perception of a single Mandarin syllable. This finding extends the results we had obtained employing relatively simple sounds to the linguistic domain. Recently, we are conducting computational research on how the auditory system determines and organizes cues of discrete linguistic elements in acoustic signals changing continuously in time.