Doctoral Dissertation Research: Socially guided allocation of attention and the memory encoding of spoken language

理解口语的一部分涉及存储语音记忆。这些记忆是一个系统的核心部分，该系统使人类能够像理解语言一样迅速和熟练地理解语言。例如，当听者听到一句话时，他们可以根据句子的发音方式(如情绪、情感等)来理解句子的意思和说话者的意思。对过去经历的记忆引导人们推断出这些含义。但人类不可能以相同的细节存储每一段记忆。一些经历被清晰地记住，而另一些则被遗忘，而另一些则被合并或只被部分储存。一个悬而未决的问题是，记忆中的这些差异是如何出现的。最近的研究表明，记忆编码依赖于说话者和群体，并与通过口语提示的社会特征密切相关。在这个项目中，研究人员调查了口语记忆差异的心理基础。了解不同说话者的认知过程是如何变化的，这对理论和社会都很重要。例如，在法律制度中，这项工作涉及证人言语记忆的可靠性。在医学上，准确的诊断依赖于医生和患者之间的对话。此外，在开发新的人工智能技术时，研究人员需要了解这些过程在人类中是如何工作的，以避免在机器中重新产生偏见。这个博士论文项目测试了这样一个假设，即听者在不同的语境中对演讲的关注程度不同，这导致了记忆的不同。当听者听得越多，记忆被记录得越强烈。这种方法与一种广泛的预期背道而驰，即语音记忆的强度主要取决于听者听到单词或发音的频率。研究人员提出了一个模型，在该模型中，听者的注意力水平是中心，并受说话人的社会特征和演讲语境的影响。对三个主要假设进行了检验。首先，语音识别和记忆编码取决于听众根据他们的声音对说话者的联想和信念。其次，这个过程是由听者的注意力驱动的，而不仅仅是一个发音被听到的频率。第三，引起注意的语言特点并不总是相同的。相反，它们取决于背景。对于每个假设，研究人员进行了两个相应的实验：一个实验侧重于对语音的记忆，另一个实验侧重于对句子意义的记忆。这些实验的结果揭示了人类如何理解和记忆演讲，以及这些过程如何与社会世界互动。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Part of understanding spoken language involves storing memories of speech. These memories are a central part of a system that allows humans to understand language as quickly and adeptly as they do. For example, when a listener hears a sentence, they can understand something about the meaning and the talker depending on how the sentence is spoken (e.g., mood, emotion, etc.). Memories from past experiences lead individuals to infer these meanings. But humans cannot store every memory with the same detail. Some experiences are remembered clearly while others are forgotten, while others are merged or only stored in part. One open question is how these differences in memory emerge. Recent research has shown that memory encoding is talker- and group-dependent and strongly associated with social traits cued through spoken language. In this project, the researchers investigate the psychological underpinnings of differences in memories for spoken language. Understanding how cognitive processes vary across diverse talkers is important for theory and for society. For example, in the legal system, this work relates to the reliability of witnesses’ memory of speech. In medicine, accurate diagnoses rely on conversations between the doctor and the patient. And, in developing new AI technologies, researchers need to understand how these processes work in humans in order to avoid recreating biases in machines. This doctoral dissertation project tests the hypothesis that listeners attend differently to speech in different contexts, and this results in differences in memory. When the listener attends more, the memory is recorded more strongly. This approach diverges from a broad expectation that the strength of memories for speech is mostly determined by how often listeners hear a word or pronunciation. The researchers propose a model in which listeners’ level of attention is central and is mediated by the social traits of the speaker and the context of the speech. Three main hypotheses are tested. First, speech recognition and memory encoding depend on associations and beliefs listeners have about speakers based on their voices. Second, this process is driven by the listener’s attention, not just how often a pronunciation has been heard. Third, the traits of speech that attract attention are not always the same. Instead, they depend on the context. For each hypothesis, the researchers conduct two corresponding experiments: One experiment focusing on memory for the sounds of speech, and the other focusing on memory for the meanings of sentences. The results of the experiments shed light on how humans understand and remember speech and how these processes interact with a social world.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.