Doctoral Dissertation Research: Socially guided allocation of attention and the memory encoding of spoken language

理解口语的一部分涉及到存储语音记忆。这些记忆是一个系统的核心部分，使人类能够快速而熟练地理解语言。例如，当听众听到一个句子时，他们可以根据句子的表达方式来理解句子的含义和说话者（例如，情绪、情感等）。来自过去经历的记忆引导个体推断这些含义。但是人类不可能把所有的记忆都储存在同一个细节上。有些经历被清楚地记住，而另一些则被遗忘，而另一些则被合并或仅部分存储。一个悬而未决的问题是这些记忆差异是如何出现的。最近的研究表明，记忆编码是说话者和群体依赖性的，并与通过口语提示的社会特征密切相关。在这个项目中，研究人员调查了口语记忆差异的心理基础。了解不同谈话者的认知过程如何变化，对理论和社会都很重要。例如，在法律的系统中，这项工作涉及证人对言语记忆的可靠性。在医学上，准确的诊断依赖于医生和病人之间的对话。而且，在开发新的人工智能技术时，研究人员需要了解这些过程在人类中是如何工作的，以避免在机器中重现偏见。 这个博士论文项目测试的假设，听众出席不同的演讲在不同的情况下，这导致记忆的差异。当听众听得越多，记忆就被记录得越强烈。这种方法偏离了一个广泛的期望，即语音记忆的强度主要取决于听众听到单词或发音的频率。研究人员提出了一个模型，在这个模型中，听众的注意力水平是中心，并由说话者的社会特征和演讲的上下文介导。三个主要假设进行了测试。首先，语音识别和记忆编码取决于听众根据声音对说话者的联想和信念。其次，这个过程是由听者的注意力驱动的，而不仅仅是一个发音被听到的频率。第三，吸引注意力的言语特征并不总是相同的。相反，它们取决于上下文。对于每一个假设，研究人员进行了两个相应的实验：一个实验侧重于对语音的记忆，另一个侧重于对句子含义的记忆。实验结果揭示了人类如何理解和记忆语音，以及这些过程如何与社会世界互动。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Part of understanding spoken language involves storing memories of speech. These memories are a central part of a system that allows humans to understand language as quickly and adeptly as they do. For example, when a listener hears a sentence, they can understand something about the meaning and the talker depending on how the sentence is spoken (e.g., mood, emotion, etc.). Memories from past experiences lead individuals to infer these meanings. But humans cannot store every memory with the same detail. Some experiences are remembered clearly while others are forgotten, while others are merged or only stored in part. One open question is how these differences in memory emerge. Recent research has shown that memory encoding is talker- and group-dependent and strongly associated with social traits cued through spoken language. In this project, the researchers investigate the psychological underpinnings of differences in memories for spoken language. Understanding how cognitive processes vary across diverse talkers is important for theory and for society. For example, in the legal system, this work relates to the reliability of witnesses’ memory of speech. In medicine, accurate diagnoses rely on conversations between the doctor and the patient. And, in developing new AI technologies, researchers need to understand how these processes work in humans in order to avoid recreating biases in machines. This doctoral dissertation project tests the hypothesis that listeners attend differently to speech in different contexts, and this results in differences in memory. When the listener attends more, the memory is recorded more strongly. This approach diverges from a broad expectation that the strength of memories for speech is mostly determined by how often listeners hear a word or pronunciation. The researchers propose a model in which listeners’ level of attention is central and is mediated by the social traits of the speaker and the context of the speech. Three main hypotheses are tested. First, speech recognition and memory encoding depend on associations and beliefs listeners have about speakers based on their voices. Second, this process is driven by the listener’s attention, not just how often a pronunciation has been heard. Third, the traits of speech that attract attention are not always the same. Instead, they depend on the context. For each hypothesis, the researchers conduct two corresponding experiments: One experiment focusing on memory for the sounds of speech, and the other focusing on memory for the meanings of sentences. The results of the experiments shed light on how humans understand and remember speech and how these processes interact with a social world.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.