CAREER: The Time Course of Bottom-up and Top-down Integration in Language Understanding

语境改变了我们解释视觉和声音的方式。介于黄色和绿色之间的色调应用于香蕉图片时看起来更黄，但当应用于柠檬时看起来更绿。“p”和“b”之间的声学模式被解释为“sto-”之后的“p”，但“sta-”之后的“b”。但是，语境实际上改变了人们对视觉和声音的感知，还是仅仅改变了它们的解释呢？长期以来，认知科学家一直在争论“自下而上”的输入信号(如语音)何时以及如何与“自上而下”的信息(语境或记忆中的知识)相结合。早期的感知过程是否保护信号的“正确”、与上下文无关的记录，或者感知过程是否立即混合了自下而上和自上而下的信息？一种观点认为，准确的认知需要及早分离自下而上和自上而下的信息，并晚些时候整合。另一种选择是，及早混合自下而上和自上而下的信息将使系统更有效率，因为它允许上下文立即指导处理。在语言理解的研究中，这一时间问题尚未得到解决，因为来自两种瞬间加工测量的证据相互矛盾。根据口头指令跟踪人们在物体上的眼球运动的研究支持即时整合：有用的信息似乎一有用处就会被使用。使用事件相关电位(事件相关电位，通过头皮电极测量皮质活动)的研究表明整合延迟：早期的大脑反应似乎只受自下而上信息的影响。这两种方法的结果很难比较，因为它们依赖于截然不同的实验设计。在这项拟议的研究中，研究者将使用允许同时比较眼神追踪和事件相关电位的实验设计来研究人类句子加工中自上而下整合的时机，目的是确定自上而下语境与自下而上信号信息何时以及如何整合。这项工作对语言技术的设计具有重要的意义。与计算机系统相比，人类高效地利用自上而下的环境，并迅速学会适应新的环境。让计算机系统像人类一样具有适应性的一个障碍是，我们没有完全理解人类如何平衡自下而上的信号和自上而下的环境。这项拟议的研究对理解和治疗语言障碍也有启发意义。例如，了解正常的感知者如何平衡和整合信号和语境，可能有助于识别导致不寻常地依赖语境的细微的自下而上的损伤。研究人员致力于将研究和培训活动整合到这个职业项目中，并将积极让本科生和研究生参与研究。研究人员还将开发课程，通过提供认知理论和时间课程方法论的实践培训，为学生的独立研究做好准备。

Context changes the way we interpret sights and sounds. A shade of color halfway between yellow and green looks more yellow when applied to a picture of a banana, but more green when applied to a lime. An acoustic pattern halfway between "p" and "b" is interpreted as "p" following "sto-" but as "b" following "sta-". But does context actually alter perception of sights and sounds, or only their interpretation? Cognitive scientists have long debated when and how "bottom-up" input signals (such as speech) are integrated with "top-down" information (context, or knowledge in memory). Do early perceptual processes protect a "correct," context-independent record of signals, or do perceptual processes immediately mix bottom-up and top-down information? One view is that accurate perception requires early separation of bottom-up and top-down information and late integration. An alternative is that early mixing of bottom-up and top-down information would make systems more efficient, by allowing context to immediately guide processing. In studies of language comprehension, this timing question is unsettled because of conflicting evidence from two measures of moment-to-moment processing. Studies tracking people's eye movements on objects upon verbal instructions support immediate integration: helpful information appears to be used as soon as it is available. Studies using ERPs (event related potentials, which measure cortical activity via scalp electrodes) suggest delayed integration: early brain responses appear to be affected only by bottom-up information. Results from the two measures have been difficult to compare because they have relied on very different experimental designs. In the proposed research the investigator will study the timing of top-down integration in human sentence processing using experimental designs that allow simultaneous comparisons of eyetracking and ERPs, with the goal of determining when and how top-down context is integrated with bottom-up signal information. The proposed work has important implications for the design of language technology. In contrast to computer systems, humans efficiently exploit top-down context, and quickly learn to adapt to new contexts. An obstacle to making computer systems as adaptable as humans is that we do not fully understand how humans balance bottom-up signals and top-down context. The proposed research also has implications for understanding and treating language impairments. For example, understanding how normal perceivers balance and integrate signal and context may help identify subtle bottom-up impairments that lead to unusual reliance on context. The investigator is committed to integrating research and training activities in this CAREER project, and will actively involve undergraduate and graduate students in the research. The investigator will also develop courses designed to prepare students for independent research by providing hands-on training in cognitive theories and time course methodologies.