Computational Techniques for Exploring Language in the Brain

人们希望与他们的计算机、平板电脑和智能手机进行直观的交互;越来越多的人希望让他们的设备执行任务，而不是点击命令。苹果的Siri和微软的Cortana是最近实现这一目标的两个尝试。我们可以让Siri设置闹钟，给母亲打电话，以及其他许多日常任务。然而，当涉及到更复杂的请求，需要更深入地理解自然语言（例如“叫救护车”），计算机就不够用了，让消费者失望。从长远来看，我们需要能够像人类一样理解语言的计算机程序。这是我研究计划的首要目标：推动计算自然语言理解的边界。具体来说，我将使用大脑图像来研究语言“体内”。 为了提高计算机的语言理解能力，我将在人们阅读不同文本片段时收集大脑图像。这些实验的目的将是研究人们如何将联合收割机的话产生复杂的意义，是超过“总和的部分”。例如，当人们读到“happy accident”这个词时，积极和消极的词不会平均成一个中性短语;相反，这个短语本身也是积极的。大脑如何处理具有不同含义的单词，以创建一个统一的含义？我将从短语、句子和段落的层面来研究这个过程，称为语义合成。 当人们孤立地阅读单词时（正如大脑成像所广泛研究的那样），与语言相关的认知过程的开始是可预测的和稳定的。然而，随着单词被组合成更复杂的语言，认知负荷增加，导致语言相关过程的开始与刺激（单词）的开始不那么“时间锁定”。我们需要新的机器学习方法来处理大脑图像中的这种变化。 脑图像数据集规模小，但维数高，这使得很难建立鲁棒的模型。然而，如果我们能够将不同人群的数据联合收割机结合起来，那么可用的数据将增加许多倍。我将创建新的机器学习算法，使用不同人的大脑图像来建立已知模式的“字典”。我将根据大脑活动的相关性来指导，使这些词典适合特定的人。 检测和描述语言理解的更复杂方面对于构建人们如何表示和操纵意义的连贯画面至关重要。我的研究将为计算语言理解的新算法提供信息，并为多学科脑成像数据集中固有的一些问题提供解决方案。我的研究的影响将超越健康大脑中的语言，并改善多种大脑疾病的诊断和治疗，包括语言障碍，睡眠障碍甚至痴呆症。

People want intuitive interactions with their computers, tablets and smartphones; increasingly, they want to ask their devices to perform tasks, not tap and click commands. Apple's Siri and Microsoft's Cortana are two recent attempts to realize this goal. We can ask Siri to set an alarm, to place a call to our mother, and many other everyday tasks. However, when it comes to more complex requests requiring a deeper understanding of natural language (e.g. “Call me an ambulance”), computers fall short, leaving the consumer disappointed. In the long term, we are in need of computer programs that can understand language as humans understand it. This is the overarching goal of my research program: to push the boundaries of computational natural language understanding. Specifically, I will use brain images to study language “in vivo”. To improve language understanding in computers, I will collect brain images while people read various segments of text. The purpose of these experiments will be to study how people combine words to produce complex meaning that is more than the “sum of the parts”. For example, when people read the words “happy accident”, the positive and negative word do not average into a neutral phrase; instead, the phrase itself is also positive. How does the brain process words, with distinct meanings, to create a unified meaning? I will study this process, called semantic composition, at the phrase, sentence and paragraph level. When people read words in isolation (as has been extensively studied with brain imaging), the onset of the language-related cognitive processes is predictable and stable. However, as words are combined to make more complex language, the cognitive load increases, causing the onset of language-related processes to be less "time-locked" to stimuli (word) onset. We need new machine learning methods that can handle such variation in brain images. Brain image datasets are small but of high dimension, which makes it difficult to build robust models. However, there will be many times more data available if we can combine data across people. I will create new machine learning algorithms that use brain images from different people to build up a "dictionary" of known patterns. I will adapt the dictionaries to specific people, using correlations in brain activity as a guide. Detecting and describing the more complex aspects of language understanding is crucial for building a coherent picture of how people represent and manipulate meaning. My research will inform new algorithms for computational language understanding, as well as provide solutions to some of the problems inherent in multi-subject brain imaging datasets. The impact of my research will reach beyond language in the healthy brain, and improve the diagnosis and treatment of a multitude of brain disorders, including language disorders, sleep disorders and even forms of dementia.