Computational Techniques for Exploring Language in the Brain

人们希望与他们的电脑、平板电脑和智能手机进行直观的互动；他们越来越希望让他们的设备执行任务，而不是点击和点击命令。苹果的Siri和微软的Cortana就是最近实现这一目标的两个尝试。我们可以要求Siri设置闹钟，给我们的母亲打电话，以及许多其他日常事务。然而，当涉及到需要更深入地理解自然语言的更复杂的请求时(例如，“叫我救护车”)，电脑供不应求，让消费者失望。从长远来看，我们需要能够像人类理解语言那样理解语言的计算机程序。这就是我的研究项目的首要目标：推动计算自然语言理解的边界。具体地说，我将使用大脑图像来研究“体内”的语言。*为了提高计算机对语言的理解能力，我将在人们阅读文本的不同部分时收集大脑图像。这些实验的目的将是研究人们如何组合单词来产生复杂的意义，而不是“部分之和”。例如，当人们读到“幸福的意外”这两个词时，这个积极和消极的词并不是平均成一个中性短语；相反，这个短语本身也是积极的。大脑如何处理具有不同含义的单词，以创造统一的含义？我将在短语、句子和段落层面研究这种称为语义合成的过程。*当人们孤立地阅读单词时(脑成像已经进行了广泛的研究)，与语言相关的认知过程的开始是可以预测和稳定的。然而，随着单词被组合成更复杂的语言，认知负荷增加，导致与语言相关的过程的开始对刺激(词)开始的时间锁定较少。我们需要新的机器学习方法来处理大脑图像中的这种变化。*脑图像数据集很小，但维度很高，这使得建立稳健的模型变得困难。然而，如果我们可以跨人组合数据，那么可用的数据将会增加很多倍。我将创建新的机器学习算法，使用来自不同人的大脑图像来建立已知模式的“词典”。我将使用大脑活动的相关性作为指导，使词典适用于特定的人。*检测和描述语言理解的更复杂的方面，对于建立一个关于人们如何表达和操纵意义的连贯图景至关重要。我的研究将为计算机语言理解提供新的算法，并为多主题脑成像数据集中固有的一些问题提供解决方案。我的研究的影响将超越健康大脑中的语言，并改进对多种大脑疾病的诊断和治疗，包括语言障碍、睡眠障碍，甚至各种形式的痴呆症。

People want intuitive interactions with their computers, tablets and smartphones; increasingly, they want to ask their devices to perform tasks, not tap and click commands. Apple's Siri and Microsoft's Cortana are two recent attempts to realize this goal. We can ask Siri to set an alarm, to place a call to our mother, and many other everyday tasks. However, when it comes to more complex requests requiring a deeper understanding of natural language (e.g. “Call me an ambulance”), computers fall short, leaving the consumer disappointed. In the long term, we are in need of computer programs that can understand language as humans understand it. This is the overarching goal of my research program: to push the boundaries of computational natural language understanding. Specifically, I will use brain images to study language “in vivo”.******To improve language understanding in computers, I will collect brain images while people read various segments of text. The purpose of these experiments will be to study how people combine words to produce complex meaning that is more than the “sum of the parts”. For example, when people read the words “happy accident”, the positive and negative word do not average into a neutral phrase; instead, the phrase itself is also positive. How does the brain process words, with distinct meanings, to create a unified meaning? I will study this process, called semantic composition, at the phrase, sentence and paragraph level.******When people read words in isolation (as has been extensively studied with brain imaging), the onset of the language-related cognitive processes is predictable and stable. However, as words are combined to make more complex language, the cognitive load increases, causing the onset of language-related processes to be less "time-locked" to stimuli (word) onset. We need new machine learning methods that can handle such variation in brain images. ******Brain image datasets are small but of high dimension, which makes it difficult to build robust models. However, there will be many times more data available if we can combine data across people. I will create new machine learning algorithms that use brain images from different people to build up a "dictionary" of known patterns. I will adapt the dictionaries to specific people, using correlations in brain activity as a guide. ******Detecting and describing the more complex aspects of language understanding is crucial for building a coherent picture of how people represent and manipulate meaning. My research will inform new algorithms for computational language understanding, as well as provide solutions to some of the problems inherent in multi-subject brain imaging datasets. The impact of my research will reach beyond language in the healthy brain, and improve the diagnosis and treatment of a multitude of brain disorders, including language disorders, sleep disorders and even forms of dementia.**