CAREER: Experimental pragmatics and semantics in visual language

职业：视觉语言中的实验语用学和语义学

• 批准号: 1844186
• 负责人: Kathryn Davidson
• 金额: $ 150万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1844186&HistoricalAwards=false
• 关键词: CAREER; Experimental; pragmatics; semantics; visual

One thing that makes human language so powerful is that its limited vocabulary can be used to share an unlimited number of ideas, and to say things that have never been said before. Understanding how words combine to express new meanings in a predictable (and therefore unlimited) way has been the focus of decades of research, and has led to increasing success by artificial intelligence systems in responding to direct queries, such as asking for directions, weather, etc. However, the meaning of language in face-to-face interactions is influenced by factors beyond just words, and even beyond intonation: nonlinguistic visual information interacts clearly and regularly with speech. For example, when someone says they "took a note" while moving their fingers as if they were typing on a keyboard, this conveys that the note was electronic; as another example, a pointing gesture toward someone while saying "my neighbor" identifies the person with the label. Technology currently exists to recognize most of these movements, but they are typically ignored for the purpose of meaning since so little is known about how they contribute and compose with other parts of a sentence. A source of valuable insight for this question comes from languages which are entirely visual: sign languages such as American Sign Language have their own full distinct grammar and their own complex rules for the meanings of word combinations, and at the same time clearly and regularly integrate visual demonstrations and pointing into sentence meaning. This project compares the role of visual (demonstrations and pointing) information in gestures used in English with well-documented counterparts in American Sign Language. It builds on current work by the project team in theoretical models of sign language linguistics and gesture, expanding to include state-of-the-art experimental methodologies and analysis. English and American Sign Language will be studied at each point in parallel, both by gathering quantitative data using responses to videos presented online and by analyzing fluent language productions. A major challenge is that research on the compositional processes that underlie how words combine into more complex meanings requires a dolid background in logic and computation, and scholars with this skill set tend to have limited overlap with the Deaf, Hard of Hearing, and fluent signing scholars who have the most insight into the quickly growing fields of sign linguistics and gesture. To correct for this gap, the project will include training opportunities for postdoctoral, graduate students and undergraduates whose studies are focused on sign languages and gesture, to provide experience with experimental methodology, statistical analysis, and logic and computation. The project will also include the development of a course and textbook dedicated to bridging this gap.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类语言之所以如此强大，是因为它有限的词汇可以用来分享无限的想法，说出以前从未说过的话。理解单词如何联合收割机在可预测的（因此是无限的）方式一直是几十年来研究的焦点，并导致人工智能系统在回应直接查询方面越来越成功，例如询问方向，天气等。非语言的视觉信息与言语清晰而有规律地相互作用。例如，当有人说他们“做了一个笔记”，同时移动他们的手指，就好像他们在键盘上打字一样，这传达了笔记是电子的;作为另一个例子，指向某人的手势，同时说“我的邻居”识别出带有标签的人。目前存在的技术可以识别这些运动中的大多数，但它们通常被忽略的意义，因为很少有人知道他们如何贡献和组成一个句子的其他部分。这个问题的一个有价值的见解来源于完全视觉化的语言：手语，如美国手语，有自己完全独特的语法和自己复杂的单词组合规则，同时清晰而有规律地将视觉演示和指向整合到句子意义中。 这个项目比较了视觉（示范和指点）信息在英语手势中的作用与美国手语中有据可查的对应物。它建立在手语语言学和手势理论模型的项目团队目前的工作基础上，扩展到包括最先进的实验方法和分析。英语和美国手语将在每一个点并行研究，无论是通过收集定量数据使用响应在线视频呈现，并通过分析流利的语言制作。一个主要的挑战是，对构成单词联合收割机如何组合成更复杂含义的基础过程的研究需要一个枯燥的逻辑和计算背景，而具有这种技能的学者往往与聋人、听力障碍者和流利的手语学者有有限的重叠，而这些学者对快速发展的手语语言学和手势领域有着最深刻的见解。为了弥补这一差距，该项目将包括为博士后、研究生和本科生提供培训机会，他们的研究重点是手语和手势，以提供实验方法、统计分析、逻辑和计算方面的经验。该项目还将包括一门课程和教科书的发展，致力于弥合这一差距。这个奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。