CAREER: Multimodal Language Processing for Natural Interfaces

职业：自然界面的多模态语言处理

• 批准号: 9876223
• 负责人: Michael Johnston
• 金额: $ 33.39万
• 依托单位: Oregon Health & Science University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 1999
• 资助国家: 美国
• 起止时间: 1999-03-01 至 2005-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9876223&HistoricalAwards=false
• 关键词: CAREER; Multimodal; Language; Processing; Natural

By providing a number of channels through which information may pass between user and computer, multimodal interfaces promise to significantly increase the bandwidth and fluidity of the interface between humans and machines. Natural language processing techniques typically focus on processing strings of words within a single dimension. This program of research seeks to develop an approach to multimodal language processing which enables parsing and interpretation of natural human input distributed across two or three spatial dimensions, time, and the acoustic dimension of speech. A declarative unification-based grammar representation is proposed for the description of multimodal integration strategies. This representation is interpreted by a multidimensional parser yielding an architecture which provides the foundation for next generation natural interfaces supporting human-computer interaction through speech, pen, gesture, and other modes. Fundamental advances in multimodal interfaces are essential to the successful migration of human-computer interaction from the desktop to small mobile computing devices, public information kiosks, and other platforms for which complex graphical user interfaces are not feasible. Multimodal interfaces stand to play a critical role both in realizing the vision of universal access to information and in the development of assistive technologies for people with disabilities.

通过提供大量的通道，信息可以通过这些通道在用户和计算机之间传递，多模式界面有望显著增加人机界面的带宽和流动性。 自然语言处理技术通常专注于处理单个维度内的单词串。 该研究计划旨在开发一种多模态语言处理方法，该方法能够解析和解释分布在两个或三个空间维度，时间和语音声学维度的自然人类输入。 提出了一种基于声明式统一的语法表示方法来描述多模态集成策略。 这种表示是由一个多维解析器产生的架构，它提供了支持下一代自然接口的基础，通过语音，笔，手势和其他模式的人机交互解释。 多模态界面的根本性进步对于人机交互从桌面到小型移动的计算设备、公共信息亭和其他复杂图形用户界面不可行的平台的成功迁移至关重要。 多模式接口将在实现普遍获取信息的愿景和开发残疾人辅助技术方面发挥关键作用。