Creating tangible interfaces by augmenting physical objects with multimodal language
Creating tangible interfaces by augmenting physical objects with multimodal language
复制标题
通过使用多模式语言增强物理对象来创建有形界面
DOI：
10.1145/359784.360305
复制
发表时间：
2001
期刊：
影响因子：
--
通讯作者：
Philip R. Cohen
中科院分区：
文献类型：
--
作者：
David R. McGee;Philip R. Cohen
Rasa是一个有形的增强现实环境，它以数字方式增强了军事指挥所现有的基于纸张的指挥和控制能力。通过观察和理解用户的语音、笔和基于触摸的多模态语言，Rasa通过计算增强了指挥所地图上的物理对象，将这些项目链接到相同的数字表示-例如，将纸质地图链接到世界，将Post-it-笔记链接到军事单位。在这里，我们给一个彻底的帐户Rasa的底层多智能体框架，其识别，理解和多模态集成组件。此外，我们研究五个属性的语言生成性，可理解性，组合性，所指性，有时，持久性，使其适合作为一种增强方法，对比这些属性的其他增强方法。正是语言的这些特性使得Rasa的用户能够扩充物理对象，将它们转换为有形的界面。
Rasa is a tangible augmented reality environment that digitally enhances the existing paper-based command and control capability in a military command post. By observing and understanding the users' speech, pen, and touch-based multimodal language, Rasa computationally augments the physical objects on a command post map, linking these items to digital representations of the same-for example, linking a paper map to the world and Post-itâ notes to military units. Herein, we give a thorough account of Rasa's underlying multiagent framework, and its recognition, understanding, and multimodal integration components. Moreover, we examine five properties of language-generativity, comprehensibility, compositionality, referentiality, and, at times, persistence-that render it suitable as an augmentation approach, contrasting these properties to those of other augmentation methods. It is these properties of language that allow users of Rasa to augment physical objects, transforming them into tangible interfaces.