Multimodal Computing Environments: Theory and Applications

9213823 Glinert这是一项为期三年的持续资助的第二年，其目标是为多模态界面设计开发理论基础。调查的基础是视觉和听觉界面（图形和声音）问题之间的相互作用，以及在界面中活动的抽象对象的时间和动态方面。同时考虑这些方面导致了一个称为元部件的面向对象构造的定义，它是一个具有内置策略的抽象对象，允许以多种不同的方式向用户显示自己。在这些表现方式中有图像、声音甚至两种媒介的组合和序列。实施将包括适合实验的微世界环境，用作受控实验的试验台，以验证界面设计过程的理论方面。这种微世界的原型包括一个平面图形的动画编辑器和一个增强的多模式界面，供视障用户使用，但不是盲人。这些例子已被用作测试潜在有用想法的初步方法。

9213823 Glinert This is the second year funding of a three-year continuing award with the objective of developing a theoretical basis for multimodal interface design. The basis for the investigation is the interplay between visual and auditory interface (graphics and sound) issues, and also the temporal and dynamic aspects of the abstract objects that are active in the interface. The simultaneous consideration of these aspects leads to the definition od an object-oriented construct called a metawidget which is an abstract object with built-in strategies which allow multiple different manners of manifesting themselves to the user. Among those modes of manifestation are image, sound or even combinations and sequences of the two media. The implementation will consist of suitable microworld environments for experimentation, used as testbeds for controlled experimentation to validate the theoretical aspects of the interface design process as they are developed. Prototypes of such microworlds include an animated editor for planar graphs and an enhanced multimodal interface for visually impaired - but not blind - users. These examples have been used as a preliminary approach to test potentially useful ideas.