Illusionary Surface Interfaces

如果我们考虑威瑟的愿景，即未来的计算机将“将自己编织进日常生活的结构中，直到它们无法与之区分开来”，那么界面设计及其与普及计算机的关系的问题就引发了一些有趣的挑战。传统计算机通常(通过视觉提供)同时参考界面的感知和实际属性--不仅暗示基本功能，而且还决定人类可能如何使用该系统。然而，这种丰富的信息可视化可能并不适合我们希望普及的计算机看起来和感觉上的方式。我们的目标是利用多感官错觉来扩大可以显示的界面属性的范围，只使用日常对象表面作为界面。与“橡皮手错觉”类似，我们将支持由增强视觉引起的视觉和触觉反馈。“橡皮手错觉”指的是，人们可以完全根据所看到的东西来感知身体触摸。我们不会改变物体的物理特性，而是使用智能眼镜和投影仪在视觉上增强它们，同时通过诱导多感官错觉来增强它们的触觉。触摸表面时的幻觉反馈为通过增加物质性来增强普适交互设计提供了巨大的潜力。通过应用这种方法，我们可以诱导出光滑和坚硬的表面柔软、不平坦、灵活或可变形的感觉。这为空闲的界面属性打开了空间，这些属性可以弥补智能对象通常具有的视觉界面提供的不足。通过用户研究，我们将探索如何使用这种多感觉-错觉交互范例来增强日常对象。我们将首先提取描述多感官表面感知的参数：例如施加的压力程度、织物的触觉等，以及由柔软、可拉伸和可变形的材料组成的表面被触摸时所引起的变形。然后，我们将使用多感官错觉来模拟模拟的多模式体验。然后，我们将探索如何调整错觉的影响参数来对多感官错觉进行编程。最后，我们将为巴克斯顿的整个输入设备分类设计、实施和评估虚幻表面交互。通过放大认知、ubiomp和交互设计的互补专业知识的结合，我们将为有用的有形和基于触摸的感觉错觉界面创建一个可扩展、统一和经过实验验证的模型。我们将为它们的有效实施提供技术框架，以便在智能个人空间和控制室中轻松部署。该模型将建立在多感觉整合理论的基础上，并构成从认知科学到新的交互范式的理论转移，允许丰富和可扩展的普遍人机交互。

If we consider Weiser’s vision that future computers will “weave themselves into the fabric of everyday life until they are indistinguishable from it", the question of interface design and how it relates to pervasive computers throws up some interesting challenges. Traditional computers typically refer (through visual affordances) to both the perceived and actual properties of the interface - suggesting not only fundamental functionalities but also determining how humans might possibly use the system. Such rich information visualization may, however, not suit the way we want pervasive computers to look and feel.We aim to exploit multisensory illusions in order to extend the range of interface properties that can be displayed, using only everyday object surfaces as interfaces. In a manner similar to the "rubber hand illusion", in which people can be induced to perceive a physical touch based purely on what they see, we will support visual and haptic feedback induced by augmented vision. Instead of changing the objects’ physicality, we will visually augment them using "smart glasses" and projectors, while at the same time augmenting them haptically by inducing multisensory illusion.Illusionary feedback when touching a surface provides great potential for enhancing pervasive interaction design through the addition of materiality. By applying this approach, we can induce the perception of smooth and stiff surfaces soft, uneven, flexible or deformable. This opens the space for idled interface properties that could compensate for the lack of visual interface affordance that smart objects often will have.Through user studies, we will explore how such a multisensory-illusion interaction-paradigm can be used to enhance everyday objects. We will first extract parameters that describe multisensory surface perception: such as the degree of pressure applied, the haptic sensation of fabric, etc. as well as the distortion induced when surfaces composed of soft, stretchable, and deformable materials are touched. We will then mimic the analog multimodal experience using multisensory illusion. Then, we will explore how to adjust the illusion affecting parameters to “program” the multisensory illusion. Finally, we will design, implement, and evaluate illusionary surface interaction for Buxton’s entire taxonomy of input devices.Through the combination of the complementary expertise of amplified cognition, ubicomp, and interaction design, we will create a scalable, unified, and experimentally validated model for useful tangibles and touch-based sensory illusion interfaces. We will provide a technological framework for their efficient implementation to enable effortless deployment in smart personal spaces and control rooms. This model will be grounded in the theory of multisensory integration and constitute a theory transfer from cognitive science towards a new interaction paradigm allowing for rich and scalable pervasive human-computer interaction.