Illusionary Surface Interfaces

如果我们考虑威瑟的愿景，未来的计算机将“编织自己到日常生活的结构，直到他们无法区分”，界面设计的问题，以及它如何与普及计算机抛出一些有趣的挑战。传统计算机通常（通过视觉启示）指的是界面的感知和实际属性-不仅暗示基本功能，而且还决定人类可能如何使用系统。然而，这种丰富的信息可视化可能不适合我们想要的普适计算机的外观和感觉。我们的目标是利用多感官错觉，以扩大可以显示的界面属性的范围，只使用日常物体表面作为界面。以类似于“橡胶手错觉”的方式，人们可以被诱导纯粹基于他们所看到的来感知物理触摸，我们将支持由增强视觉引起的视觉和触觉反馈。我们将不改变物体的物理特性，而是使用“智能眼镜”和投影仪在视觉上增强它们，同时通过诱导多感官错觉在触觉上增强它们。触摸表面时的错觉反馈通过增加物质性为增强普遍交互设计提供了巨大的潜力。通过应用这种方法，我们可以诱导光滑和坚硬表面的感觉柔软，不均匀，灵活或可变形。这为闲置的界面属性打开了空间，这些属性可以弥补智能对象通常缺乏的视觉界面启示。通过用户研究，我们将探索如何使用这种多感官错觉交互范式来增强日常对象。我们将首先提取描述多感官表面感知的参数：例如施加的压力程度，织物的触觉等，以及由柔软，可拉伸和可变形材料组成的表面被触摸时引起的变形。然后，我们将使用多感官错觉来模拟模拟多模态体验。然后，我们将探讨如何调整错觉的影响参数，“程序”的多感官错觉。最后，我们将设计，实现和评估巴克斯顿的整个输入设备分类的错觉表面交互。通过结合放大认知，ubicomp和交互设计的互补专业知识，我们将创建一个可扩展的，统一的，实验验证的模型，用于有用的有形和基于触摸的感官错觉界面。我们将为它们的有效实施提供技术框架，以便在智能个人空间和控制室中轻松部署。该模型将以多感官整合理论为基础，并构成从认知科学向新的交互范式的理论转移，从而实现丰富和可扩展的普遍人机交互。

If we consider Weiser’s vision that future computers will “weave themselves into the fabric of everyday life until they are indistinguishable from it", the question of interface design and how it relates to pervasive computers throws up some interesting challenges. Traditional computers typically refer (through visual affordances) to both the perceived and actual properties of the interface - suggesting not only fundamental functionalities but also determining how humans might possibly use the system. Such rich information visualization may, however, not suit the way we want pervasive computers to look and feel.We aim to exploit multisensory illusions in order to extend the range of interface properties that can be displayed, using only everyday object surfaces as interfaces. In a manner similar to the "rubber hand illusion", in which people can be induced to perceive a physical touch based purely on what they see, we will support visual and haptic feedback induced by augmented vision. Instead of changing the objects’ physicality, we will visually augment them using "smart glasses" and projectors, while at the same time augmenting them haptically by inducing multisensory illusion.Illusionary feedback when touching a surface provides great potential for enhancing pervasive interaction design through the addition of materiality. By applying this approach, we can induce the perception of smooth and stiff surfaces soft, uneven, flexible or deformable. This opens the space for idled interface properties that could compensate for the lack of visual interface affordance that smart objects often will have.Through user studies, we will explore how such a multisensory-illusion interaction-paradigm can be used to enhance everyday objects. We will first extract parameters that describe multisensory surface perception: such as the degree of pressure applied, the haptic sensation of fabric, etc. as well as the distortion induced when surfaces composed of soft, stretchable, and deformable materials are touched. We will then mimic the analog multimodal experience using multisensory illusion. Then, we will explore how to adjust the illusion affecting parameters to “program” the multisensory illusion. Finally, we will design, implement, and evaluate illusionary surface interaction for Buxton’s entire taxonomy of input devices.Through the combination of the complementary expertise of amplified cognition, ubicomp, and interaction design, we will create a scalable, unified, and experimentally validated model for useful tangibles and touch-based sensory illusion interfaces. We will provide a technological framework for their efficient implementation to enable effortless deployment in smart personal spaces and control rooms. This model will be grounded in the theory of multisensory integration and constitute a theory transfer from cognitive science towards a new interaction paradigm allowing for rich and scalable pervasive human-computer interaction.