The Multimodal Approach of Rendering Soft Textures on Touch Displays

触控设备正成为我们日常生活中最重要的配件之一，因为它能够运行大量的任务，而且与电脑相比便携。如今，它们很容易获得，便宜和可编程的输入工具被用于各种电子设备。然而，越来越多的应用程序和软件已经开始包括复杂和关键的任务，而这些任务大多是基于视觉线索的显示器上操作的。然而，为了更可靠和舒适的人机交互，必须及时对触摸显示器实施精确的触觉反馈。例如，在显示器上再现触觉纹理感觉已成为触觉领域的新兴课题。在过去的十年中，具有里程碑意义的研究引入了基于测量的渲染技术，通过记录表面信息来创建逼真的触觉纹理。然而，这种方法不可避免地将感知上冗余的数据与必要的纹理信息混合在一起。因此，本研究的第一个目标是确定触觉的感知极限，以便在显示器上创建可信的纹理感觉所需的信息量。这一目标也将使用听觉反馈进行研究，因为感知纹理是触觉和听觉模式的复杂融合过程。此前已有多项研究表明听觉反馈对触觉纹理感知的影响。由于这种效果，将评估听觉反馈对进一步简化触觉数据和增强振动合理性的影响。这项研究的另一个问题是，与传统的动觉设备相反，触摸显示器由于其固定的玻璃表面而受到触觉反馈的限制。最近，一些研究介绍了不同的触觉反馈的实现思路，这些研究仅使用简单的刚度模型验证了它们的驱动机制。除了这些动觉显示器外，甚至传统的力反馈设备也被用于使用线性二阶刚度模型来渲染软材料，该模型通常用于复制环境中的动态。另一方面，软质材料由于其非线性粘弹性特性，不能用线性模型很好地描述。因此，精确的柔软效果可能不会在任何动觉设备上产生。因此，本研究的第二个目标是在触摸显示器上再现各种软质材料对应的真实柔软效果。这一目标将通过研究人体刚度感觉和各种非线性刚度模型来实现，以精确地模拟柔软响应，避免不必要的复杂模型产生无法感知的细节。在本研究的最后，将建立基于感知的多模态软纹理渲染策略。

Touch devices are becoming one of the most significant accessory of our daily lives due to being capable of running numerous tasks and being portable as opposed to computers. Nowadays, they are easily accessible, cheap and programmable input tools being used at various electronic devices. However, increasing number of applications and software have begun including complicated and critical tasks, and these tasks have been mostly operated on displays based on only visual cues. However, for more reliable and comfortable human-display interaction, precise haptic feedback should be promptly implemented to touch displays. For instance, reproducing tactile texture sensation on displays has been an emerging topic in the haptic field. In the last decade, landmark studies introduced measurement-based rendering techniques that enabled creating realistic haptic textures by recording surface information. However, such methods inevitably blend perceptually redundant data with the necessary texture information. Therefore, the first goal of this study is to determine the perceptual limits of the tactile sensation so that necessary amount of information can be driven for creating plausible texture feeling on displays. This goal will be also investigated using auditory feedback due to the fact that perceiving a texture is a complex fusion process of tactile and auditory modalities. Previously, various studies showed the effect of auditory feedback on tactile texture perception. Thanks to this effect, the influence of auditory feedback on further tactile data simplification and augmenting the plausibility of vibrations will be evaluated. Another concern of this study arises from the fact that touch displays have been limited with tactile feedback due to their fixed glass surface as opposed to traditional kinesthetic devices. Lately, several studies introduced different implementation ideas of kinesthetic feedback on touch displays, and these studies validated their actuation mechanisms using only simple stiffness models. Apart from these kinesthetic displays, even traditional force feedback devices have been used to render soft materials using linear second-order stiffness model which is commonly used to replicate the dynamics in the environment. On the other hand, soft materials cannot be well-described using linear models due to their nonlinear viscoelastic properties. Because of this, precise softness effect may not be generated on any kinesthetic devices. Thus, the second goal of this study is to reproduce realistic softness effect corresponding to various soft materials on touch displays. This aim will be realized by investigating human stiffness sensation and various non-linear stiffness models to mimic softness response precisely avoiding unnecessary complex models producing unperceivable details. At the end of this study, perception-based multimodal soft texture rendering strategy will be established.