Mid-Air Touch Technology for Accelerated Digital Human Integration and Enhanced Multisensory Experiences

在这个FLF提案中，我们通过应用人机交互（HCI）的原则，通过多感官体验的透镜扩展了空中触觉技术的进步。传统的用户界面通常采用物理触摸，在触摸屏上按按钮和录音。空中互动技术用手势识别算法和全息触摸取代了“屏幕录音”。摄像头跟踪你的手，从远处激活计算机系统上的动作，而相控超声波阵列在半空中为你的手掌和指尖提供触觉。这种独特的组合使得以前只在科幻电影中看到的新颖的交互范例成为可能（例如，《少数派报告》。我们现在可以在沉浸式3D环境中与计算机、数字对象以及彼此进行交互，我们不仅可以看到和听到，还可以触摸和感受，而无需佩戴或持有任何额外的控制器。迄今为止，半空技术的创新一直集中在硬件和软件开发上，以推进与准确性、识别和渲染相关的工程方法。然而，人们对这些技术如何影响人类行为知之甚少，因此不知道如何利用人类的感知来改善互动性和帮助社会。这个奖学金是在使这些新颖的和创新的互动模式的发展，通过特定的多感官体验的透镜，不仅采取了从物理到半空触摸互动的飞跃，但也与其他感官形式（视觉，音频，嗅觉）的最前沿。通过研究跨感官协会和空中触摸到多感官体验的整合，这个奖学金渴望创造更多的情感参与和引人注目的数字体验，其中人类感官与真实的世界一样重要。然后，利用多感官刺激对人类决策的影响，我们采取独特的视角来理解这种新颖的空中接口对社会责任的影响。特别是，我们专注于提高用户的代理感（SoA）与自治系统交互时。今天，智能系统涉及增加的自动化并为我们做出许多决定（汽车，智能家居，机器人），这往往会降低SoA，即，控制自己行为的感觉。通过了解多感官体验增强的空中交互对SoA的影响，我们的目标是设计和开发响应更快的系统，为用户提供控制感，并加强负责任的交互。作为该项目的结果，我们的目标是帮助创造一个未来，在自动驾驶汽车中的全息显示器可以触摸和感受，并且由聚焦超声图案产生的这种触觉维度被设计为促进对自主系统的信任感，在未来，我们可以与朋友和家人进行3D视频通话，同时能够通过发送聚焦光束产生的触觉来传达我们的情感。这个FLF将通过从多感官角度系统地理解空中触觉的作用来挑战HCI中现有的交互范例，旨在设计数字交互，让您可以看到，听到，触摸和嗅觉，这种多感官体验有助于在用户和技术之间共享代理。我们的方法将打破传统的空中技术研究，围绕涉及心理学，神经科学和人机交互的空中技术建立一个新的包容性多学科研究生态系统，使这些技术的影响不仅限于硬件和软件，而且还对社会产生积极影响，加速人类体验的数字化。

In this FLF proposal, we expand the advances of mid-air haptic technology through the lens of multisensory experiences by applying the principles of human-computer interaction (HCI). Traditional user interfaces typically employ physical touch, e.g., pressing buttons and taping on touchscreens. Mid-air interaction technology replaces "taping on screens" with hand-gesture recognition algorithms and holographic touch. Cameras track your hands to activate actions on a computer system from a distance while phased ultrasonic arrays deliver tactile sensations to your palms and fingertips in mid-air. This unique combination enables novel interaction paradigms previously only seen in sci-fi movies (e.g., Minority Report). We can now interact with computers, digital objects and with each other in immersive 3D environments that we can not only see and hear, but can also touch and feel, without the need to wear or hold any additional controllers.To date, innovation in mid-air technologies has been focussed on hardware and software development to advance engineering methods related to accuracy, recognition and rendering. However, very little is known about how these technologies influence human behaviour and therefore it is unknown how to exploit human perception to improve interactivity and help society. This fellowship is at the forefront of enabling the evolution of those novel and innovative interaction paradigms through the specific lens of multisensory experiences not just taking a leap from physical to mid-air touch interactions, but also integrating it with other sensory modalities (vision, audio, smell). Through studying the cross-sensory associations and integration of mid-air touch into multisensory experiences, this fellowship aspires to create more emotionally engaging and compelling digital experiences where the human senses are as important as in the real world. Then, taking advantage of the effect of multisensory stimuli on human decision-making, we take a unique perspective on understanding the impact of such novel mid-air interfaces on societal responsibility. In particular, we focus on improving the users' sense of agency (SoA) when interacting with autonomous systems. Today, intelligent systems involve increased automation and make many decisions for us (cars, smart homes, robots), which tend to reduce the SoA i.e., the feeling of being in control of one's actions. By understanding the effects of mid-air interactions enhanced by multisensory experiences on the SoA, we aim to design and develop more responsive systems that provide the user with a feeling of being in control and strengthen responsible interactions.As a result of this project, we ambition to help create a future where the holographic display in autonomous vehicles can be touched and felt, and that this tactile dimension generated by focused ultrasound patterns is designed to promote a sense of trust towards the autonomous system, and a future where we can have a 3D video call with friends and family whilst being able to communicate our affection by sending tactile sensations generated by a beam of focused ultrasound and even perceive smells from their environment.This FLF will challenge existing interaction paradigms in HCI through a systematic understanding of the role of mid-air haptics from a multisensory perspective, aiming to design digital interactions where you can see, hear, touch and smell, and this multisensory experience helps to share agency between users and technology. Our approach will break from the conventional studies in mid-air technology by fostering a new and inclusive ecosystem of multidisciplinary research around mid-air technology involving psychology, neuroscience and HCI so that the impact of these technology is not limited to hardware and software, but also provides a positive impact on society with respect an accelerated digitisation of human experiences.