HUMAN-Touch: Physical and Neurocognitive AI models of Ultrasound Haptics

当我们触摸物理物体时，会发生一系列机械事件，振动通过手的硬组织和软组织传递。然后，在物体操纵过程中产生的信号通过上升的感觉通路转化为神经信号，我们的大脑将其解释为触摸。当与来自我们其他感官、记忆和期望的信号相结合时，这些信息形成了我们对物理和心理世界的认识。有了现代技术，有可能产生令人叹为观止的图形的沉浸式环境，但能够逼真而不引人注目地模拟触觉的触摸技术(也称为触觉)才刚刚开始出现。这一未来领袖联谊会(FLF)旨在通过全面了解超声波半空触觉的物理和心理物理维度，释放非接触触摸技术的新潜力。为此，我们将领导声学、生物物理学、神经科学和人工智能(AI)领域的突破性研发。空中触觉是指电子控制的超声波扬声器(相控阵)集合，这些扬声器在3D空间中共同产生复杂的声场，可以用我们的双手触摸和感觉。因此，全息3D物体和表面可以在半空中被“触觉”并与之交互，而不需要佩戴或持有任何专门的控制器；这一特点在公共显示界面中特别受欢迎，可以限制病原体的传播。结合增强和虚拟现实解决方案，该技术允许设计和远程协作场景，如钢铁侠和少数民族报告等科幻电影中经常看到的场景。近年来，半空触觉的研发一直在加速，但几乎只专注于硬件进步、声信号处理和人机交互(HCI)用例。我们相信，超声半空触觉的真正潜力仍未被发掘，这是一个独特的机会，可以由这个FLF来开发。目前的空中触觉显示，如Ultraaleap商业化的显示，只针对一种类型的触摸感受器(机械感受器)，这限制了设备的表现力。捕捉声波如何与皮肤相互作用的生物物理模型还处于初级阶段，还没有得到实验验证。没有将相控阵输出、声波聚焦波、皮肤振动、机械感受器和心理物理经验联系起来的生成和计算模型。这个联谊会将是第一个把这些联系在一起的人。我们将从声学和生物物理学的基本原理(即声学和生物物理学)一直到感知和神经认知来研究超声波空中触觉。我们将了解局部声能如何产生非局部皮肤振动，这些振动如何激活皮肤中的不同触摸感受器，以及感受器如何编码信息，然后我们的体感系统将其理解为触摸。一旦正向问题拼凑在一起，我们的目标是使用机器学习来构建生成性人工智能模型，使我们能够解决逆问题。应该使用什么输入超声信号来创造手持高质量纸张的触觉感觉？今天，即使我们知道这样的事情是可能的，也没有科学的方法来回答这样的问题。通过这种FLF应用程序，能够在与超声半空触觉相关的不同科学领域之间架起一座桥梁，从而创建对全息触摸的整体理解。这项为期4年的全职、减少工作时间的FLF将支持一个由该研究员领导的由2名博士后助理(RAS)组成的跨学科和灵活的团队，同时托管在世界上唯一一家将空中触觉商业化的公司，从而为该研究员提供访问独特资源、工程见解以及直接产生经济和社会影响的途径。

When we touch a physical object, a sequence of mechanical events occurs whereby vibration is transmitted via the hard and soft tissues of the hand. The signals generated during object manipulation are then transduced into neural signals via ascending sensory pathways that our brain interprets as touch. When combined with signals from our other senses, memories and expectations, this information forms our realisation of the physical and psychological worlds. With modern technology, it is possible to generate immersive environments with breath-taking graphics, yet touch technologies (also known as haptics) capable of realistically and unobtrusively emulating the sense of touch have only just began to emerge. This future leaders fellowship (FLF) aims to unlock new potential in non-contact touch technologies by holistically understand both the physical and psychophysical dimensions of ultrasound mid-air haptics. To that end, we will lead ground-breaking R&D across acoustics, biophysics, neuroscience and artificial intelligence (AI). Mid-air haptics refers to electronically controlled collections of ultrasound speakers (phased arrays) that collectively generate complex acoustic fields in 3D space that can be touched and felt with our bare hands. Holographic 3D objects and surfaces can therefore be "haptified" and interacted with in mid-air, without the need to wear or hold any specialised controllers; a feature particularly appreciated in public display interfaces to limit the spread of pathogens. Coupled with augmented and virtual reality solutions, the technology allows the design and remote collaboration scenarios that are often seen in Sci-Fi movies such as Iron Man and Minority Report.R&D in mid-air haptics has been accelerating in recent years, yet has almost exclusively focused on hardware advancements, acoustic signal processing, and human-computer interaction (HCI) use cases. We believe that the true potential of ultrasound mid-air haptics is still unexplored, an opportunity uniquely available to be exploited by this FLF. Current mid-air haptics displays, such as those commercialised by Ultraleap only target one type of touch receptors (mechanoreceptors), which limits the device expressivity. Biophysical models capturing how acoustic waves interact with the skin are at their infancy and are experimentally unverified. Generative and computational models connecting phased array output, acoustic focusing waves, skin vibrations, mechanoreceptors, and psychophysical experiences are absent. This fellowship will be the first to thread these together. We will study ultrasonic mid-air haptics from first principles (i.e., acoustics and biophysics) all the way to perception and neurocognition. We will understand how localised acoustic energy generates non-localised skin vibrations, how those vibrations activate different touch receptors in the skin, and how receptors encode information that our somatosensory system then understands as touch. Once the forward problem is pieced together, our aim is to use machine learning to construct generative AI models enabling us to solve the inverse problem. What input ultrasound signals should be used to create the tactile sensation of holding a high-quality piece of paper? Today, there is no scientific way of answering such a question, even if we know that something like this is possible. Being able to bridge the different scientific fields related to ultrasonic mid-air haptics to create a holistic understanding of holographic touch is uniquely enabled by this FLF application.This 4-year, full-time, reduced hours FLF will support a cross-disciplinary and agile team of 2 postdoctoral research associates (RAs) led by the fellow, while being hosted at the only company in the world that is commercialising mid-air haptics, thus providing the fellowship with access to unique resources, engineering insights, and a direct pathway to economic and societal impact.