HUMAN-Touch: Physical and Neurocognitive AI models of Ultrasound Haptics

HUMAN-Touch：超声触觉的物理和神经认知人工智能模型

• 批准号: MR/W013576/1
• 负责人: William Frier
• 金额: $ 98.45万
• 依托单位: Ultraleap
• 依托单位国家: 英国
• 项目类别: Fellowship
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=MR%2FW013576%2F1
• 关键词: HUMAN; Touch; Physical; Neurocognitive; AI

When we touch a physical object, a sequence of mechanical events occurs whereby vibration is transmitted via the hard and soft tissues of the hand. The signals generated during object manipulation are then transduced into neural signals via ascending sensory pathways that our brain interprets as touch. When combined with signals from our other senses, memories and expectations, this information forms our realisation of the physical and psychological worlds. With modern technology, it is possible to generate immersive environments with breath-taking graphics, yet touch technologies (also known as haptics) capable of realistically and unobtrusively emulating the sense of touch have only just began to emerge. This future leaders fellowship (FLF) aims to unlock new potential in non-contact touch technologies by holistically understand both the physical and psychophysical dimensions of ultrasound mid-air haptics. To that end, we will lead ground-breaking R&D across acoustics, biophysics, neuroscience and artificial intelligence (AI). Mid-air haptics refers to electronically controlled collections of ultrasound speakers (phased arrays) that collectively generate complex acoustic fields in 3D space that can be touched and felt with our bare hands. Holographic 3D objects and surfaces can therefore be "haptified" and interacted with in mid-air, without the need to wear or hold any specialised controllers; a feature particularly appreciated in public display interfaces to limit the spread of pathogens. Coupled with augmented and virtual reality solutions, the technology allows the design and remote collaboration scenarios that are often seen in Sci-Fi movies such as Iron Man and Minority Report.R&D in mid-air haptics has been accelerating in recent years, yet has almost exclusively focused on hardware advancements, acoustic signal processing, and human-computer interaction (HCI) use cases. We believe that the true potential of ultrasound mid-air haptics is still unexplored, an opportunity uniquely available to be exploited by this FLF. Current mid-air haptics displays, such as those commercialised by Ultraleap only target one type of touch receptors (mechanoreceptors), which limits the device expressivity. Biophysical models capturing how acoustic waves interact with the skin are at their infancy and are experimentally unverified. Generative and computational models connecting phased array output, acoustic focusing waves, skin vibrations, mechanoreceptors, and psychophysical experiences are absent. This fellowship will be the first to thread these together. We will study ultrasonic mid-air haptics from first principles (i.e., acoustics and biophysics) all the way to perception and neurocognition. We will understand how localised acoustic energy generates non-localised skin vibrations, how those vibrations activate different touch receptors in the skin, and how receptors encode information that our somatosensory system then understands as touch. Once the forward problem is pieced together, our aim is to use machine learning to construct generative AI models enabling us to solve the inverse problem. What input ultrasound signals should be used to create the tactile sensation of holding a high-quality piece of paper? Today, there is no scientific way of answering such a question, even if we know that something like this is possible. Being able to bridge the different scientific fields related to ultrasonic mid-air haptics to create a holistic understanding of holographic touch is uniquely enabled by this FLF application.This 4-year, full-time, reduced hours FLF will support a cross-disciplinary and agile team of 2 postdoctoral research associates (RAs) led by the fellow, while being hosted at the only company in the world that is commercialising mid-air haptics, thus providing the fellowship with access to unique resources, engineering insights, and a direct pathway to economic and societal impact.

当我们触摸物体时，会发生一系列机械事件，从而通过手部的硬组织和软组织传递振动。在物体操作过程中产生的信号然后通过我们的大脑解释为触摸的上行感觉通路转换为神经信号。当与来自我们其他感官、记忆和期望的信号相结合时，这些信息形成了我们对物理和心理世界的认识。借助现代技术，可以生成具有令人惊叹的图形的沉浸式环境，但能够真实且不显眼地模拟触觉的触摸技术（也称为触觉）才刚刚开始出现。这个未来的领导者奖学金（FLF）旨在通过全面了解超声半空触觉的物理和心理物理维度来释放非接触式触摸技术的新潜力。为此，我们将引领声学、生物物理学、神经科学和人工智能（AI）领域的突破性研发。半空触觉指的是电子控制的超声扬声器（相控阵列）集合，这些扬声器在3D空间中共同产生复杂的声场，可以用我们的双手触摸和感受。因此，全息3D物体和表面可以在半空中进行“触觉化”和交互，而无需佩戴或持有任何专门的控制器;这一功能在公共显示界面中特别受欢迎，以限制病原体的传播。结合增强现实和虚拟现实解决方案，该技术可以实现科幻电影中常见的设计和远程协作场景，如钢铁侠和少数派报告。近年来，空中触觉的研发一直在加速，但几乎完全集中在硬件进步，声学信号处理和人机交互（HCI）用例上。我们相信，超声半空触觉的真正潜力尚未被探索，这是一个独特的机会，可由本FLF利用。目前的半空触觉显示器，例如由Ultraleap商业化的那些，仅针对一种类型的触摸感受器（机械感受器），这限制了设备的表现力。捕捉声波如何与皮肤相互作用的生物物理模型还处于起步阶段，未经实验验证。生成和计算模型连接相控阵输出，声聚焦波，皮肤振动，mechanoreceptors，和心理物理的经验是不存在的。这个团契将是第一个把这些串在一起的。我们将从第一原理（即，声学和生物物理学）一直到感知和神经认知。我们将了解局部声能如何产生非局部皮肤振动，这些振动如何激活皮肤中不同的触觉感受器，以及感受器如何编码信息，然后我们的体感系统将其理解为触摸。一旦正问题被拼凑在一起，我们的目标是使用机器学习来构建生成AI模型，使我们能够解决逆问题。应该使用什么样的输入超声波信号来产生握持高质量纸张的触感？今天，没有科学的方法来回答这个问题，即使我们知道这样的事情是可能的。能够弥合与超声波半空中触觉相关的不同科学领域，以创建对全息触摸的整体理解，这是通过该FLF应用程序实现的。这个为期4年的全职FLF将支持由研究员领导的由2名博士后研究助理（RA）组成的跨学科和敏捷团队，同时托管在世界上唯一一家将半空中触觉商业化的公司，从而为奖学金提供了获得独特资源、工程见解以及直接实现经济和社会影响的途径。