ActivATOR - Active AudiTiOn for Robots

声音中的生命发生在运动中。作为人类的倾听者，听觉——倾听的能力——是由我们的身体和环境之间的物理相互作用形成的。我们将运动与听觉感知结合起来，以便更好地听（例如，通过接近感兴趣的声源），识别物体（例如，通过触摸物体并听产生的声音），检测故障（例如，通过移动物体来听异常的吱吱声），以及卸载思想（例如，通过敲击表面来回忆音乐片段）。因此，理解和利用运动中的声音的能力是嵌入式人工智能（AI）的基本先决条件。该项目将为主动机器人试听提供基础的概率框架，使具身代理能够控制自己身体的运动（“自我运动”），以便在现实的声学环境（家庭、公共空间和涉及多个竞争声源的环境）中进行听觉注意。通过将声音与运动相结合，该项目将使机器能够想象、控制和利用物理与环境相互作用的听觉后果。通过改变机器通过声音感知生命的方式，研究成果将对新兴市场至关重要，这些市场将使机器人能够增强而不是与人类竞争，从而超越人类身体的局限性（感官准确性、力量、耐力、记忆力）。因此，拟议的研究有可能改变和颠覆涉及机器听力的一系列行业，从人机增强（智能假肢、辅助听力技术、脑机接口）到人机协作（行星探索、搜救、危险物质清除）和自动化（环境监测、自动驾驶汽车、医疗保健中的人工智能辅助诊断）。这个项目将考虑一个协作机器人（cobot）的具体案例研究，它可以增强听力受损的人类伙伴的听觉体验。听力损失是英国第二大常见的残疾，影响着1100万人。听力丧失会影响态势感知和沟通能力，这可能会影响心理健康，在极端情况下还会影响认知功能。然而，由于各种复杂的原因，从不适到社会耻辱，只有200万人选择佩戴助听器。这个项目的目标是开发一种协作机器人，它将增强听障人士的听觉体验。协作机器人将在人类伴侣的家中自主移动，协助完成日常任务。我们的研究将使协作机器人能够利用自我运动来学习声音场景的内部表征（孩子们的叽叽叽喳声、水壶的沸腾声、配偶的呼救声）。协作机器人将通过个人智能设备（手表、手机）与合作伙伴进行交互。使用人机交互界面，协作机器人将通过振动信息提醒其合作伙伴重大事件（呼叫帮助），并通过交互式地图分享其听觉体验，这些地图将听觉线索可视化，并指示显着性（例如，响度，自发性）和效价（积极与关注）。与智能设备相比，协作机器人将具有独特的能力，可以主动关注和探索不确定事件（敲打楼上），并采取行动（协助配偶，呼叫救护车），而无需在个人空间（浴室，卧室）永久安装设备。因此，该项目有可能通过实现长期独立生活、保护隐私和促进包容性来改变听障人士的生活。

Life in sound occurs in motion. As human listeners, audition - the ability to listen - is shaped by physical interactions between our bodies and the environment. We integrate motion with auditory perception in order to hear better (e.g., by approaching sound sources of interest), to identify objects (e.g., by touching objects and listening to the resulting sound), to detect faults (e.g., by moving objects to listen to anomalous creaks), and to offload thought (e.g., by tapping surfaces to recall musical pieces). Therefore, the ability to make sense of and exploit sounds in motion is a fundamental prerequisite for embodied Artificial Intelligence (AI). This project will pioneer the underpinning, probabilistic framework for active robot audition that enables embodied agents to control the motion of their own bodies ('ego-motion') for auditory attention in realistic, acoustic environments (households, public spaces, and environments involving multiple, competing sound sources).By integrating sound with motion, this project will enable machines to imagine, control and leverage the auditory consequences of physical interactions with the environment. By transforming the ways in which machines make sense of life in sound, the research outcomes will be pivotal for new, emerging markets that enable robots to augment, rather than rival, humans in order to surpass the limitations of the human body (sensory accuracy, strength, endurance, memory). Therefore, the proposed research has the potential to transform and disrupt a whole host of industries involving machine listening, ranging from human-robot augmentation (smart prosthetics, assistive listening technology, brain-computer interfaces) to human-robot collaboration (planetary exploration, search-and-rescue, hazardous material removal) and automation (environmental monitoring, autonomous vehicles, AI-assisted diagnosis in healthcare). This project will consider the specific case study of a collaborative robot ('cobot') that augments the auditory experience of a hearing-impaired human partner. Hearing loss is the second most common disability in the UK, affecting 11M people. The loss of hearing affects situational awareness as well as the ability to communicate, which can impact on mental health and, in extreme cases, cognitive function. Nevertheless, for complex reasons that range from discomfort to social stigma, only 2M people choose to wear hearing aids.The ambition of this project is to develop a cobot that will augment the auditory experience of a hearing-impaired person. The cobot will move autonomously within the human partner's household to assist with everyday tasks. Our research will enable the cobot to exploit ego-motion in order to learn an internal representation of the acoustic scene (children chattering, kettle boiling, spouse calling for help). The cobot will interface with its partner through an on-person smart device (watch, mobile phone). Using the human-cobot interface, the cobot will alert its partner of salient events (call for help) via vibrating messages, and share its auditory experiences via interactive maps that visualise auditory cues and indicate saliency (e.g., loudness, spontaneity) and valence (positive vs concerning).In contrast to smart devices, the cobot will have the unique capability to actively attend to and explore uncertain events (thump upstairs), and take action (assist spouse, call ambulance) without the need for permanently installed devices in personal spaces (bathroom, bedroom). Therefore, the project has the potential to transform the lives of people with hearing impairments by enabling long-term independent living, safeguarding privacy, and fostering inclusivity.