Using touch to enhance auditory perception

使用触摸来增强听觉感知

• 批准号: EP/W032422/1
• 负责人: James Rankin
• 金额: $ 72.39万
• 依托单位: UNIVERSITY OF EXETER
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FW032422%2F1
• 关键词: Using; touch; enhance; auditory; perception

Difficulty in following a single voice or conversation in noisy situations is one of the main frustrations for people with hearing impairment, even when fitted with modern hearing aids. Tactile (touch) sensation combined with sound is widely used in commonplace technology like mobile phones and video game controllers. This research investigates whether tactile stimulation (e.g. vibrations on the wrist), if synchronised with the sound around us, could improve our ability to separate out different sound sources (e.g. voices) in noisy situations. The research aims to demonstrate the benefit of this approach to help people with hearing loss, which leads to significantly reduced quality of life, impacts productivity in working life, affects 1 in 6 individuals in the UK and costs the NHS of 450 million pounds per year.Sound sources can be separated based on differences in e.g. their pitch, location and timing: people talk at different speeds and start/stop stop talking at different times. One of the main features used to distinguish voices is their so-called fundamental frequency, e.g. male voices tend to sound lower than female voices. We are able to distinguish tactile vibrations in this same range of fundamental frequencies. If some important sound information, perhaps missing due to hearing loss, is automatically transformed into tactile stimulation, could this help to separate one voice from another? This research will use behavioural experiments with carefully designed combinations of sound and touch stimuli to answer this question. It aims to demonstrate the constraints for how this approach works best, firstly for normal-hearing listeners and then for people with hearing loss. The research will be supported by computational modelling to simulate processing in the brain that combines information from our senses of hearing and touch. This will help to link our own findings from behavioural experiments to research with brain imaging and other approaches used to understand how sound is processed with information from other senses. In summary, the project investigates how tactile sensation affects sound source separation, how its effect is modified with impaired hearing and how information across the senses is combined through computations in the brain. These advances will provide a foundation of knowledge to improve strategies and technology to help with people impaired hearing by supporting the development of hearing aids that are enhanced by touch stimulation, e.g. on the wrist.

即使配备了现代助听器，在嘈杂的环境中难以跟随单一声音或对话也是听力障碍人士的主要挫败感之一。触觉（触摸）感觉与声音相结合广泛应用于移动的电话和视频游戏控制器等普通技术中。这项研究调查了触觉刺激（例如手腕上的振动），如果与我们周围的声音同步，是否可以提高我们在嘈杂环境中区分不同声源（例如声音）的能力。该研究旨在证明这种方法对帮助听力损失患者的好处，听力损失导致生活质量显著降低，影响工作生活中的生产力，影响英国六分之一的人，每年花费NHS 4.5亿英镑。声源可以根据音调，位置和时间的差异进行分离：人们以不同的速度说话，并且在不同的时间开始/停止说话。用于区分声音的主要特征之一是所谓的基频，例如男性声音往往比女性声音低。我们能够在相同的基频范围内区分触觉振动。如果一些重要的声音信息（可能由于听力损失而丢失）自动转换为触觉刺激，这是否有助于将一个声音与另一个声音分开？这项研究将使用精心设计的声音和触摸刺激组合的行为实验来回答这个问题。它旨在展示这种方法如何最好地工作的限制，首先是正常听力的听众，然后是听力损失的人。这项研究将得到计算模型的支持，以模拟大脑中结合听觉和触觉信息的处理过程。这将有助于将我们自己从行为实验中的发现与大脑成像和其他方法的研究联系起来，这些方法用于了解声音如何与其他感官的信息一起处理。总之，该项目研究了触觉如何影响声源分离，其效果如何随着听力受损而改变，以及如何通过大脑中的计算将感官信息结合起来。这些进步将为改善策略和技术提供知识基础，通过支持开发通过触摸刺激（例如手腕上的触摸刺激）增强的助听器来帮助听力受损的人。

项目成果

Improving speech perception for hearing-impaired listeners using audio-to-tactile sensory substitution with multiple frequency channels.

• DOI: 10.1038/s41598-023-40509-7
• 发表时间: 2023-08-16
• 期刊: Scientific reports
• 影响因子: 4.6