Music4u: Personalized Objective Deep Learning Models to Make Music More Accessible for Cochlear Implant Users

音乐在人们的生活中扮演着重要的角色，是许多社会文化和教育活动的一部分。音乐是最复杂的声音信号，因为它利用了人类听觉系统的全部动态、带宽和分辨率。此外，音乐是人类的一部分，通过情感将不同地点和思维模式的人联系在一起。人工耳蜗可以恢复听力受损或失聪的人的听力，但仅仅是为了恢复语言的可理解性，而不是其他声学信号。因此，这些设备无法恢复听障人士的音乐知觉。根据世界卫生组织的数据，全球约有3.6亿人患有听力损失。听力损失极大地限制了人际交流的程度，经常导致社会孤立，并已发展成为一个重要的社会经济因素。在过去的二十年里，人工耳蜗的研究主要集中在提高噪声中的语音性能上。然而，最近的科学证据表明，音乐是人类大脑发育的重要听觉输入--就认知、情感和听觉处理能力而言。Music4u改变了人工耳蜗研究的基本视角，因此将使用音乐技术作为提高听力表现的关键，从而提高人工耳蜗者的生活质量。Music4u研究如何让人工耳机用户更容易接触到器乐，考虑到他们的个人听力表现。首先，将调查音乐的基本元素之间的首选平衡，以使其对人工耳蜗用户更有乐趣。这种基本的理解将被用来设计一种新的信号处理算法，致力于改善音乐感知和加速他们的听力表现。该算法基于深度神经网络进行信源分离和后验增强。该算法旨在改进器乐的源分离算法。在后续阶段，将降低算法的复杂度，以显示其在人工耳蜗声处理器中的潜在应用。该算法的个性化是通过仪器辨别的电生理测量进行的。这项措施的第一个候选者是基于对乐器数量较少的复调音乐选段的选择性关注。将电生理测量的性能与实际人工耳蜗者的仪器辨别行为测量进行比较。总而言之，Music4u项目旨在进行基础研究，设计一种可以为每个人工耳蜗用户提供个性化的技术，以改善他们的音乐体验，目的是通过将他们融入社交音乐活动来提高他们的生活质量。

Music plays an essential role in people’s lives and is part of many socio-cultural and educational events. Music is the most complex acoustic signal as it uses the full dynamic, bandwidth, and resolution of the human auditory system. Moreover, music is part of being human and connects people through emotion across places and mind-sets. Cochlear implants can restore hearing for the hearing impaired or deaf but have been solely designed to restore speech intelligibility rather than other acoustic signals. For this reason, these devices fail at restoring music perception for the hearing impaired. According to the World Health Organization around 360 million people worldwide suffer from hearing loss. Hearing loss significantly limits the extent of interpersonal communication, often leads to social isolation, and has developed into a significant socio-economic factor. Over the last two decades research in cochlear implants has mainly been focused on improving speech performance in noise. However, recent scientific evidence points at music as an important auditory input for the development of the human brain – in terms of cognitive, emotional as well as auditory processing abilities. Music4u changes the fundamental perspective of cochlear implant research and therefore will use music technology as the key to improve the hearing performance and consequently the quality of life of cochlear implant users. Music4u investigates how to make instrumental music more accessible for cochlear implant users considering their individual hearing performance. First the preferred balance between the basic elements of the music to make it more enjoyable for cochlear implant users will be investigated. This fundamental understanding will be used to design a new signal processing algorithm dedicated to improve music perception and to accelerate their hearing performance. The algorithm is based on a deep neural network for source separation and posterior enhancement. The algorithm is intended to improve state of the art source separation algorithms for instrumental music. In a subsequent phase the complexity of the algorithm will be reduced to show its potential application in cochlear implant sound processor. The personalization of the algorithm is conducted through an electrophysiological measure of instrument discrimination. The first candidate for this measure is based on selective attention to polyphonic music excerpts with a low number of instruments. The performance with the electrophysiological measure will be compared to behavioral measures of instrument discrimination in actual cochlear implant users. In summary, the music4u project aims at conducting basic research to design a technology that can be personalized to each cochlear implant user to improve their music experience with the aim to improve their quality of life by integrating them into social-musical activities.