Unifying audio signal processing and machine learning: a fundamental framework for machine hearing

现代技术正在导致大量的音频数据。例如，每分钟有超过72小时的非结构化和未标记的音轨被上传到互联网网站。迫切需要自动系统来识别音频内容，以便这些音轨可以被标记用于分类和搜索。此外，越来越多的录音是在含有多个声源和噪音的挑战性环境中在手持设备上进行的。这种未经策划和嘈杂的数据需要自动系统来清理音频内容并从混合物中分离源。与此相关的是，目前用于听力受损者的设备在噪声中表现不佳。事实上，这就是为什么英国有600万人可以从助听器中受益，但他们不使用助听器的主要原因（每年价值180亿英镑的市场）。安装人工耳蜗的患者也受到类似的限制，随着人口老龄化，更多的人受到影响。很明显，音频识别和增强方法需要阻止我们淹没在音频数据中，用于在听力设备中处理，并支持新的技术创新。目前解决这些问题的方法使用音频信号处理（将音频数据转换为方便的格式并降低数据速率）和机器学习（去除噪声，分离源或对内容进行分类）的组合。人们普遍认为，这两个领域今后必须日益融合。然而，这个联盟目前是一个麻烦的联盟，有四个问题。效率低下：当我们有大量的数据时，这些方法效率太低（如对于web上的音轨的情况）或对于实时应用（如助听器中所必需的）贫困模型：机器学习模块往往在统计上受到限制。信号处理模块是不适应的，尽管其他领域的证据，如计算机视觉，这表明自动调谐导致显著的性能增益信号处理模块引入非机器学习模块无法捕获的线性失真。在这个项目中，我们通过引入一个新的理论框架来解决这四个限制它将信号处理和机器学习结合在一起。关键的一步是将信号处理模块视为解决推理问题。由于机器学习模块通常以这种方式构建，因此这两个模块可以集成到一个统一的方法中，从而使两个领域的技术完全集成。在该项目中，我们将使用新方法来开发高效，丰富，自适应和无失真的音频去噪，源分离和识别方法。我们将评估降噪和源分离算法对听力障碍者的影响，以及音频-声音轨迹数据的音频识别算法。我们相信这个新的框架将为新兴的机器听觉领域奠定基础。未来，机器听觉将被部署在从音乐处理任务到增强现实系统（结合计算机视觉技术）的广泛应用中。我们认为，这一项目将启动这一扩散。

Modern technology is leading to a flood of audio data. For example, over seventy two hours of unstructured and unlabelled sound-tracks are uploaded to internet sites every minute. Automatic systems are urgently needed for recognising audio content so that these sound-tracks can be tagged for categorisation and search. Moreover, an increasing proportion of recordings are made on hand-held devices in challenging environments that contain multiple sound sources and noise. Such uncurated and noisy data necessitate automatic systems for cleaning the audio content and separating sources from mixtures. On a related note, devices for the hearing impaired currently perform poorly in noise. In fact, this is a major reason why six million people in the UK who would benefit from a hearing aid, do not use them (a market worth £18 billion p.a.). Patients fitted with cochlear implants suffer from similar limitations, and as the population ages more people are affected. It is clear that audio recognition and enhancement methods are required to stop us drowning in audio-data, for processing in hearing devices, and tosupport new technological innovations. Current approaches to these problems use a combination of audio signal processing (which places the audio data into a convenient format and reduces the data-rate) and machine learning (which removes noise, separates sources, or classifies the content). It is widely believed that these two fields must become increasingly integrated in the future. However, this union is currently a troubled one, suffering from four problems. Inefficiency: The methods are too inefficient when we have vast amounts of data (as is the case for audio-tracks on the web) or for real-time applications (such as is necessary in hearing aids)Impoverished models: The machine learning modules tend to be statistically limited.Unadapted: The signal processing modules are unadapted despite evidence from other fields, like computer vision, which suggests that automatic tuning leads to significant performance gains Distorted mixtures: The signal processing modules introduce non-linear distortions which are not captured by the machine learning modules.In this project we address these four limitations by introducing a new theoretical framework which unifies signal processing and machine learning. The key step is to view the signal processing module as solving an inference problem. Since the machine-learning modules are often framed in this way, the two modules can be integrated into a single coherent approach allowing technologies from the two fields to be completely integrated. In the project we will then use the new approach to develop efficient, rich, adaptive, and distortion free approaches to audio denoising, source separation and recognition. We will evaluate the the noise reduction and source separations algorithms on the hearing impaired, and the audio recognition algorithms on audio-sound track data.We believe this new framework will form a foundation of the emerging field of machine hearing. In the future, machine hearing will be deployed in a vast range of applications from music processing tasks to augmented reality systems (in conjunction with technologies from computer vision). We believe that this project will kick start this proliferation.