Unifying audio signal processing and machine learning: a fundamental framework for machine hearing

现代技术带来了大量的音频数据。例如，每分钟有超过72小时的未结构化和未标记的音轨上传到互联网网站。人们迫切需要自动系统来识别音频内容，以便对这些音轨进行分类和搜索。此外，在包含多种声源和噪音的挑战性环境中，越来越多的录音是在手持设备上进行的。这种未经整理和嘈杂的数据需要自动系统来清洗音频内容和从混合物中分离源。与此相关的是，听力受损的设备目前在噪音方面表现不佳。事实上，这就是为什么英国有600万人本可以从助听器中受益，却没有使用助听器的主要原因（一个每年价值180亿英镑的市场）。植入人工耳蜗的患者也受到类似的限制，随着人口老龄化，越来越多的人受到影响。很明显，我们需要音频识别和增强方法来阻止我们淹没在音频数据中，以便在听力设备中进行处理，并支持新的技术创新。目前解决这些问题的方法结合了音频信号处理（将音频数据转换为方便的格式并降低数据速率）和机器学习（去除噪声、分离源或对内容进行分类）。人们普遍认为，这两个领域在未来必将日益融合。然而，这一联盟目前陷入困境，面临四大问题。效率低下：当我们有大量的数据（比如网络上的音轨）或实时应用（比如助听器）时，这些方法的效率太低了。贫穷的模型：机器学习模块往往在统计上受到限制。未适应：信号处理模块未适应，尽管来自其他领域的证据，如计算机视觉，表明自动调谐导致显著的性能提升。扭曲的混合：信号处理模块引入了机器学习模块无法捕获的非线性扭曲。在这个项目中，我们通过引入一个统一信号处理和机器学习的新理论框架来解决这四个限制。关键步骤是将信号处理模块视为解决推理问题。由于机器学习模块通常以这种方式构建，这两个模块可以集成到一个统一的方法中，从而使两个领域的技术完全集成。在这个项目中，我们将使用新的方法来开发高效、丰富、自适应和无失真的音频去噪、源分离和识别方法。我们将评估针对听力障碍的降噪和声源分离算法，以及针对音轨数据的音频识别算法。我们相信这个新框架将为机器听力这一新兴领域奠定基础。在未来，机器听力将被广泛应用于从音乐处理任务到增强现实系统（与计算机视觉技术结合）的各种应用中。我们相信这个项目将启动这种扩散。

Modern technology is leading to a flood of audio data. For example, over seventy two hours of unstructured and unlabelled sound-tracks are uploaded to internet sites every minute. Automatic systems are urgently needed for recognising audio content so that these sound-tracks can be tagged for categorisation and search. Moreover, an increasing proportion of recordings are made on hand-held devices in challenging environments that contain multiple sound sources and noise. Such uncurated and noisy data necessitate automatic systems for cleaning the audio content and separating sources from mixtures. On a related note, devices for the hearing impaired currently perform poorly in noise. In fact, this is a major reason why six million people in the UK who would benefit from a hearing aid, do not use them (a market worth £18 billion p.a.). Patients fitted with cochlear implants suffer from similar limitations, and as the population ages more people are affected. It is clear that audio recognition and enhancement methods are required to stop us drowning in audio-data, for processing in hearing devices, and tosupport new technological innovations. Current approaches to these problems use a combination of audio signal processing (which places the audio data into a convenient format and reduces the data-rate) and machine learning (which removes noise, separates sources, or classifies the content). It is widely believed that these two fields must become increasingly integrated in the future. However, this union is currently a troubled one, suffering from four problems. Inefficiency: The methods are too inefficient when we have vast amounts of data (as is the case for audio-tracks on the web) or for real-time applications (such as is necessary in hearing aids)Impoverished models: The machine learning modules tend to be statistically limited.Unadapted: The signal processing modules are unadapted despite evidence from other fields, like computer vision, which suggests that automatic tuning leads to significant performance gains Distorted mixtures: The signal processing modules introduce non-linear distortions which are not captured by the machine learning modules.In this project we address these four limitations by introducing a new theoretical framework which unifies signal processing and machine learning. The key step is to view the signal processing module as solving an inference problem. Since the machine-learning modules are often framed in this way, the two modules can be integrated into a single coherent approach allowing technologies from the two fields to be completely integrated. In the project we will then use the new approach to develop efficient, rich, adaptive, and distortion free approaches to audio denoising, source separation and recognition. We will evaluate the the noise reduction and source separations algorithms on the hearing impaired, and the audio recognition algorithms on audio-sound track data.We believe this new framework will form a foundation of the emerging field of machine hearing. In the future, machine hearing will be deployed in a vast range of applications from music processing tasks to augmented reality systems (in conjunction with technologies from computer vision). We believe that this project will kick start this proliferation.