Enhancing Audio Transformation through the Integration of Machine Learning and Digital Signal Processing Techniques

机器学习技术在各种音频处理应用中显示出了巨大的潜力。然而，目前人工智能研究的趋势倾向于更大、计算更密集的“黑匣子”模型。虽然这些模型可能会取得令人印象深刻的表现，但它们往往缺乏必要的透明度和调整能力，无法与音乐家的特定艺术愿景和要求保持一致。为了在音乐制作中发挥真正的作用，音频转换工具需要具有响应性、高度可定制性和实时性能。该博士项目的目标是探索将经典数字信号处理(DSP)技术集成到机器学习框架中，以创建快速、模块化和创造性的音频转换模型。通过将传统数字信号处理算法中的音乐理论结构与机器学习的生成性建模能力相结合，本研究旨在实现两个主要目标：提高基于机器学习的音频转换模型的健壮性、可控性和透明性；扩展数字信号处理算法的特征集和能力，同时保持其轻量级和实时的性能优势。首先，它涉及到将传统的DSP算法转换为参数驱动的音频生成和转换工具。其次，该项目寻求设计轻量级机器学习模型，通过利用这些算法固有的音乐理论基础，降低训练期间的计算成本，并增强模型对小训练集的弹性。最后，该项目旨在实现提高这些模型的推理效率的技术，即使在计算资源有限的基本计算机上也能实现快速性能。通过弥合经典DSP技术和用于音频处理的机器学习框架之间的差距，该项目寻求开发健壮和透明的音频转换模型。通过结合传统DSP算法中使用的音乐理论结构，同时利用机器学习的力量，将创建一个新的音乐音频转换和生成器工具包。这一工具包将为音乐家提供更高的灵活性、响应性和实时性能，最终推动音乐制作领域的音频处理技术。

Machine learning techniques have demonstrated significant potential in various audio processing applications. However, the current trend in AI research favours larger, more computationally intensive "black box" models. While these models may achieve impressive performance, they often lack the transparency and tweak-ability necessary to align with the specific artistic vision and requirements of musicians. To be truly useful in music production, audio transformation tools need to be responsive, highly customisable, and capable of real-time performance.The objective of this PhD project is to explore the integration of classical digital signal processing (DSP) techniques into machine learning frameworks to create fast, modular and creative audio transformation models. By leveraging the explicit use of musical theoretical structures employed in traditional DSP algorithms with the expanded generative modelling capabilities of machine learning, this research aims to achieve two primary goals:Improving the robustness, controllability and transparency of machine learning-based audio transformation modelsExpanding the feature set and capabilities of DSP algorithms while maintaining their lightweight and real-time performance benefits.The project aims to accomplish these objectives through several key tasks. Firstly, it involves transforming traditional DSP algorithms into parametrically driven tools for audio generation and transformation. Secondly, the project seeks to design lightweight machine learning models that, by leveraging the music theoretical foundation inherent in these algorithms, reduce computational costs during training and enhance the models' resilience to small training sets. Lastly, the project aims to implement techniques that enhance the inference efficiency of these models, enabling fast performance even on basic computers with limited compute resources.By bridging the gap between classical DSP techniques and machine learning frameworks for audio processing, this project seeks to develop robust and transparent audio transformation models. By incorporating the musical theoretical structures used in traditional DSP algorithms while harnessing the power of machine learning, a toolkit of novel musical audio transformations and generators will be created. This toolkit will offer musicians enhanced flexibility, responsiveness, and real-time performance, ultimately advancing audio processing technology in the music production domain.