Enhancing Audio Transformation through the Integration of Machine Learning and Digital Signal Processing Techniques

机器学习技术已经在各种音频处理应用中表现出巨大的潜力。然而，目前人工智能研究的趋势倾向于更大、计算更密集的“黑匣子”模型。虽然这些模型可以实现令人印象深刻的性能，但它们往往缺乏与音乐家的特定艺术视野和要求保持一致所需的透明度和调整能力。为了在音乐制作中真正有用，音频转换工具需要响应迅速，高度可定制，并能够实时执行。这个博士项目的目标是探索将经典数字信号处理（DSP）技术集成到机器学习框架中，以创建快速，模块化和创造性的音频转换模型。通过利用传统DSP算法中使用的音乐理论结构与机器学习的扩展生成建模功能，本研究旨在实现两个主要目标：提高基于机器学习的音频转换模型的鲁棒性，可控性和透明度扩展DSP算法的功能集和功能，同时保持其轻量级和实时性能优势。首先，它涉及将传统的DSP算法转换为用于音频生成和转换的参数驱动工具。其次，该项目旨在设计轻量级机器学习模型，通过利用这些算法中固有的音乐理论基础，降低训练过程中的计算成本，并增强模型对小训练集的适应能力。最后，该项目旨在实现提高这些模型的推理效率的技术，即使在计算资源有限的基础计算机上也能实现快速性能。通过弥合经典DSP技术与音频处理机器学习框架之间的差距，该项目旨在开发强大而透明的音频转换模型。通过整合传统DSP算法中使用的音乐理论结构，同时利用机器学习的力量，将创建一个新颖的音乐音频转换和生成器工具包。该工具包将为音乐家提供增强的灵活性、响应能力和实时性能，最终推动音乐制作领域的音频处理技术。

Machine learning techniques have demonstrated significant potential in various audio processing applications. However, the current trend in AI research favours larger, more computationally intensive "black box" models. While these models may achieve impressive performance, they often lack the transparency and tweak-ability necessary to align with the specific artistic vision and requirements of musicians. To be truly useful in music production, audio transformation tools need to be responsive, highly customisable, and capable of real-time performance.The objective of this PhD project is to explore the integration of classical digital signal processing (DSP) techniques into machine learning frameworks to create fast, modular and creative audio transformation models. By leveraging the explicit use of musical theoretical structures employed in traditional DSP algorithms with the expanded generative modelling capabilities of machine learning, this research aims to achieve two primary goals:Improving the robustness, controllability and transparency of machine learning-based audio transformation modelsExpanding the feature set and capabilities of DSP algorithms while maintaining their lightweight and real-time performance benefits.The project aims to accomplish these objectives through several key tasks. Firstly, it involves transforming traditional DSP algorithms into parametrically driven tools for audio generation and transformation. Secondly, the project seeks to design lightweight machine learning models that, by leveraging the music theoretical foundation inherent in these algorithms, reduce computational costs during training and enhance the models' resilience to small training sets. Lastly, the project aims to implement techniques that enhance the inference efficiency of these models, enabling fast performance even on basic computers with limited compute resources.By bridging the gap between classical DSP techniques and machine learning frameworks for audio processing, this project seeks to develop robust and transparent audio transformation models. By incorporating the musical theoretical structures used in traditional DSP algorithms while harnessing the power of machine learning, a toolkit of novel musical audio transformations and generators will be created. This toolkit will offer musicians enhanced flexibility, responsiveness, and real-time performance, ultimately advancing audio processing technology in the music production domain.