A review of differentiable digital signal processing for music and speech synthesis
A review of differentiable digital signal processing for music and speech synthesis
复制标题
DOI：
10.3389/frsip.2023.1284100
复制
发表时间：
2024-01
期刊：
影响因子：
--
通讯作者：
B. Hayes;Jordie Shier;Gyorgy Fazekas;Andrew McPherson;C. Saitis
中科院分区：
文献类型：
--
作者：
B. Hayes;Jordie Shier;Gyorgy Fazekas;Andrew McPherson;C. Saitis
术语“可微分数字信号处理”描述了一系列技术，其中损失函数梯度通过数字信号处理器反向传播，促进其集成到神经网络中。本文综述了可微分音频信号处理的文献，重点介绍了它在音乐和语音合成中的应用。我们目录的应用程序的任务，包括音乐表演渲染，声音匹配，和语音转换，讨论使用这种方法的动机和影响。这是伴随着数字信号处理操作的概述，已经实现了微分，这是进一步支持的网络书籍包含实用的建议微分合成器编程（https：//intro2ddsp.github.io/）。最后，我们强调了开放的挑战，包括优化病理学，对现实世界条件的鲁棒性和设计权衡，并讨论了未来的研究方向。
The term “differentiable digital signal processing” describes a family of techniques in which loss function gradients are backpropagated through digital signal processors, facilitating their integration into neural networks. This article surveys the literature on differentiable audio signal processing, focusing on its use in music and speech synthesis. We catalogue applications to tasks including music performance rendering, sound matching, and voice transformation, discussing the motivations for and implications of the use of this methodology. This is accompanied by an overview of digital signal processing operations that have been implemented differentiably, which is further supported by a web book containing practical advice on differentiable synthesiser programming (https://intro2ddsp.github.io/). Finally, we highlight open challenges, including optimisation pathologies, robustness to real-world conditions, and design trade-offs, and discuss directions for future research.