Jazz Solo Instrument Classification with Convolutional Neural Networks, Source Separation, and Transfer Learning
Jazz Solo Instrument Classification with Convolutional Neural Networks, Source Separation, and Transfer Learning
复制标题
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
J. S. Gomez;J. Abeßer;Estefanía Cano
中科院分区：
文献类型：
--
作者：
J. S. Gomez;J. Abeßer;Estefanía Cano
主要的乐器识别在合奏录音仍然是一个具有挑战性的任务，特别是如果密切相关的乐器，如中音和次中音萨克斯管需要区分。在本文中，我们基于最近提出的基于混合深度神经网络的仪器识别算法：卷积层和全连接层的组合，用于学习特征频谱-时间模式。我们系统地评估了谐波/打击乐和独奏/伴奏源分离算法作为预处理步骤，以减少乐器识别步骤之前多个乐器之间的重叠。对于爵士合奏录音中独奏乐器识别的特定用例，我们进一步应用迁移学习技术对先前训练的乐器识别模型进行微调，以对六种爵士独奏乐器进行分类。我们的研究结果表明，作为预处理步骤的源分离和迁移学习都明显提高了识别性能，特别是对于高度相似的工具的较小子集。
Predominant instrument recognition in ensemble recordings remains a challenging task, particularly if closely-related instruments such as alto and tenor saxophone need to be distinguished. In this paper, we build upon a recently-proposed instrument recognition algorithm based on a hybrid deep neural network: a combination of convolutional and fully connected layers for learning characteristic spectral-temporal patterns. We systematically evaluate harmonic/percussive and solo/accompaniment source separation algorithms as pre-processing steps to reduce the overlap among multiple instruments prior to the instrument recognition step. For the particular use-case of solo instrument recognition in jazz ensemble recordings, we further apply transfer learning techniques to fine-tune a previously trained instrument recognition model for classifying six jazz solo instruments. Our results indicate that both source separation as pre-processing step as well as transfer learning clearly improve recognition performance, especially for smaller subsets of highly similar instruments.