Music-visual feature mapping and deep tagging

阿波罗音乐商店是一家总部位于蒙特利尔的公司，专门从事数字音乐的授权和发行。他们代表数百名艺术家，其中超过75%是加拿大人。该项目的目标是改善阿波罗客户的用户体验，这些客户正试图为广告、游戏和其他应用程序寻找特定类型的音乐。我们将追求两个主要目标。第一个是探索如何基于音视频特征提取和映射来自动将库存视频与音乐轨道关联和调整。第二个目标是使用深度学习技术来构建自动使用描述性关键字标记音乐的软件。深度学习已被证明在各种机器学习任务中非常有效。数据扩充和半监督学习是在标签数据量有限的情况下训练深度神经网络的技术。阿波罗音乐将提供对他们的数字目录的访问，以及他们的策展人提供的关于曲目的现有元数据。Tzanetakis博士的研究小组在音乐信息检索和机器学习方面拥有强大的国际影响力，这两个研究领域将为该项目贡献想法、概念和算法。两名在计算机视觉/深度学习/音频信号处理和手势识别/人机交互方面经验互补的博士生将通过该项目获得资助，并与阿波罗音乐商店的设计师和工程师合作。

Apollo Music Store is a Montreal based company that specializes on licensing and distributing digitally music.They represented hundreds of artists and more than 75% of them are Canadian. The goal of this project is toimprove the user experience of Apollo customers who are trying to find specific kinds of music for advertising,games and other applications. Two main objectives will be pursued. The first is to explore how toautomatically associate and adjust stock video with music tracks based on audio and visual feature extractionand mapping. The second objective is to use deep learning techniques to build software that automatically tagsthe music with descriptive keywords. Deep learning has been shown to be very effective in a variety ofmachine learning tasks. Data augmentation and semi-supervised learning are techniques used to train deepneural networks when the amount of labeled data is limited. Apollo Music will provide access to their digitalcatalog as well as existing meta-data their curators have provided about the tracks. The research group ofDr.Tzanetakis has a strong international presence in Music Information Retrieval and Machine Learning.which are research areas that will contribute ideas, concepts, and algorithms to the project. Two PhD studentswith complementary experience in computer vision/deep learning/audio signal processing and gesturerecognition/human-computer interaction will be funded through the project and work with the designers andengineers at Apollo Music Store.