An Integrated Audio-Symbolic Model of Music Similarity

相似性是音乐的一个核心方面。它可以作为测试音乐感知和认知理论的基准，有助于回答实际应用，如音乐检索和推荐，它是一个涉及音乐许多方面的首要问题。然而，音乐相似性理论大多集中在符号表征上，其中音乐结构如旋律和和声的发展被解决，但表现表现的方面，如微计时和音色被忽视。另一方面，基于音频的模型，通常是基于音频特征的距离，可以捕捉到诸如音色、动态和节奏变化等表演细节，但却无法捕捉到随着时间的推移而展开的音乐结构。数字音乐实验室（DML）项目正在开发的音频转录和校准方面的最新进展，以及用于包含音频和符号内容的音乐收藏的音乐分析工具的可用性，正在改变音乐研究的前景。通过DML和“多源光学音乐识别”大数据项目生成或公布的声学和符号音乐数据的大型数据集鼓励一种方法，可以将基于符号和音频的分析结合到联合相似模式中。这为音乐信息检索和音乐学研究的新工具开辟了巨大的潜力，因为符号结构和音质纹理等声学信息之间的相互作用很少得到解决，它可以揭示迄今为止未被注意或无法解释的方面。如果成功，将有助于打破音乐推荐领域的玻璃天花板。本项目的目的是建立一个初始框架，并对符号旋律和结构相似性模型与基于音频的模型的整合进行实验。模型捕捉各种相似性概念的能力将根据音乐收藏中的文化信息（例如流派、风格、作曲家）以及用户注释或相似性评级进行评估。这项工作将建立在参与者建模音频相似性（Wolff & Weyde 2013），音频转录和基于符号和音频的模型集成（Benetos, Ewert & Weyde 2014），符号旋律相似性（Marsden 2012）以及概率音乐和表演建模（Spiro, Gold & Rink 2010, Abdallah et al 2012）的经验之上。

Similarity is a central aspect of music. It can help as a benchmark for testing theories of music perception and cognition, it is useful to answer for practical applications, such as music retrieval and recommendation, and it is an overarching question that relates to many aspects of music. However, theories of musical similarity have mostly focused on symbolic representations, where musical structures such as melodic and harmonic development are addressed but the aspects or expressive performance, such as micro-timing and timbre are ignored. On the other hand, audio based models, typically distances based on audio features, can capture details of the performance such as timbre, dynamics and tempo changes, but little of the musical structure as it unfolds over time.Recent progress in audio transcription and alignment and the availability of music analysis tools for music collections with audio and symbolic content, which are being developed in the Digital Music Lab (DML) project, are changing the landscape of research in music. Large datasets of acoustic and symbolic music data that are generated or unveiled through the DML and "Optical Music Recognition from Multiple Sources" Big Data projects encourage an approach that can combine symbolic and audio based analyses into a joint similarity mode. This opens a great potential to new tools for research in music information retrieval and musicology, as the interaction between symbolic structure and acoustic information such as timbral texture has rarely been addressed and it could reveal aspects that have been unnoticed or unexplained so far. If successful, it might contribute to breaking the glass ceiling in music recommendation. The aim of this project is to develop an initial framework and conduct experiments on an integration of symbolic melodic and structural similarity models with audio based models. The models ability to capture various notions of similarity will be evaluated on cultural information in music collections (e.g. genre, style, composer) as well as user annotations or ratings of similarity. This work will build on the experience of the participants in modelling audio similarity (Wolff & Weyde 2013), audio transcription and the integration of symbolic and audio based models (Benetos, Ewert & Weyde 2014), symbolic melodic similarity (Marsden 2012) and probabilistic music and performance modelling (Spiro, Gold & Rink 2010, Abdallah et al 2012).