Computational Analysis of Music Audio Recordings: A Cross-Version Approach

音乐录音的计算分析：跨版本方法

• 批准号: 531250483
• 负责人: Professor Dr.-Ing. Christof Weiß
• 金额: --
• 依托单位: Center for Artificial Intelligence and Data Science
• 依托单位国家: 德国
• 项目类别: Independent Junior Research Groups
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/531250483?language=en
• 关键词: Computational; Analysis; Music; Audio; Recordings

The computational analysis of music audio recordings constitutes a highly interdisciplinary research area, involving domain knowledge from musicology and music theory as well as methods from signal processing and machine learning. From a computer science perspective, the variety and complexity of music audio data poses enormous challenges, which are specific to this domain. First, music is multi-faceted, being characterized by different semantic dimensions such as time, pitch, timbre, or style, which need to be disengtangled to obtain interpretable representations. Second, music analysis comprises hierarchically related tasks such as the estimation of pitches, chords, local keys, and global keys or the detection of onsets, beats, downbeats, and structural boundaries, suggesting the use of multi-task approaches. Third, music data is complex, consisting of highly correlated sources whose components overlap in time and frequency. Furthermore, musical notions are often ambiguous and subjective, thus demanding for interpretable methods and multiple annotators. Fourth, music scenarios are often data-scarce, lacking the availability of large amounts of annotated data. As a consequence, analysis methods frequently overfit to implicit biases in the training datasets, become sensitive to small perturbations, and do not generalize well to unseen data. The data scarcity poses particular challenges for deep-learning approaches, which are nowadays dominating the field. These approaches achieved substantial improvements for many music analysis tasks but often hit a kind of "glass ceiling" above which further progress is hard to achieve and to measure. To overcome this problem, this project adopts a cross-version approach by exploiting datasets of classical music, which contain several modalities (score and audio), several performances (interpretations and arrangements), and several annotations (multiple experts) for each musical work. Such datasets allow for transferring annotations between versions and for systematically evaluating the robustness of deep-learning methods by testing generalization along different dimensions, e. g., to other versions of a work, other works by a composer, or other composers from a historical period. As a main conceptual contribution, we apply and further develop such cross-version strategies, exploiting them to better understand the analysis methods and to improve these methods using suitable training and fusion strategies. Based on this cross-version approach, we address the specific challenges of music data, aiming for analysis methods that are of particular use for computational musicology, and progressing towards novel methodological strategies in the wider field of the digital humanities.

音乐录音的计算分析是一个高度交叉的研究领域，涉及音乐学和音乐理论的领域知识，以及信号处理和机器学习的方法。从计算机科学的角度来看，音乐音频数据的多样性和复杂性带来了巨大的挑战，这是该领域特有的。首先，音乐是多方面的，具有不同的语义维度，如时间、音高、音色或风格，需要分离这些维度以获得可解释的表示。其次，音乐分析包括层次化的相关任务，如估计音高、和弦、局部音调和全局音调，或检测开始音、节拍、下节拍和结构边界，建议使用多任务方法。第三，音乐数据是复杂的，由高度相关的源组成，这些源的各分量在时间和频率上重叠。此外，音乐概念往往是模棱两可的和主观的，因此需要可解释的方法和多个注释者。第四，音乐场景往往是数据稀缺的，缺乏大量注释数据的可用性。因此，分析方法经常过度适应训练数据集中的隐含偏差，对小的扰动变得敏感，并且不能很好地概括到看不见的数据。数据稀缺给深度学习方法带来了特别的挑战，这些方法目前在该领域占据主导地位。这些方法在许多音乐分析任务上取得了实质性的改进，但往往会遇到一种“玻璃天花板”，超过这种天花板就很难取得进一步的进展，也很难衡量。为了克服这一问题，该项目采用了交叉版本的方法，利用古典音乐的数据集，其中包括每部音乐作品的几种形式(配乐和音频)、几种表演(解释和编排)和几种注释(多名专家)。这样的数据集允许在版本之间传输注释，并通过测试沿着不同维度的概括，例如对作品的其他版本、作曲家的其他作品或来自历史时期的其他作曲家的概括，系统地评估深度学习方法的稳健性。作为一个主要的概念贡献，我们应用并进一步发展了这种交叉版本策略，利用它们更好地理解分析方法，并使用合适的训练和融合策略来改进这些方法。基于这种跨版本的方法，我们解决了音乐数据的具体挑战，目标是寻找对计算音乐学特别有用的分析方法，并在更广泛的数字人文领域朝着新的方法论策略迈进。