Advancing Computational Musicology: Semi-supervised and unsupervised segmentation and annotation of musical collections (ACMus)

在过去的几十年里，信息通信技术的快速发展已经转化为创造和管理数字文化遗产资源的大量研究和开发工作。虽然数字文化遗产时代使文化内容的可及性更广泛，但它也带来了挑战，包括信息的长期获取、可持续性和数据量的快速增长。特别是对于音乐遗产，主要的挑战在于需要能够结合音乐元素的语义检索技术，如节奏、和声、旋律、音乐织体和音色。这种自动检索技术将为音乐学研究开辟新的范式，使音乐结构和相似性的大规模分析，节奏，旋律和和声模式的识别，以及音乐传统的时间演变分析成为可能。有了这些技术，将为加强现有音乐学研究提供新的机会，从而提高效率，并为大规模数据操作和可视化提供新的可能性，这是迄今为止不可能的。本项目的目标是开发半监督和无监督语义音乐检索方法，用于音乐收藏的自动标注，重点研究四个方面：音乐、语音和歌声识别、乐器合奏识别、节拍识别和音阶检测。该工作方案结合并适应了半监督和无监督技术，用于音乐结构标注、声学场景分割、环境音频标记、调音和语调分析、乐器分类和参数化、音乐/语音识别和歌唱声音检测。该项目有望取得以下研究成果：•高效可靠的机器学习方法，用于音乐数据的自动分割和分类——关于音乐合奏、节拍、音阶和音乐/语音识别——允许以最小的人为干预处理大型集合。•基于该项目中开发的方法并由专家音乐学家验证的新颖而强大的音乐学分析工作流程。由于这项研究的多学科性质，项目团队将由信号处理，机器学习，音乐信息检索MIR，音乐学和音乐领域的研究人员组成。它将作为两个德国机构，弗劳恩霍夫数字媒体技术研究所IDMT和伊尔梅瑙技术研究所，以及两所哥伦比亚大学，安蒂奥基亚大学和玻利瓦尔尼亚大学之间的双边项目进行。

The rapid advancement of ICT technologies in the last decades has translated into numerous research and development efforts for creation and management of digital cultural heritage resources. While the era of digital cultural heritage has resulted in wider accessibility of cultural content, it has also come with its own challenges, which include long-term access to information, sustainability, and rapid growth of data volume. Particularly for musical heritage, the main challenge lays in the need for semantic retrieval techniques capable of incorporating musical elements such us rhythm, harmony, melody, musical texture, and timbre. Such automatic retrieval techniques will open new paradigms for musicological research, enabling large-scale analysis of musical structure and similarity, rhythmic, melodic and harmonic pattern recognition, and analysis of time evolution of music traditions. With these techniques, new opportunities for enhancing existing musicological research will be provided, leading to increased efficiency, and to new possibilities for large-scale data manipulation and visualization not possible to date. The goal of this project is the development of semi-supervised and unsupervised semantic music retrieval methods for the automatic annotation of musical collections, focusing on four aspects: music, speech and singing voice discrimination, musical instrument ensemble recognition, musical meter recognition, and musical scale detection. The work program combines and adapts semi-supervised and unsupervised techniques for purposes of music structure annotation, acoustic scene segmentation, and environmental audio tagging, previous work on tuning and intonation analysis, musical instrument classification and parametrization, music/speech discrimination, and singing voice detection. The following research outcomes are expected from this project:• Efficient and reliable machine learning methods for automatic segmentation and classification of musical data – with respect to musical ensemble, meter, scale, and music/speech discrimination - that allow processing of large collections with minimal human intervention.• Novel and powerful workflows for musicological analysis based on the methodologies developed in this project and validated by expert musicologists.Due to the multidisciplinary nature of this research, the project team will be composed of researchers in the fields of signal processing, machine learning, music information retrieval MIR, musicology, and music. It will be conducted as a bilateral project between two German institutions, Fraunhofer Institute for Digital Media Technology IDMT and Technische Universität Ilmenau, and two Colombian universities, Universidad de Antioquia and Universidad Pontificia Bolivariana.