Computational Analysis of Georgian Vocal Music and Beyond

在该项目的第一阶段(初步提案)，我们的主要目标是通过使用音频信号处理和音乐信息检索(MIR)的计算方法，促进以格鲁吉亚传统声乐为重点的民族音乐研究。通过开发应用于具体音乐场景的新的计算工具，我们探索了计算机辅助方法在人文科学中可重现和语料库驱动的研究的潜力。此外，通过系统地处理和注释独特的实地录音收藏，我们为保存和传播丰富的格鲁吉亚音乐遗产作出了贡献。在项目的第二阶段(续签提案)，我们开阔了视野，为自己设定了新的目标。首先，我们将结合传统的基于模型的方法和最新的数据驱动方法，系统地扩展和改进我们用于分析声乐的计算工具。特别是，我们希望在众所周知的困难的MIR任务上取得实质性进展，例如估计多个基频以及分析复调演唱中的和声和旋律语调方面。为了探索我们方法的可扩展性和适用性，我们超越了传统的格鲁吉亚声乐，并考虑了其他录制的演唱语料库，包括西方合唱音乐、儿童歌曲和来自不同音乐文化的传统音乐。该项目第二阶段的另一个基本目标是探索新型接触式麦克风的潜力，以克服以前使用的耳机和喉部麦克风的一些限制。我们计划使用传感器来最大限度地减少外部噪音，同时对几赫兹到2200赫兹之间的频率范围内的身体振动提供高灵敏度。由喉咙引起的振动的基本频率(以及几个泛音)组成，这种广泛的频率范围使语音和歌唱以及身体振动的分析能够低至心跳。这种新技术将为生成基于深度学习的最新MIR技术所需的高质量训练数据奠定基础，并为研究歌手在歌唱过程中如何同步他们的一些身体功能(如心跳变异性、呼吸)开辟了新的途径。

In the project's first phase (initial proposal), our main objective was to advance ethnomusicological research focusing on traditional Georgian vocal music by employing computational methods from audio signal processing and music information retrieval (MIR). By developing novel computational tools applied to a concrete music scenario, we explored the potential of computer-assisted methods for reproducible and corpus-driven research within the humanities. Furthermore, by systematically processing and annotating unique collections of field recordings, we contributed to the preservation and dissemination of the rich Georgian musical heritage. In the second phase of the project (renewal proposal), we broaden our perspective and set ourselves new goals. First, we will systematically expand and improve our computational tools for analyzing vocal music by combining traditional model-based and recent data-driven approaches. In particular, we want to achieve substantial progress in notoriously difficult MIR tasks such as estimating multiple fundamental frequencies and analyzing harmonic and melodic intonation aspects in polyphonic singing. To explore the scalability and applicability of our methods, we go beyond traditional Georgian vocal music and consider other corpora of recorded singing, including Western choral music, children's songs, and traditional music from different musical cultures. Another fundamental goal for the project's second phase is to explore the potential of novel contact microphones that overcome some limitations of the previously used headset and larynx microphones. We plan to use sensors to minimize external acoustic noise while offering high sensitivity to body vibrations in a frequency range between a few Hertz and 2200 Hertz. Comprising the fundamental frequency of the vibrations caused by the larynx (as well as several overtones), this extensive frequency range enables the analysis of speech and singing as well as of body vibrations as low as the heartbeat. Such novel technology will lay the basis for generating high-quality training data as required for recent MIR techniques based on deep learning and open new paths for investigating how singers synchronize some of their body functions (e.g., heartbeat variability, respiration) during singing.