Computational Analysis of Georgian Vocal Music and Beyond

在项目的第一阶段（初始提案），我们的主要目标是通过使用音频信号处理和音乐信息检索（MIR）的计算方法，推进以传统格鲁吉亚声乐为重点的民族音乐学研究。通过开发应用于具体音乐场景的新型计算工具，我们探索了计算机辅助方法在人文学科中可重复和语料库驱动研究的潜力。此外，通过系统地处理和注释独特的现场录音收藏，我们为保存和传播丰富的格鲁吉亚音乐遗产做出了贡献。在项目的第二阶段（更新建议），我们拓宽了视野，为自己设定了新的目标。首先，我们将通过结合传统的基于模型的方法和最近的数据驱动方法，系统地扩展和改进用于分析声乐的计算工具。特别是，我们希望在众所周知的困难的MIR任务中取得实质性进展，例如估计多个基本频率以及分析复调歌唱中的谐波和旋律语调方面。为了探索我们方法的可扩展性和适用性，我们超越了传统的格鲁吉亚声乐，并考虑了其他录制歌唱的语料库，包括西方合唱音乐，儿童歌曲和来自不同音乐文化的传统音乐。该项目第二阶段的另一个基本目标是探索新型接触式麦克风的潜力，克服以前使用的耳机和喉部麦克风的一些限制。我们计划使用传感器来最大限度地减少外部噪音，同时在几赫兹到2200赫兹之间的频率范围内对人体振动提供高灵敏度。包括由喉部引起的振动的基本频率（以及几个泛音），这个广泛的频率范围使语音和唱歌以及低至心跳的身体振动的分析成为可能。这种新技术将为最近基于深度学习的MIR技术所需的高质量训练数据的生成奠定基础，并为研究歌手在唱歌过程中如何同步一些身体功能（例如心跳变异性、呼吸）开辟新的途径。

In the project's first phase (initial proposal), our main objective was to advance ethnomusicological research focusing on traditional Georgian vocal music by employing computational methods from audio signal processing and music information retrieval (MIR). By developing novel computational tools applied to a concrete music scenario, we explored the potential of computer-assisted methods for reproducible and corpus-driven research within the humanities. Furthermore, by systematically processing and annotating unique collections of field recordings, we contributed to the preservation and dissemination of the rich Georgian musical heritage. In the second phase of the project (renewal proposal), we broaden our perspective and set ourselves new goals. First, we will systematically expand and improve our computational tools for analyzing vocal music by combining traditional model-based and recent data-driven approaches. In particular, we want to achieve substantial progress in notoriously difficult MIR tasks such as estimating multiple fundamental frequencies and analyzing harmonic and melodic intonation aspects in polyphonic singing. To explore the scalability and applicability of our methods, we go beyond traditional Georgian vocal music and consider other corpora of recorded singing, including Western choral music, children's songs, and traditional music from different musical cultures. Another fundamental goal for the project's second phase is to explore the potential of novel contact microphones that overcome some limitations of the previously used headset and larynx microphones. We plan to use sensors to minimize external acoustic noise while offering high sensitivity to body vibrations in a frequency range between a few Hertz and 2200 Hertz. Comprising the fundamental frequency of the vibrations caused by the larynx (as well as several overtones), this extensive frequency range enables the analysis of speech and singing as well as of body vibrations as low as the heartbeat. Such novel technology will lay the basis for generating high-quality training data as required for recent MIR techniques based on deep learning and open new paths for investigating how singers synchronize some of their body functions (e.g., heartbeat variability, respiration) during singing.