New machine learning methods for music: learning at multiple timescales

针对音乐的机器学习算法的设计已经取得了很大的进展。然而，大多数方法忽略了全局时间结构，将音频文件切成五秒或更短的片段，并单独处理每个片段。由于忽略了长时间尺度结构，这些算法无法学习音乐意义的重要高级指标，如旋律和和弦进度。这一差距定义了这项研究提案中涉及的核心挑战：我们如何在多个时间尺度上处理音乐，以便同时从音频中学习重要的局部和全球结构？这项研究的目标是开发新的机器学习算法，用于音乐推荐、仿人音乐表演和自适应上下文感知视频游戏音乐的生成。我们的目标是设计层次化的多时间尺度算法，将多个层次的证据结合起来。我们认为，对于试图学习音乐是如何在时间上组织的机器学习模型来说，音乐节拍是一个特别有信息量的优先事项。能够以类似人类的方式跟踪节拍的模型知道音乐信号中何时会发生重要事件，从而避免了对音乐中所有可能的时间滞后之间的相关性进行指数级长的搜索。音乐相关研究解决了与音乐信息检索和音乐表演相关的几个困难和相关的任务。具体任务包括预测两个音频序列之间的相似性，预测音频的艺术家、流派或风格，以逼真的方式表演音乐，生成风格合适并适合特定环境(如惊悚或平静)的音乐(如视频游戏)。除了贡献与行业相关的应用程序外，这里进行的基础研究还将推动时间序列数据的机器学习算法的最先进水平，并将进一步加深我们对音乐家如何创作音乐以及听众如何感知音乐的理解。

Great advances have been made in the design of machine learning algorithms for music. However, most approaches ignore global temporal structure, chopping an audio file into segments of five seconds or less and treating each segment individually. By ignoring long-timescale structure, these algorithms are unable to learn important high-level indicators of musical meaning such as melody and chord progressions. This gap defines the core challenge addressed in this research proposal: how can we approach music at multiple timescales so as to in parallel learn important local and global structure from audio? The objective of this research program is to develop new machine learning algorithms for music recommendation, human-like music performance and adaptive context-aware generation of music for video games. Our goal is to design hierarchical multi-timescale algorithms that incorporate evidence at many levels. We believe that musical meter is a particularly informative prior for a machine learning model trying to learn how music is organized in time. A model that can track meter in a human-like manner knows when important events will occur in the music signal, thus avoiding an exponentially-long search for correlation among all possible time lags in the music. The music related research addresses several difficult and relevant tasks related to music information retrieval and to music performance. Specific tasks include predicting the similarity between two audio sequences, predicting the artist, genre or style of audio, performing music in a human-realistic fashion, generating music (e.g. for video games) that is stylistically appropriate and that fits a particular context (e.g. scary or calm). In addition to contributing industry-relevant applications, the fundamental research carried out here will advance the state-of-the-art in machine learning algorithms for time series data and will further our understanding of how music is produced by musicians and perceived by listeners.