Research on Signal and Information Processing for Automatic Music Analysis, Recognition and Generation

本研究的目的是通过基于概率模型的方法，探索和建立具有复调结构和/或平行时间结构的复调音乐信号和对象的信号和信息处理技术，旨在主要处理音乐信号和包含复调(如和弦)或同时(如伴奏)的信息。典型的应用包括自动音乐转录、自动编曲、音乐信息检索和音乐修改。对于基于调和-时间结构模型的多基音分析，我们建立了HTC(Harmonic-Time Cluping)方法，该方法通过估计具有调和结构和时间连续性的时-频平面上的混合高斯分布的声学目标模型的参数来估计多个基音。基于这种方法，我们开发了一种分析复调信号并将其转换为MIDI数据…的技术对于节奏和节奏的估计，我们开发了一种自动转录技术，该技术基于隐马尔可夫模型(HMM)，通过识别节奏和估计节奏作为潜在变量，从MIDI信息中重建潜在的乐谱。我们还开发了一个自动伴奏系统，它可以根据用户演奏的复调部分随节奏变化、出错和跳到音乐中的任意点进行伴奏。针对多个不同周期的信号在时间域中的分离问题，本文提出了一种单通道源分离问题的解决方法，该方法的目的是通过辅助函数法从混合信号中分离出具有不同基波周期的信号，辅助函数法是EM算法的扩展。我们还开始了基于非负矩阵分解的多基音分析的研究，并开发了一种技术，通过在将观察到的谱图矩阵因式分解为包含尽可能少的基向量的矩阵和音符活动矩阵的乘积的框架内，以尽可能稀疏的方式解决观察和分解之间的误差最小化问题，来估计音色向量和音符活动间隔。在计算和声理论方面，我们尝试以隐马尔可夫模型和随机上下文无关文法为基础，对音乐学校教授的和声理论这一基本的作曲理论进行整理，使计算机能够处理。这为自动和声分析、旋律自动调和、基于和声的自动作曲等奠定了基础。本研究的特点是将语音识别方法应用于音乐信息处理领域，并将所开发的方法应用于语音识别和手写字符识别。较少

The objective of this research was to newly explore and establish techniques for signal and information processing of polyphonic music signals and objects with polyphonic structure and/or parallel temporal structure via approaches based on probabilistic models, aiming to deal mainly with music signals and information containing polyphony (e.g. chords) or simultaneity (e.g. accompaniment). Typical applications include automatic music transcription, automatic arrangement, music information retrieval, and music modification. Concerning multipitch analysis based on the harmonic-temporal-structured model, we established the HTC (Harmonic-Temporal Clustering) method, which estimates multiple pitches by estimating the parameters of the acoustic object model comprised of a mixture of Gaussian distributions on the time-frequency plane with harmonic structure and temporal continuity. Based on this method, we developed a technique for analyzing polyphonic signals and converting them into MIDI dat … More a. As for rhythm and tempo estimation, we developed an automatic transcription technique which reconstructs the underlying score from MIDI information through recognition of rhythm and estimation of tempo as a latent variable, based on a HMM (Hidden Markov Model). We also developed an automatic accompaniment system which plays an accompaniment following the user playing one part of polyphony with changing tempo, making mistakes, and jumping to arbitrary points in the music. Concerning separation of multiple signals with distinct periods in the time domain, we developed a method for solving a single-channel source separation problem which aims for separation of signals with distinct fundamental periods from their mixture through the auxiliary function method, an extension of the EM algorithm. We also started research on multipitch analysis based on nonnegative matrix factorization, and developed a technique for estimating timbre vectors and note activity intervals by solving the minimization problem of an error between observation and decomposition in such a way that the result is as sparse as possible, within the framework of factorization of the observed spectrogram matrix into a product of a matrix containing as few basis vectors as possible and a note activity matrix. As for computational harmony theory, we made attempt to arrange the harmony theory taught in music schools, a basic compositional theory, so that computers can handle it, based on HMM and stochastic context free grammar. This laid the groundwork for automatic harmonic analysis, automatic harmonization of melodies, automatic composition based on harmonics, etc. This research is characterized as applying the methodology of speech recognition to the music information processing area, as well as applying the developed methods to speech recognition and hand-written character recognition. Less