Research on Signal and Information Processing for Automatic Music Analysis, Recognition and Generation

本研究的目的是通过基于概率模型的方法，探索和建立复调音乐信号和具有复调结构和/或平行时间结构的对象的信号和信息处理技术，主要针对包含复调（如和弦）或同时性（如伴奏）的音乐信号和信息。典型的应用包括自动音乐转录、自动编曲、音乐信息检索和音乐修改。在基于谐波-时间结构模型的多音高分析中，我们建立了谐波-时间结构聚类（HTC）方法，该方法通过估计具有谐波结构和时间连续性的时频面上混合高斯分布的声目标模型的参数来估计多音高。在此基础上，我们开发了一种分析复调信号并将其转换为MIDI数据的技术。a.在节奏和速度估计方面，我们开发了一种自动转录技术，该技术基于隐马尔可夫模型（HMM），通过识别节奏和估计速度作为潜在变量，从MIDI信息中重建底层分数。我们还开发了一个自动伴奏系统，它可以随着用户演奏复调的一个部分，改变速度，犯错误，跳到音乐的任意点来播放伴奏。针对时域内不同周期的多信号分离问题，我们提出了一种解决单通道源分离问题的方法，该方法旨在通过辅助函数法（EM算法的扩展）将不同基本周期的信号从混合信号中分离出来。我们还开始了基于非负矩阵分解的多音高分析研究，并开发了一种估计音色矢量和音符活动间隔的技术，通过解决观测和分解之间误差的最小化问题，使结果尽可能稀疏，在将观测谱图矩阵分解为包含尽可能少的基向量的矩阵和音符活动矩阵的乘积的框架内。在计算和声理论方面，我们尝试在HMM和随机上下文无关语法的基础上，将音乐学校教授的和声理论这一基础作曲理论进行整理，使计算机能够处理。这为自动和声分析、旋律自动和声、基于和声的自动作曲等奠定了基础。本研究的特点是将语音识别的方法应用于音乐信息处理领域，并将所开发的方法应用于语音识别和手写体识别。少

The objective of this research was to newly explore and establish techniques for signal and information processing of polyphonic music signals and objects with polyphonic structure and/or parallel temporal structure via approaches based on probabilistic models, aiming to deal mainly with music signals and information containing polyphony (e.g. chords) or simultaneity (e.g. accompaniment). Typical applications include automatic music transcription, automatic arrangement, music information retrieval, and music modification. Concerning multipitch analysis based on the harmonic-temporal-structured model, we established the HTC (Harmonic-Temporal Clustering) method, which estimates multiple pitches by estimating the parameters of the acoustic object model comprised of a mixture of Gaussian distributions on the time-frequency plane with harmonic structure and temporal continuity. Based on this method, we developed a technique for analyzing polyphonic signals and converting them into MIDI dat … More a. As for rhythm and tempo estimation, we developed an automatic transcription technique which reconstructs the underlying score from MIDI information through recognition of rhythm and estimation of tempo as a latent variable, based on a HMM (Hidden Markov Model). We also developed an automatic accompaniment system which plays an accompaniment following the user playing one part of polyphony with changing tempo, making mistakes, and jumping to arbitrary points in the music. Concerning separation of multiple signals with distinct periods in the time domain, we developed a method for solving a single-channel source separation problem which aims for separation of signals with distinct fundamental periods from their mixture through the auxiliary function method, an extension of the EM algorithm. We also started research on multipitch analysis based on nonnegative matrix factorization, and developed a technique for estimating timbre vectors and note activity intervals by solving the minimization problem of an error between observation and decomposition in such a way that the result is as sparse as possible, within the framework of factorization of the observed spectrogram matrix into a product of a matrix containing as few basis vectors as possible and a note activity matrix. As for computational harmony theory, we made attempt to arrange the harmony theory taught in music schools, a basic compositional theory, so that computers can handle it, based on HMM and stochastic context free grammar. This laid the groundwork for automatic harmonic analysis, automatic harmonization of melodies, automatic composition based on harmonics, etc. This research is characterized as applying the methodology of speech recognition to the music information processing area, as well as applying the developed methods to speech recognition and hand-written character recognition. Less