Research on Signal and Information Processing for Automatic Music Analysis, Recognition and Generation

本研究的目的是通过基于概率模型的方法，新探索和建立用于具有复调结构和/或平行时间结构的复调音乐信号和对象的信号和信息处理的技术，旨在主要处理包含复调（例如和弦）或重复性（例如伴奏）的音乐信号和信息。典型的应用包括自动音乐转录、自动编排、音乐信息检索和音乐修改。针对基于谐波时间结构模型的多音调分析，提出了基于谐波时间结构模型的多音调分析方法HTC（Harmonic-Temporal Clustering），该方法通过估计时频平面上具有谐波结构和时间连续性的高斯混合分布声学对象模型的参数来估计多音调。在此基础上，我们提出了一种分析复音信号并将其转换为复音数据的方法， 关于我们 a.至于节奏和克里思估计，我们开发了一种自动转录技术，通过识别节奏和估计克里思作为潜在变量，基于HMM（隐马尔可夫模型），从数据库信息重建基础分数。我们还开发了一个自动伴奏系统，播放伴奏随着用户播放复调的一部分，改变克里思，犯错误，并跳转到音乐中的任意点。针对时域中多个不同周期信号的分离问题，提出了一种基于EM算法的辅助函数方法，解决了单通道信号源分离问题，该方法的目标是从混合信号中分离出具有不同基本周期的信号。我们还开始了基于非负矩阵分解的多音高分析的研究，并开发了一种技术，用于通过解决观察和分解之间的误差的最小化问题来估计音色向量和音符活动间隔，从而使结果尽可能稀疏，在将所观察到的频谱图矩阵分解为包含尽可能少的基向量的矩阵和注释的乘积的框架内活动矩阵至于计算和声理论，我们尝试安排音乐学校教授的和声理论，一个基本的作曲理论，使计算机可以处理它，基于HMM和随机上下文无关语法。这奠定了基础，自动谐波分析，自动协调的旋律，自动合成的基础上，谐波等。本研究的特点是应用语音识别的方法，音乐信息处理领域，以及应用开发的方法，语音识别和手写字符识别。少

The objective of this research was to newly explore and establish techniques for signal and information processing of polyphonic music signals and objects with polyphonic structure and/or parallel temporal structure via approaches based on probabilistic models, aiming to deal mainly with music signals and information containing polyphony (e.g. chords) or simultaneity (e.g. accompaniment). Typical applications include automatic music transcription, automatic arrangement, music information retrieval, and music modification. Concerning multipitch analysis based on the harmonic-temporal-structured model, we established the HTC (Harmonic-Temporal Clustering) method, which estimates multiple pitches by estimating the parameters of the acoustic object model comprised of a mixture of Gaussian distributions on the time-frequency plane with harmonic structure and temporal continuity. Based on this method, we developed a technique for analyzing polyphonic signals and converting them into MIDI dat … More a. As for rhythm and tempo estimation, we developed an automatic transcription technique which reconstructs the underlying score from MIDI information through recognition of rhythm and estimation of tempo as a latent variable, based on a HMM (Hidden Markov Model). We also developed an automatic accompaniment system which plays an accompaniment following the user playing one part of polyphony with changing tempo, making mistakes, and jumping to arbitrary points in the music. Concerning separation of multiple signals with distinct periods in the time domain, we developed a method for solving a single-channel source separation problem which aims for separation of signals with distinct fundamental periods from their mixture through the auxiliary function method, an extension of the EM algorithm. We also started research on multipitch analysis based on nonnegative matrix factorization, and developed a technique for estimating timbre vectors and note activity intervals by solving the minimization problem of an error between observation and decomposition in such a way that the result is as sparse as possible, within the framework of factorization of the observed spectrogram matrix into a product of a matrix containing as few basis vectors as possible and a note activity matrix. As for computational harmony theory, we made attempt to arrange the harmony theory taught in music schools, a basic compositional theory, so that computers can handle it, based on HMM and stochastic context free grammar. This laid the groundwork for automatic harmonic analysis, automatic harmonization of melodies, automatic composition based on harmonics, etc. This research is characterized as applying the methodology of speech recognition to the music information processing area, as well as applying the developed methods to speech recognition and hand-written character recognition. Less