Research on Signal and Information Processing for Automatic Music Analysis, Recognition and Generation

自动音乐分析、识别和生成的信号和信息处理研究

• 批准号: 17300054
• 负责人: SAGAYAMA Shigeki
• 金额: $ 10.72万
• 依托单位: The University of Tokyo
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (B)
• 财政年份: 2005
• 资助国家: 日本
• 起止时间: 2005 至 2007
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-17300054/
• 关键词: musical signal processing; musical information processing; multipitch analysis; musical sound separation; rhythm recognition; harmonic analysis; speech recognition; hand-written character recognition; 非負行列分解; 調波成分分析; 多重ピッチ推定; 音楽スペクトル解析; 手書き漢字・

The objective of this research was to newly explore and establish techniques for signal and information processing of polyphonic music signals and objects with polyphonic structure and/or parallel temporal structure via approaches based on probabilistic models, aiming to deal mainly with music signals and information containing polyphony (e.g. chords) or simultaneity (e.g. accompaniment). Typical applications include automatic music transcription, automatic arrangement, music information retrieval, and music modification. Concerning multipitch analysis based on the harmonic-temporal-structured model, we established the HTC (Harmonic-Temporal Clustering) method, which estimates multiple pitches by estimating the parameters of the acoustic object model comprised of a mixture of Gaussian distributions on the time-frequency plane with harmonic structure and temporal continuity. Based on this method, we developed a technique for analyzing polyphonic signals and converting them into MIDI dat … More a. As for rhythm and tempo estimation, we developed an automatic transcription technique which reconstructs the underlying score from MIDI information through recognition of rhythm and estimation of tempo as a latent variable, based on a HMM (Hidden Markov Model). We also developed an automatic accompaniment system which plays an accompaniment following the user playing one part of polyphony with changing tempo, making mistakes, and jumping to arbitrary points in the music. Concerning separation of multiple signals with distinct periods in the time domain, we developed a method for solving a single-channel source separation problem which aims for separation of signals with distinct fundamental periods from their mixture through the auxiliary function method, an extension of the EM algorithm. We also started research on multipitch analysis based on nonnegative matrix factorization, and developed a technique for estimating timbre vectors and note activity intervals by solving the minimization problem of an error between observation and decomposition in such a way that the result is as sparse as possible, within the framework of factorization of the observed spectrogram matrix into a product of a matrix containing as few basis vectors as possible and a note activity matrix. As for computational harmony theory, we made attempt to arrange the harmony theory taught in music schools, a basic compositional theory, so that computers can handle it, based on HMM and stochastic context free grammar. This laid the groundwork for automatic harmonic analysis, automatic harmonization of melodies, automatic composition based on harmonics, etc. This research is characterized as applying the methodology of speech recognition to the music information processing area, as well as applying the developed methods to speech recognition and hand-written character recognition. Less

本研究的目的是通过基于概率模型的方法，探索和建立具有复调结构和/或平行时间结构的复调音乐信号和对象的信号和信息处理技术，旨在主要处理音乐信号和包含复调(如和弦)或同时(如伴奏)的信息。典型的应用包括自动音乐转录、自动编曲、音乐信息检索和音乐修改。对于基于调和-时间结构模型的多基音分析，我们建立了HTC(Harmonic-Time Cluping)方法，该方法通过估计具有调和结构和时间连续性的时-频平面上的混合高斯分布的声学目标模型的参数来估计多个基音。基于这种方法，我们开发了一种分析复调信号并将其转换为MIDI数据…的技术对于节奏和节奏的估计，我们开发了一种自动转录技术，该技术基于隐马尔可夫模型(HMM)，通过识别节奏和估计节奏作为潜在变量，从MIDI信息中重建潜在的乐谱。我们还开发了一个自动伴奏系统，它可以根据用户演奏的复调部分随节奏变化、出错和跳到音乐中的任意点进行伴奏。针对多个不同周期的信号在时间域中的分离问题，本文提出了一种单通道源分离问题的解决方法，该方法的目的是通过辅助函数法从混合信号中分离出具有不同基波周期的信号，辅助函数法是EM算法的扩展。我们还开始了基于非负矩阵分解的多基音分析的研究，并开发了一种技术，通过在将观察到的谱图矩阵因式分解为包含尽可能少的基向量的矩阵和音符活动矩阵的乘积的框架内，以尽可能稀疏的方式解决观察和分解之间的误差最小化问题，来估计音色向量和音符活动间隔。在计算和声理论方面，我们尝试以隐马尔可夫模型和随机上下文无关文法为基础，对音乐学校教授的和声理论这一基本的作曲理论进行整理，使计算机能够处理。这为自动和声分析、旋律自动调和、基于和声的自动作曲等奠定了基础。本研究的特点是将语音识别方法应用于音乐信息处理领域，并将所开发的方法应用于语音识别和手写字符识别。较少

项目成果

Music and speech signal processing using harmonic-temporal clustering

使用谐波时间聚类进行音乐和语音信号处理

• 发表时间: 2008
• 期刊: Proc. of Acoustics' 08
• 作者: Jonathan Le Roux;Hirokazu Kameoka;Nobutaka Ono;Alain de Cheveigne;Shigeki Sagayama
• 通讯作者: Shigeki Sagayama

音楽情報解析方法及び装置

音乐信息分析方法及装置

• 发表时间: 2006

音楽音響信号の調認識手および調推定装置

音乐音频信号的调性识别及调性估计装置

• 发表时间: 2005

手書き数式の認識装置及び認識方法

手写数学公式识别装置及识别方法

• 发表时间: 2005

対数圧伸符号化イヒされた音声信号のロスレス符号化のための線形予測分析

对数压扩音频信号无损编码的线性预测分析

• 发表时间: 2008
• 期刊: 日本音響学会春季研究発表会講演論文集 1
• 作者: 堀 豊;守谷 健弘;原田 登;鎌本 優;小野 順貴;嵯峨山 茂樹
• 通讯作者: 嵯峨山 茂樹