Robot musical accompaniment: integrating audio and visual cues for real-time synchronization with a human flutist
Robot musical accompaniment: integrating audio and visual cues for real-time synchronization with a human flutist
复制标题
DOI：
10.1109/iros.2010.5650427
复制
发表时间：
2010-12
期刊：
影响因子：
--
通讯作者：
Angelica Lim;Takeshi Mizumoto;Louis-Kenzo Cahier;Takuma Otsuka;Toru Takahashi;Kazunori Komatani;T. Ogata
中科院分区：
文献类型：
--
作者：
Angelica Lim;Takeshi Mizumoto;Louis-Kenzo Cahier;Takuma Otsuka;Toru Takahashi;Kazunori Komatani;T. Ogata
音乐家经常有以下问题：他们有一个乐谱，需要2个或更多的球员，但他们没有人与谁练习。到目前为止，乐谱演奏机器人已经存在，但它们缺乏适应能力，无法与其他玩家的克里思变化同步。换句话说，如果人类加快游戏速度，机器人也应该加快速度。然而，计算机伴奏系统恰恰允许这种自适应能力。我们提出了第一步，使这些伴奏能力的音乐机器人。我们引入了一种新的节拍跟踪范式，使用2种类型的感觉输入（视觉和音频），使用我们自己的视觉线索识别系统和最先进的声学开始检测技术。初步实验表明，通过耦合这两种模式，机器人演奏者可以与长笛演奏者同步开始和停止演奏，并在半秒内检测克里思变化。
Musicians often have the following problem: they have a music score that requires 2 or more players, but they have no one with whom to practice. So far, score-playing music robots exist, but they lack adaptive abilities to synchronize with fellow players' tempo variations. In other words, if the human speeds up their play, the robot should also increase its speed. However, computer accompaniment systems allow exactly this kind of adaptive ability. We present a first step towards giving these accompaniment abilities to a music robot. We introduce a new paradigm of beat tracking using 2 types of sensory input - visual and audio - using our own visual cue recognition system and state-of-the-art acoustic onset detection techniques. Preliminary experiments suggest that by coupling these two modalities, a robot accompanist can start and stop a performance in synchrony with a flutist, and detect tempo changes within half a second.