HRI: The Robotic Musician - Facilitating Novel Musical Experiences and Outcomes through Human Robot Interaction

这个项目的目标是开发一个机器人音乐家，它将与人类创造有意义和鼓舞人心的音乐互动，从而带来新的音乐体验和结果。该机器人将结合音乐感知、互动和即兴创作的计算建模，并具有以物理和视觉方式产生旋律和和声响应的能力。PI的基本假设是，人类和电脑玩家之间的实时合作可以利用他们独特的优势组合，创造出新的、引人入胜的音乐。因此，该项目旨在将人类的创造力、情感和审美判断与计算机的算法计算能力结合起来。PI认为，一个具有感知能力和即兴能力的机器人将通过在声音和视觉上将计算机带入物理世界，从而最好地促进这种互动。与基于计算机和扬声器的交互式音乐系统不同，物理拟人化机器人将与人类进行熟悉的、声音丰富的和视觉上的互动。为了与人建立直观和鼓舞人心的社会协作，机器人将基于人类感知的计算模型分析现场音乐，并产生人类无法实现的算法响应。在PI之前主要关注节奏的感知机器人鼓手工作的基础上，该项目将开发一种可以听、分析和演奏旋律和和声音乐的机器人。它将使用一个四臂机制来演奏马林巴琴（一种基于旋律木槌的乐器），并根据一组音乐感知的认知模型，如旋律吸引力、张力和相似性，从现场输入推断音乐意义。基于这种分析，机器人将根据分形、细胞自动机和遗传算法等数学结构生成音乐响应。为机器人开发的交互方案将包括同步和顺序操作，并将解决诸如节拍跟踪和风格适应等方面的问题。项目成果将包括对音乐感知和认知、人机交互、计算机辅助协作和即兴创作的基本贡献。更广泛的影响：以音乐的吸引力为基础，该项目将通过研讨会和高知名度的音乐会，帮助将HRI的令人兴奋的潜力吸引到公众的注意，这些音乐会将特别设计，以吸引那些不经常被音乐，数学，工程和科学吸引的学生的兴趣和想象力。在HRI研究界，与声音感知相关的项目成果，以及与机器人分析和机械能力相结合的人类表达、情感和美学，将有助于研究人员在各种非音乐领域开展富有成效的人机合作。PI希望这个项目也能在他在佐治亚理工学院发起的音乐技术研究中心发挥关键作用。

The objective of this project is to develop a robotic musician that will create meaningful and inspiring musical interactions with humans, leading to novel musical experiences and outcomes. The robot will combine computational modeling of music perception, interaction, and improvisation, with the capacity to produce melodic and harmonic acoustic responses in physical and visual manners. The PI's underlying hypothesis is that real-time collaboration between humans and computer-based players can capitalize on the combination of their unique strengths to produce new and compelling music. The project, therefore, aims to combine human creativity, emotion, and aesthetic judgment with the algorithmic computational capability of computers. The PI believes that a perceptual and improvisatory robot will best facilitate such interactions by bringing the computer into the physical world both acoustically and visually. Unlike computer- and speaker-based interactive music systems, a physical anthropomorphic robot will create familiar, acoustically rich, and visual interactions with humans. In order to create intuitive as well as inspiring social collaboration with people, the robot will analyze live music based on computational models of human perception, and will generate algorithmic responses that are humanly impossible. Building on and extending the PI's previous work on a perceptual robotic drummer that was primarily focused on rhythm, this project will develop a robot that can listen to, analyze, and play melodic and harmonic music as well. It will use a four-arm mechanism to play the marimba (a melodic mallet-based instrument), and infer musical meaning from live input based on a set of cognitive models of musical percepts such as melodic attraction, tension, and similarity. Based on this analysis, the robot will generate musical responses informed by mathematical constructs such as fractals, cellular automata and genetic algorithms. The interaction schemes to be developed for the robot will include synchronous and sequential operations, and will address aspects such as beat tracking and style adaptation. Project outcomes will include fundamental contributions to music perception and cognition, human-robotic interaction, computer assisted collaboration, and improvisation. Broader Impacts: Building on the engaging power of music, this project will help bring the exciting potential of HRI to the attention of the general public through workshops and high visibility concerts, which will be designed in particular to capture the interest and imagination of students who are not regularly drawn to music, mathematics, engineering, and the sciences. Within the HRI research community, project outcomes relating to sound perception, and to the integration of human expression, emotion, and aesthetics with robotic analytical and mechanical capabilities, will help researchers develop productive human-robot collaborations in a variety of non-musical domains. The PI expects the project will also play a pivotal role for the research center in music technology that he has initiated at Georgia Tech.