HRI: The Robotic Musician - Facilitating Novel Musical Experiences and Outcomes through Human Robot Interaction

该项目的目标是开发一位机器人音乐家，它将与人类创造有意义和鼓舞人心的音乐互动，导致新的音乐体验和结果。该机器人将结合音乐感知、互动和即兴表演的计算建模，以及以物理和视觉方式产生旋律和和声响应的能力。PI的基本假设是，人类和基于计算机的玩家之间的实时协作可以利用他们独特的优势组合来制作新的和引人注目的音乐。因此，该项目旨在将人类的创造力、情感和审美能力与计算机的算法计算能力结合起来。PI认为，感知和即兴的机器人将最好地促进这种互动，它将计算机带入物理世界，无论是听觉上还是视觉上。与基于计算机和扬声器的交互式音乐系统不同，物理拟人机器人将与人类进行熟悉的、丰富的听觉和视觉互动。为了与人类建立直观和鼓舞人心的社交协作，机器人将基于人类感知的计算模型分析现场音乐，并将生成人类不可能做出的算法响应。在PI之前主要专注于节奏的感知机器人鼓手工作的基础上，这个项目将开发一个可以听、分析和演奏旋律和和声音乐的机器人。它将使用四臂机制演奏木琴(一种基于旋律木槌的乐器)，并根据一套音乐感知的认知模型，如旋律吸引力、张力和相似性，从现场输入推断音乐含义。基于这一分析，机器人将产生由数学构造(如分形学、元胞自动机和遗传算法)提供信息的音乐响应。将为机器人开发的交互方案将包括同步和顺序操作，并将解决节拍跟踪和风格适应等方面的问题。项目成果将包括对音乐感知和认知、人机交互、计算机辅助协作和即兴创作的基本贡献。更广泛的影响：该项目将以音乐的魅力为基础，通过讲习班和知名度较高的音乐会，帮助公众关注HRI的激动人心的潜力，特别是将激发那些不经常对音乐、数学、工程和科学感兴趣的学生的兴趣和想象力。在HRI研究界内，与声音感知以及人类表达、情感和美学与机器人分析和机械能力相结合的项目成果将帮助研究人员在各种非音乐领域发展富有成效的人-机器人合作。PI预计，该项目还将为他在佐治亚理工学院发起的音乐技术研究中心发挥关键作用。

The objective of this project is to develop a robotic musician that will create meaningful and inspiring musical interactions with humans, leading to novel musical experiences and outcomes. The robot will combine computational modeling of music perception, interaction, and improvisation, with the capacity to produce melodic and harmonic acoustic responses in physical and visual manners. The PI's underlying hypothesis is that real-time collaboration between humans and computer-based players can capitalize on the combination of their unique strengths to produce new and compelling music. The project, therefore, aims to combine human creativity, emotion, and aesthetic judgment with the algorithmic computational capability of computers. The PI believes that a perceptual and improvisatory robot will best facilitate such interactions by bringing the computer into the physical world both acoustically and visually. Unlike computer- and speaker-based interactive music systems, a physical anthropomorphic robot will create familiar, acoustically rich, and visual interactions with humans. In order to create intuitive as well as inspiring social collaboration with people, the robot will analyze live music based on computational models of human perception, and will generate algorithmic responses that are humanly impossible. Building on and extending the PI's previous work on a perceptual robotic drummer that was primarily focused on rhythm, this project will develop a robot that can listen to, analyze, and play melodic and harmonic music as well. It will use a four-arm mechanism to play the marimba (a melodic mallet-based instrument), and infer musical meaning from live input based on a set of cognitive models of musical percepts such as melodic attraction, tension, and similarity. Based on this analysis, the robot will generate musical responses informed by mathematical constructs such as fractals, cellular automata and genetic algorithms. The interaction schemes to be developed for the robot will include synchronous and sequential operations, and will address aspects such as beat tracking and style adaptation. Project outcomes will include fundamental contributions to music perception and cognition, human-robotic interaction, computer assisted collaboration, and improvisation. Broader Impacts: Building on the engaging power of music, this project will help bring the exciting potential of HRI to the attention of the general public through workshops and high visibility concerts, which will be designed in particular to capture the interest and imagination of students who are not regularly drawn to music, mathematics, engineering, and the sciences. Within the HRI research community, project outcomes relating to sound perception, and to the integration of human expression, emotion, and aesthetics with robotic analytical and mechanical capabilities, will help researchers develop productive human-robot collaborations in a variety of non-musical domains. The PI expects the project will also play a pivotal role for the research center in music technology that he has initiated at Georgia Tech.