HCC: Small: Multi Modal Music Intelligence for Robotic Musicianship

HCC：小型：机器人音乐的多模式音乐智能

• 批准号: 1017169
• 负责人: Gil Weinberg
• 金额: $ 45.67万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-08-01 至 2014-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1017169&HistoricalAwards=false
• 关键词: HCC; Small; Multi; Modal; Music

The PI's objective in this project is to advance the field of robotic musicianship by enabling multi-modal communication between human and robotic musicians, and by embedding knowledge-based musical intelligence in real-time musical interactions. To these ends the PI will seek to understand how temporal structures in music are represented and processed by humans, and he will develop a vision system that enables a robotic musician to anticipate a human musician's gestures. Modeled after human-human musical interaction, the artificial vision system will complement the musical listening system developed in the PI's prior work, thereby allowing a robot to better synchronize its playing with human improvisers. In addition, the PI will transcribe and analyze (using statistical tools such as Hidden Markov Models) a large corpus of works by the great classical composers as well as by masters of jazz; the off-line analysis, coupled with interviews, surveys, and focus groups, will advance our knowledge regarding cognitive and mechanical aspects of group play and the role of visual and physical cues in live performance, and will enable the PI to construct a long-term cultural knowledge base that will be added to the short-term real-time improvisation techniques already developed previously. Collectively, project outcomes will lead to a comprehensive model of human and artificial musicianship, which the PI will evaluate using behavioral measures (such as time differences in synchronization), subject questionnaires, and a Turing-like test to evaluate the quality of the musical interaction.Broader Impacts: This project will make fundamental contribution to our knowledge in areas such as musicianship, human-robotic interaction, computer assisted collaboration, and improvisation. It will help bring human-robot interaction to the general public through high visibility concerts that capture the interest and imagination of students who are not regularly drawn to math, the sciences, or engineering by creating novel musical collaborations between humans and machines. The project will serve as a testbed for future forms of musical interactions, bringing perceptual and algorithmic aspects of computer music into the physical world both visually and acoustically, which may inspire people to play and think about music in new ways. Ultimately, the research is expected to shed light on broader concepts such as human and artificial creativity and expression, and the feasibility for machines to create, or assist in creating, meritorious aesthetic and artistic products.

PI在该项目中的目标是通过实现人类和机器人音乐家之间的多模态通信，以及通过在实时音乐交互中嵌入基于知识的音乐智能来推进机器人音乐艺术领域。 为此，PI将试图了解音乐中的时间结构是如何被人类表示和处理的，他将开发一个视觉系统，使机器人音乐家能够预测人类音乐家的手势。 模仿人类与人类的音乐互动，人工视觉系统将补充PI先前工作中开发的音乐聆听系统，从而使机器人能够更好地与人类即兴演奏者同步。 另外，PI将转录并分析（使用统计工具，如隐马尔可夫模型）一个伟大的古典作曲家以及爵士乐大师的作品的大型语料库;离线分析，加上访谈，调查和焦点小组，将促进我们对群体游戏的认知和机械方面以及视觉和身体线索在现场表演中的作用的认识，并将使PI能够构建一个长期的文化知识库，该知识库将被添加到先前已经开发的短期实时即兴创作技术中。 总的来说，项目成果将导致人类和人工音乐的综合模型，PI将使用行为措施进行评估（如同步的时间差），受试者问卷调查，以及一个类似图灵的测试来评估音乐互动的质量。该项目将为我们在音乐、人机交互、计算机辅助协作等领域的知识做出根本性贡献，和即兴创作 它将有助于通过高能见度的音乐会将人机互动带给公众，通过在人类和机器之间创造新颖的音乐合作来吸引那些不经常被数学，科学或工程吸引的学生的兴趣和想象力。 该项目将作为未来音乐互动形式的试验平台，将计算机音乐的感知和算法方面引入物理世界的视觉和听觉，这可能会激发人们以新的方式演奏和思考音乐。 最终，这项研究有望揭示更广泛的概念，如人类和人工的创造力和表达，以及机器创造或协助创造有价值的美学和艺术产品的可行性。