Improvising with the dead

与死者即兴创作

• 批准号: RGPIN-2017-06239
• 负责人: Tzanetakis, George
• 金额: $ 1.68万
• 依托单位: University of Victoria
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=749935
• 关键词: Improvising; dead

The end goal of the proposed research is rather simple to state. I would like to develop technologies to support the currently impossible experience of improvising music together with one or more musicians that are dead. This experience should be as close to actual music playing as possible. Although this goal is simple to state it is challenging to achieve and requires important advances in different research topics from several disciplines.Improvisation is present in all music cultures and can be identified and appreciated by most music listeners even in cultures they are not familiar with. The processes that govern improvisation are not well understood and also depend on the specific music culture studied. They are also intrinsically tied to the physical actions of playing instruments. For example to understand and appreciate a solo by B.B King it is not sufficient to look at what notes are played but one also needs to consider the visceral connection between his body and the guitar. I believe that we are at a time when the experience of improvising with a dead musician is moving from science fiction to the realm of possibility as there are several recent key technological developments that can be leveraged for this purpose. Imagine being able to analyze the audio recordings of a legendary jazz musician, for example the incredible saxophone player Charlie Parker, in order to create a model of how he played and improvised. The model could be used to create a computer controlled virtual Charlie Parker that a jazz student could play with and learn from. In order to approximate the playing experience as accurately as possible augmented or virtual reality (AR/VR) devices would be used while playing an actual acoustic instrument.There is resurgence of AR/VR with many key industrial players involved. In the new interfaces for musical expression community there is extensive experience building effective cyber-physical systems for music. Music information retrieval (MIR) techniques analyze millions of songs to extract content information to automatically recommend music. MIR techniques can also be used to extract information about what is being played in real time. My group has been active in these research areas and will continue to do so with this new focus. Students working on these project will obtain experience in audio signal processing, machine learning, augmented and virtual reality, human- computer interaction (HCI), programming languages and artificial intelligence (AI). As the virtual and physical words blend, new ways of interaction will be developed. The proposed work can also lead to advances in AI and the tools used to build complicated, real-time, AI systems and shape how humans and such systems interact in the future. Finally, the advances envisioned in this proposal have the potential to radically transform how we create, distribute and perceive music (and multimedia in general) content in the future.

拟议研究的最终目标很容易说明。我想开发技术来支持与一个或多个已故音乐家一起即兴创作音乐这一目前不可能实现的体验。这种体验应该尽可能接近实际的音乐播放。虽然这个目标很简单，但要实现它是具有挑战性的，并且需要在几个学科的不同研究主题中取得重要进展。即兴创作存在于所有的音乐文化中，即使在他们不熟悉的文化中，大多数音乐听众也能识别和欣赏即兴创作。控制即兴创作的过程还没有被很好地理解，也取决于所研究的特定音乐文化。它们也与演奏乐器的身体动作有着内在的联系。例如，要理解和欣赏B.B King的独奏，仅仅看音符是不够的，还需要考虑他的身体和吉他之间的内在联系。我相信，我们正处在这样一个时代，与已故音乐家一起即兴创作的经历正从科幻小说变成现实，因为最近有几项关键的技术发展可以用于这一目的。想象一下，能够分析传奇爵士音乐家的录音，例如令人难以置信的萨克斯管演奏家查理·帕克，为了创建一个他如何演奏和即兴创作的模型。这个模型可以用来创建一个计算机控制的虚拟查理·帕克，爵士乐学生可以和他一起玩，并向他学习。为了尽可能准确地接近演奏体验，在演奏实际的声学乐器时将使用增强或虚拟现实（AR/VR）设备。AR/VR正在复苏，许多主要的行业参与者都参与其中。在音乐表现的新界面方面，建立有效的音乐网络物理系统有着丰富的经验。音乐信息检索（MIR）技术对数百万首歌曲进行分析，提取内容信息，自动推荐音乐。MIR技术还可用于提取有关实时播放内容的信息。我的小组一直活跃在这些研究领域，并将继续这样做，以这个新的重点。参与这些项目的学生将获得音频信号处理、机器学习、增强现实和虚拟现实、人机交互（HCI）、编程语言和人工智能（AI）方面的经验。随着虚拟语言和实体语言的融合，新的互动方式将被开发出来。拟议的工作还可以推动人工智能和用于构建复杂、实时的人工智能系统的工具的进步，并塑造人类和这些系统在未来的互动方式。最后，本提案所设想的进步有可能在未来从根本上改变我们创作、分发和感知音乐（以及一般的多媒体）内容的方式。