Autonomous and reactive objects for sound and light generation

这个项目的目标是探索设计和开发具体化的、反应性的、自主的代理，这些代理可以创建声音、音乐和光，以对彼此及其环境做出反应，其中包括其他代理和人。有形的互动融合了艺术、设计和技术，创造了感官装置，其中人的参与是关键。他们的作品利用了一些最基本的本能，在共享的空间里玩耍和探索。他们为公司、音乐团体、城市和博物馆制作了品牌互动体验。拟议的项目以他们开发的现有技术及其局限性为依据。这项工作的新贡献将在三个领域：自主性，新兴的创造力，以及声音/音乐的产生。目前，有形交互部署的系统主要集中在视觉上，因此他们正寻求扩展到声音和音乐生成领域。由于需要有线或无线通信，并且难以对多个组件进行编程，因此很难部署需要协调的多个组件的安装。我们将开发原型立方体，其中包含一台能够产生光图案和声音的小尺寸嵌入式计算机。这些简单的具体化代理还将能够通过麦克风和摄像头感知他们的环境包括其他代理和人。我们将使用多代理方法进行协调，其中每个代理独立于其他代理操作，但由其他代理通知，而不进行任何中央协调。我们将展示它们如何在声音、音乐和光方面的互动中产生有趣的行为。Tzanetakis博士的研究小组在计算机音乐、音乐表达的新接口和音乐信息检索方面拥有强大的国际影响力，所有这些都是将为该项目贡献想法、概念和算法的研究领域。两名在嵌入式系统/信号处理和手势识别/音乐生成方面具有免费经验的博士生将通过该项目获得资助，并与有形互动的设计师和工程师合作。

The goal of this project is to explore the design and development of embodied, reactive, autonomous agents that create sound, music, and light in reaction to each other and their environment which includes other agents and people. Tangible Interaction blends art, design, and technology to create sensory installation where people participation is key. Their work taps into some of the most basic instincts, to play and explore in a shared space. They have produced branded interactive experiences for companies, music groups, cities, and museums. The proposed project is informed by existing technologies they have developed and their limitations. The novel contributions of this work will be in three areas: autonomy, emergent creativity, and sound/music generation. Currently the systems that Tangible Interaction deploys are mostly focused on visuals so they are looking to expand into sound and music generation. Installations with multiple components that require coordination are difficult to deploy due to the need for wired or wireless communication and the difficulty of programming multiple components. We will develop prototype cubes that contain a small factor embedded computer that are capable of generating light patterns and sound. These simple embodied agents will also be able to sense through microphones and cameras their environment includes other agents and people. We will use a multi-agent approach for coordination where each agent operates independently of the other agents but is informed by other agents without any central coordination. We will show how interesting behaviour can emerge from their interactions both in terms of sound and music as well as light. The research group of Dr. Tzanetakis has a strong international presence in Computer Music, New Interfaces of Music Expression and Music Information Retrieval all of which are research areas that will contribute ideas, concepts, and algorithms to the project. Two PhD students with complimentary experience in embedded systems/signal processing and gesture recognition/music generation will be funded through the project and work with the designers and engineers at Tangible Interactive.