Transforming Transformation: 3D Models for Interactive Sound Design

《转型：互动音响设计的3D模型》将通过探索以人为中心的创造性声音设计方法，开辟音乐技术研究的新领域。该项目将开发一种概念验证系统，使声音能够通过手的运动来操纵，就像它是一个看不见的3D物体一样。互动将通过在虚拟3D空间内的实时可视化来增强。这项研究提出了声音设计的“自然”界面的概念，考虑到已经学会的行为和天生的技能，如运动记忆、手势和空间意识。其目的是让音乐家和音效设计师在与技术合作时能够在声音中进行思考，促进从以技术为中心到以人为中心的模式的转变。作为一个试点项目，我们将专注于一种特定的音频处理技术：声音空间化，以期在以后的更大项目中扩大我们的研究成果，解决更广泛的声音设计实践。简而言之，我们的系统将探索这样一种想法，即我们可以让用户感觉到，他们通过用手拿起并移动声音，就好像直接在空间中定位声音一样，目的是为未来更详细的研究提供范例。声音设计需要操纵声音，以达到戏剧性、现实主义、音乐性或情感效果。通常，使用诸如过滤、时间拉伸、音调改变、颗粒化、衰减和平移等技术来处理音频。这些技术通过诸如“滑块”和“刻度盘”之类的离散控件向声音设计者展示，每个控件都控制底层信号处理系统中的一个参数。相比之下，人类的声音概念的特点是多态的、感性的、隐喻的、象征的和拟声的联想。这些联想很多都是跨感官的，比如：“温暖的”、“明亮的”、“遥远的”、“金属的”。此外，它们暗示了肉体，并引发了直接的准触觉操纵的可能性。空间化尤其如此，在声音空间中存在一种声音局部化的感觉，进一步唤起了有形的概念：‘移动’或‘放置’声音，甚至‘推动’声音以提供给定方向的惯性。因此，我们的研究挑战了现有的声音设计模型，并探索了想象的声音和声音结果之间的‘自然’和直接有形联系的概念。为了实现这一目标，我们将利用PI在领先的音乐学院内的研究实验室提供的巨大好处。在音乐学院的环境中，该项目为该项目提供了立即和不受限制地接触到来自各种背景的高质量音乐家的机会。这有助于将研究牢牢地植根于艺术实践，使艺术思想既可以检验现有的模型，也可以作为新模型的基础。正式的用户体验审查将在项目的里程碑阶段进行，允许我们在使用开发的系统时展示用户对其体验的默契和意见。研究方法还将要求使用该系统制作一部新的音乐作品，将在项目最后一个月的公共研讨会和音乐会上首演。这既是在“真实世界”创造性场景中进行测试的机会，也是在公共论坛上传播项目成果的一种手段。还将有一个使用正在开发的系统的项目中期的“实验性堵塞”，这将在互联网上现场直播。音乐技术研究在音响设计行业有着良好的影响记录，特别是通过程序音频生成等技术在游戏音频方面的影响。我们对以人为中心的声音设计新方法的探索将开辟一个新的研究领域，通过在创造性实践中的坚实基础，跨学科的影响将更加丰富。

'Transforming Transformation: 3D Models for Interactive Sound Design' will open up a new field of music technology research by exploring human-centred approaches to creative sound design. The project will involve the development of a proof-of-concept system, which enables sound to be manipulated through hand movements as if it were an invisible 3D object. The interaction will be enhanced by real-time visualisation within a virtual 3D space. The research surfaces the notion of a 'natural' interface for sound design taking account of already learned behaviour and innate skills such as motor memory, gesture and spatial awareness. The aim is to enable musicians and sound designers to 'think in sound' when working with technology, catalysing a shift from technology-centric to human-centric models. As a pilot project, we will concentrate on one specific audio processing technique: sound spatialisation, with a view to later amplifying our research findings in a larger project, addressing a wider range of sound design practices. In simple terms: our system will explore the idea that we can make users feel as if they are directly positioning sound in space by 'picking it up and moving it' with their hands, with the aim of serving as an exemplar for future, more detailed studies.Sound design entails the manipulation of sound for dramatic, realistic, musical, or emotional effect. Typically, audio is processed using techniques such as filtering, time-stretching, pitch shifting, granulation, attenuation and panning. These techniques are exposed to the sound designer through discrete controls such as 'sliders' and 'dials' each of which controls a single parameter in the underlying signal processing system. By contrast, human concepts of sound are characterised by polymorphous, perceptual, metaphorical, symbolic and onomatopoeic associations. Many of these associations are cross-sensory, for example: 'warm', 'bright', 'distant', 'metallic'. Furthermore, they suggest physicality and invite the possibility of direct quasi-tactile manipulation. This is especially true of spatialisation, where there exists a sense of sound localisation within an acoustic space, further evoking the idea of tangibility: 'moving' or 'placing' a sound, or even 'pushing' a sound to provide inertia in a given direction.Our research therefore challenges existing sound design models and explores the notion of a 'natural' and direct corporeal link between imagined sound and acoustic results. In order to achieve this, we will draw upon the immense benefits offered by the PI's research lab within a leading music Conservatoire. Being based within a Conservatoire environment, provides the project with immediate and unfettered access to high-quality musicians from a range of backgrounds. This serves to ground the research firmly in artistic practice, allowing artistic ideas to both test extant models and serve as the basis for new ones. Formal user experience reviews will be undertaken at milestone stages in the project allowing us to surface users' tacit views and opinions on their experience when using the developed system. The research method will also entail the commissioning of a new musical work utilising the system, to be premiered in a public workshop and concert in the final month of the project. This will serve as both an opportunity for testing in a 'real world' creative scenario and a means of disseminating the projects outputs in a public forum. There will also be a mid-project 'experimental jam' using the in-development system, which will be broadcast live on the internet. Music technology research has a strong track record for impacts in the sound design industry, particularly in game audio through techniques such as procedural audio generation. Our exploration of novel human-centred approaches to sound design will open up a new field of research and through a firm grounding in creative practice, cross-disciplinary impacts will be all the richer.