Cognitive and Structural Approaches to Contemporary Audiovisual Computer Aided Composition

《神经科学》杂志最近的研究支持了视觉艺术和实验音乐界的许多人长期持有的观点，即我们对声音的体验和光的体验之间存在着一种感官联系。这些联系已经被证明是动态的，根据声音和视觉材料的组织方式以可靠的方式变化。这种认识的哲学含义和潜在的文化影响是巨大的。因此，它正在开辟一条研究人类视听感知本质的前沿道路，这在以前被认为是不可能的。此外，用于同时操作声音和视觉材料的实时软件的最新进展，以及越来越强大的多媒体技术的成本下降，正在使探索多感官，结构相关的视听合成技术成为可能，这些技术将彻底改变我们对运动图像和声音的感知以及我们与运动图像和声音的互动。与此同时，在认知、神经科学、心理学和计算领域的研究正在突出具体的新方法来构建和感知视听材料，这可能更加重要。这项研究揭示了理解、参与和操纵我们对声音和视觉的意识体验的策略。这些策略是可靠的，可重复的，并且证明了意识本身的局限性。就像视听体验的多感官性质已经成为实验性视听工作的一个特征一样，科学和艺术研究中的其他非凡启示已经并正在成为现有实验性视听实践的重要组成部分。随着研究揭示出有助于我们理解我们如何与宇宙相互作用的证据，这些联系只会增加。这些事实本身就支持了本研究项目所提出的与当代理论和实践的具体接触。这项研究将与神经科学、认知、心理学、计算、计算机音乐、电影和视觉艺术等领域的顶尖学者进行互动。此外，它还将与来自电视，电影和电脑游戏设计的当代广播媒体从业者联系。它的目的是发现和揭示新的概念，在视听作品的创作中，使用的技术是通过对我们与视听材料的接触的理解的发展而发现的。它将通过合作、咨询、实验、软件开发、视听作曲实践、电影制作、数字媒体实践、计算机音乐研究、生成和算法探索以及人工智能研究来实现这一目标。这个持续的跨学科实践项目将得到金史密斯学院优秀的研究文化的支持。它位于Goldsmiths电子音乐工作室，将受益于该机构在视觉感知、音乐认知、人工智能和认知神经心理学领域的研究人员的直接参与。因此，研究将在一个有效支持其成功开发和完成的环境中进行。这项研究将产生具体的作品，利用认知和结构方法来创作视听作品，并在关注当代声音，音乐，电影，视频和数字技术的重要国际活动中获得高知名度。它将参与当代理论和实践在节日，也通过出版。此外，它将实例化，发展和保持与广播行业的相关话语，并导致对我们与视听和数字媒体文化互动的一些最引人入胜和最重要问题的更多理解。

Recent research in Neuroscience supports the long-held belief of many in the Visual arts and Experimental music community that there is a sensory connection between our experience of sound and our experience of light. These connections have been shown to be dynamic, changing in a reliable fashion depending on the way in which sonic and visual material is organised. The philosophical implications and potential cultural impact of this realisation is enormous. As such, it is carving a path for cutting edge research into the nature of human audiovisual perception in ways which have previously been thought impossible. Furthermore, recent progress into the development of real-time software for the simultaneous manipulation of sonic and visual material, in addition to the falling cost of increasingly more powerful multimedia technology, is making possible the exploration of multisensory, structurally related audiovisual composition techniques in ways which will revolutionise our perception of, and our interaction with moving images and sound.Concurrently, research in the fields of Cognition, Neuroscience, Psychology and Computation are highlighting specific new approaches to the structuring and perception of audiovisual material which may potentially be even more significant. This research reveals strategies for the understanding, engagement with and manipulation of our conscious experience of sound and vision. These strategies are reliable, repeatable, and demonstrative of the limits of consciousness itself. In the same way that the multisensory nature of audiovisual experience is already a feature of experimental audiovisual work, other extraordinary revelations in scientific and arts research have been and are becoming a major part of existing experimental audiovisual practice. These connections will only increase as research reveals evidence which aids in our understanding of how we interact with the universe. These facts alone support the specific engagement with contemporary theory and practice proposed by this research project.This research will engage and interact with leading academics in the fields of Neuroscience, Cognition, Psychology, Computation, Computer Music, Film and Visual arts. In addition it will liaise with contemporary Broadcast media practitioners from Television, Film and Computer Games design. It aims to discover and reveal novel concepts in the creation of audiovisual works using techniques uncovered through developments in the understanding of our engagement with audiovisual material. It will achieve this through collaboration, consultation, experimentation, software development, Audiovisual composition practice, Film making, Digital Media practice, Computer Music research, generative and algorithmic exploration and Artificial Intelligence research. This continuing program of interdisciplinary practice will be supported by the excellent research culture at Goldsmiths College. Housed in the Goldsmiths Electronic Music Studio, it will benefit from the direct involvement of institution's own researchers in the fields of Visual Perception, Music Cognition, Artificial Intelligence and Cognitive Neuropsychology. As such, the research will be taking place within an environment which will effectively support its successful development and completion.This research will result in the creation of specific works which exploit cognitive and structural approaches to audiovisual composition, and in addition gain high visibility at key international events which focus on contemporary approaches to Sound, Music, Film, Video and digital technology. It will engage with contemporary theory and practice at festivals, and also through publication. Furthermore, it will instantiate, develop and maintain a relevant discourse with the broadcast industries, and lead to an increased understanding of some of the most engaging and important issues surrounding our interaction with audiovisual and digital media culture.