Sonic Phonic - Towards a Generative Alphabet

声音和图像之间的联系的概念可以追溯到希腊哲学家毕达哥拉斯和亚里士多德，他们推测了音乐音阶和彩虹光谱之间的相关性。从传统的艺术工作室实践(素描、绘画、蒙太奇)到媒体艺术的开端，艺术家和作家一直在探索图像、声音和文本关系的概念。技术变革时期，如19世纪之交视听媒体技术的进步和21世纪的数字革命，影响了概念的进步，如20世纪20年代的前卫电影和音乐，20世纪20年代的S，60年代的视频艺术和新音乐，以及目前的数字媒体艺术、数字叙事、人工智能和计算机处理。新技术的迅速发展和传播在人类经历的许多领域产生了变化，计算机创造了并正在改变文化，激发了艺术和科学研究的广泛创新。互动技术和数字环境的融合为语言、文本和声音媒体的呈现创造了更大的能力和可能性。它使观众和界面之间、感官和视觉、听觉和触觉的不同模式之间建立了一种新的关系。神经学家Antonio Damasio最近关于大脑对图像和与意义相关的知觉认知的潜意识参与的研究，为图像、文本和声音感知领域的发展提供了信息，这可能会导致我们感知环境的转变和我们认知进化的可能性。这项研究源于我作为一名艺术家对探索语言、交流和视觉感知概念的艺术作品的兴趣汇聚，以及与装置和多媒体设计中摄影、数字成像、版画和视频等媒体并列相关的问题。目前，我对进化计算过程特别感兴趣，这种过程允许在系统中并置和分层这样的概念和对象。正是这些通过计算融合的概念和视觉空间影响了字母媒体的交叉培养的概念，以及字母表中新结构的新视听想法的出现的可能性。这些研究问题是通过亲身体验儿童参与图像与现实、符号与声音之间的关联而产生的。字母表与象形文字和表意文字的历史联系，以及后来向字母符号(字形)-语音(音素)关系的索引的演变，通过扩展图像、声音和文本以及运动图像、动画和绘画的通感并置的可能性，为数字和交互媒体提供了重新构建符号-声音关系的上下文。目的和对象本研究的目的是创建一个交互式字母原型，以考虑数字(图像、声音和文本)关系的发展和出现的特征和属性，使用位于装置核心的生成数据库的体系结构通过原型艺术品的安装，受邀参与者与不断演变的生成字母空间的互动，并从这些实验性互动中收集数据，以评估参与者和数据库的行为。潜在的应用和优势研究将有助于现有的图像、文本和声音相关概念的知识技术变革影响文化认知和交流的潜力艺术家和作家对字母表的探索使用新的交互界面，数据库和计算处理。它的好处可能包括一种构建数字字母表的中间方法，其中图像、声音和文本作为生成系统的未来潜力可能导致字母表和叙事中的新含义和新关系。长期研究的实际应用包括用于数字写作和叙事、教育、语言应用、娱乐的在线自适应人机字母数据库，以及对在电子学习环境中发展理解的贡献。有了这种潜力，未来的研究很可能会开发在线出版的原型，其中改编可能包括上传新内容的能力，包括基于图像、视频、声音和文本的媒体。可以通过在线进化的合作字母表或替代地为一系列应用创建个人定制字母/语言来开发新的结构，例如对于难语症中风患者，其中有可能进行研究，以结合数据库检查个人图像、声音和人工制品的使用，以触发语言的重新学习。在我的背景研究过程中，我与心理学家进行了讨论，他们对开发声波音响的潜力感兴趣。进一步研究的其他可能性在于动觉触觉界面与生成性数据库和数字内容的开发的相关性。

The concept of an association between sound and image can be traced back to Greek philosophers Pythagoras and Aristotle who speculated on the correlation between musical scale and the rainbow spectrum. From traditional art studio practise (drawing, painting, montage) to the beginnings of media art, the concept of image, sound and text relationships have been explored by artists and writers. Periods of technological change such as advances in audio-visual media technology at the turn of the 19th Centuryand digital revolution of the 21st Century have influenced advances in concepts such as in avant-garde film and music in the 1920's, video art and new music in the 1960's and currently digital media art, digital narrative, artificial intelligence and computational processing. The rapid development and spread of new technologies has effected change in many spheres of human experience, computers have created and are transforming culture, motivating simultaneous widespread innovation in artistic and scientific research. The convergence of interactive technologies and digital environment has created expanded capacities and possibilities for the presentation of language, text and sound media. It has enabled a new relationship between the viewer and the interface, between the senses and the different modes of visual, aural and tactile perception. Recent research by Neuroscientist Antonio Damasio concerning the brain's preconscious engagement with images and perceptual cognition in relation to meaning has informed developments in the area of the perception of image, text and sound, which has the potential to lead towards a shift in our perceptual environment and possibility our cognitive evolution.This research arises from converging interests in my practice as an artist relating to artworks exploring concepts of language, communication and visual perception and issues relating to the juxtaposition of media such as photography, digital imaging, printmaking and video in installation and multimedia design. Currently I have a particular interest in evolutionary computational processes that allow juxtaposition and layering of such concepts and objects within a system. It is these computationally blended conceptual and visual spaces that has influenced the concept of a cross fertilization of alphabetic media and the possibility of emergent audio/visual ideas for new constructs in the alphabet. The research questions have arisen through personal experience of children's' engagement with correlations made between images and reality, symbols and sound. The Alphabet, with its historical association with pictograms and ideograms and subsequent evolution to the indexing of letter symbol (Grapheme)-phonic sound (Phoneme) relationships, provides a context for digital and interactive media to re-frame the symbol-sound relationship by expanding the possibilities for the synesthetic juxtaposition of image, sound and text as well as moving image, animation and drawing.Aims and ObjectivesThe aim of the research is the creation of an interactive alphabetic prototype to consider the development and emergent characteristics and properties of digital (image, sound and text) relationships, using the architecture of a generative database at the heart of the installation.The objective of the interactive alphabet prototype is to facilitate, through the installation of the prototype artwork, the interaction of invited participants with the evolving generative alphabetic space and gather data from these experimental interactions to evaluate the behaviours of participant and database.Potential Applications and BenefitsThe Research will contribute to existing knowledge ofConcepts in image, text and sound correlationThe potential for technological change to influence the cognition and communication of cultureExplorations of the alphabet by artists and writersArtworks using new interactive interfaces, databases and computational processing.Benefits might include an intermedia approach to constructs of digital alphabets where future potential for image sound and text as a generative system may lead to new meaning and relationships in alphabets and narrative.Practical applications for long-term research include online adaptive human-machine alphabetic databases for digital writing and narrative, education, language applications, entertainment, and contributions to the development of understanding in e-learning environments. With that potential, it is likely that future research will develop the prototype for online publication where adaptations may include the ability for uploading of new content including imagery, video, sound and text based media. New constructs could be developed through an online evolving collaborative alphabet or alternatively creation of individual bespoke alphabets/language for a range of applications such as for dysphasic stroke patients, where there is potential for research to examine the use of personal imagery, sounds and artefacts in conjunction with the database to trigger the re-learning of language. Through my background research process I have had discussions with psychologists who are interested in developing the potential of sonic phonic for this purpose. Other possibilities for further research lie in the relevance of kinesthetic haptic interface to the development of the generative database and digital content.