MIMIC: Musically Intelligent Machines Interacting Creatively

这个项目是对计算机和艺术领域发生的重大变化的直接回应。人工智能和机器学习的最新发展正在导致研究人员如何创作音乐和艺术的革命（Broad和Grierson，2016）。然而，这项技术还没有被整合到针对创意的软件中。由于机器学习的复杂性，以及缺乏可用的工具，这些方法只能由专家使用。为了解决这一问题，我们将创造新的、用户友好的技术，使外行用户（作曲家和业余音乐家）能够理解并将这些新的计算技术应用于他们自己的创造性工作中。机器学习支持创造性活动的潜力正在以显著的速度增长，无论是在创造性理解方面还是在潜在的应用方面。音乐和声音生成领域的新兴工作从音乐机器人扩展到生成应用程序，从高级机器收听到可以以任何给定风格作曲的设备。通过利用互联网作为一个实时的软件生态系统，拟议的项目将研究如何使这种技术最好地接触艺术家，并发挥其潜力，从根本上改变该领域的创作实践。而不是专注于计算机作为一个原始的创作者，我们将创建平台，最新的技术可以被艺术家使用作为他们的日常创作实践的一部分。目前人工智能的研究，特别是机器学习，已经导致人工智能系统在语音和图像识别（Cortana，Siri等）等领域的性能取得了令人难以置信的飞跃。谷歌和其他公司已经展示了如何将这些方法用于创造性目的，包括生成语音和音乐（DeepMinds的WaveNet和谷歌的洋红），图像（Deep Dream）和游戏智能（DeepMind的AlphaGo）。该项目的研究人员一直在使用深度学习、卷积神经网络（CNN）、递归神经网络（RNN）、长短期记忆网络（LSTM）和其他方法来开发可供艺术家用来创作声音和音乐的智能系统。我们已经是世界上第一个创建可重复使用的软件的公司，这些软件可以“听”大量的录音，并以这些录音为例，在音频级别上创建全新的录音。在这个为期三年的项目中，我们将开发和推广可供音乐家和艺术家用于创作全新音乐和声音的创意系统。我们将展示这种方法如何影响其他形式的媒体，如电影和视觉艺术的未来。为此，我们将利用现有最方便的公共论坛：万维网，建立一个创造性的平台。我们将通过为新手用户开发一种高级实时编码语言来实现这一目标，并使用简化的隐喻来理解包括深度学习在内的复杂技术。我们还将发布我们为更高级的用户创建的机器学习库，这些用户希望将机器学习技术作为其创意工具的一部分。该项目将涉及整个最终用户，包括研究生，专业艺术家和在线学习环境的参与者。我们将尽早宣传我们的工作，获得必要的反馈，以提供坚实的最终产品和成果。这些技术的有效性已经在Sonic Pi和Ixi Lang等系统中得到了证明，这些系统已经在AHRC通过Live Coding Network（AH/L007266/1）和EC H2020项目RAPID-MIX中得到了支持。最后，这项研究将有力地促进围绕音乐和艺术的未来的对话，巩固英国在这些领域的领导地位。

This project is a direct response to significant changes taking place in the domain of computing and the arts. Recent developments in Artificial Intelligence and Machine Learning are leading to a revolution in how music and art is being created by researchers (Broad and Grierson, 2016). However, this technology has not yet been integrated into software aimed at creatives. Due to the complexities of machine learning, and the lack of usable tools, such approaches are only usable by experts. In order to address this, we will create new, user-friendly technologies that enable the lay user - composers as well as amateur musicians - to understand and apply these new computational techniques in their own creative work.The potential for machine learning to support creative activity is increasing at a significant rate, both in terms of creative understanding and potential applications. Emerging work in the field of music and sound generation extends from musical robots to generative apps, and from advanced machine listening to devices that can compose in any given style. By leveraging the internet as a live software ecosystem, the proposed project examines how such technology can best reach artists, and live up to its potential to fundamentally change creative practice in the field. Rather than focussing on the computer as an original creator, we will create platforms where the newest techniques can be used by artists as part of their day-to-day creative practices. Current research in artificial intelligence, and in particular machine learning, have led to an incredible leap forward in the performance of AI systems in areas such as speech and image recognition (Cortana, Siri etc.). Google and others have demonstrated how these approaches can be used for creative purposes, including the generation of speech and music (DeepMinds's WaveNet and Google's Magenta), images (Deep Dream) and game intelligence (DeepMind's AlphaGo). The investigators in this project have been using Deep Learning, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory Networks (LSTMs), and other approaches to develop intelligent systems that can be used by artists to create sound and music. We are already among the first in the world to create reusable software that can 'listen' to large amounts of sound recordings, and use these as examples to create entirely new recordings at the level of audio. Our systems produce outcomes that out-perform many other previously funded research outputs in these areas.In this three-year project, we will develop and disseminate creative systems that can be used by musicians and artists in the creation of entirely new music and sound. We will show how such approaches can affect the future of other forms of media, such as film and the visual arts. We will do so by developing a creative platform, using the most accessible public forum available: the World Wide Web. We will achieve this through development of a high level live coding language for novice users, with simplified metaphors for the understanding of complex techniques including deep learning. We will also release the machine learning libraries we create for more advanced users who want to use machine learning technology as part of their creative tools. The project will involve end-users throughout, incorporating graduate students, professional artists, and participants in online learning environments. We will disseminate our work early, gaining the essential feedback required to deliver a solid final product and outcome. The efficacy of such techniques has been demonstrated with systems such as Sonic Pi and Ixi Lang, within a research domain already supported by the AHRC through the Live Coding Network (AH/L007266/1), and by EC in the H2020 project, RAPID-MIX. Finally, this research will strongly contribute to dialogues surrounding the future of music and the arts, consolidating the UK's leadership in these fields.