HCC: Small: Building Audio Interfaces with Crowdsourced Concept Maps and Active Transfer Learning

美国在软件和多媒体内容（如音乐、电影）方面处于世界领先地位。为了保持这一点，我们必须不断提高软件和媒体制作的标准。用于媒体制作的软件工具（例如音频制作套件Protools）通常具有复杂的界面，其概念化的方式使得除了最专业的人之外的任何人都难以意识到这些工具的强大功能。复杂的界面和陡峭的学习曲线可能会阻碍有创造力的人使用这些工具进行最好的工作。在这里，我们专注于音频制作工具。我们提出了一种以用户为中心的方法，以消除现有音频制作工具与许多人（包括专业音乐家和更广泛的公众）工作的概念框架之间的巨大脱节。我们开发的工具会自动适应用户的概念框架，而不是强迫用户去适应工具。在适当情况下，这些工具将通过与以前的用户互动而获得的主动学习（迁移学习），加快和加强其适应。这些工具还将自动构建众包音频概念图。这将有助于为计算机辅助的定向学习提供便利，使工具使用者能够扩展其概念框架和能力。通过让人们按照自己的方式操纵音频，并通过定向学习来增强他们对这些工具的了解，我们希望改变交互体验，使计算机成为支持和增强创造力的设备，而不是障碍。所开发的工具将可直接用于实践音乐家，也将促进公众的学习和创造力。这些技术也将适用于助听器的个性化和听力学家的新诊断系统。我们的工具个性化方法是人机交互的核心工作，应该推广到其他创造性活动（例如图像处理）。主动学习和迁移学习的进步将对机器学习研究人员具有重要价值。找到音频的可量化参数与实践音乐家用来描述声音的语言和隐喻之间的关系是这项工作的核心。这对认知科学家、语言学家、人工智能研究人员和工程师都很感兴趣。音频术语的概念图也应该被证明对机器翻译有用。将人类描述性术语映射到机器可操作参数的技术的广泛应用将改变艺术家和科学家的期望。艺术家将能够探索新的创造力，目前需要在截然不同的领域（例如信号处理和绘画）投入大量时间。这有可能改变信息科学，并导致新的创造力认知模型，为技术和艺术的教育和研究新方法奠定基础。

The United States is a world-leader in software and in multimedia content (e.g. music, film). To remain so, we must continually raise the bar in both software and media production. Software tools for media production (e.g. the audio production suite Protools) often have complex interfaces, conceptualized in ways that makes it difficult for any but the most expert to realize the power of these tools. Complex interfaces and steep learning curves can discourage creative people from doing their best work with such tools. Here, we focus on audio production tools. We propose a user-centered approach to remove the great disconnect between existing audio production tools and the conceptual frameworks within which many people work, both expert musicians and the broader public. The tools we develop will automatically adapt to the user's conceptual framework, rather than forcing the user to adapt to the tools. Where appropriate, the tools will speed and enhance their adaptation using active learning informed by interaction with previous users (transfer learning). The tools will also automatically build a crowdsourced audio concept map. This will help provide facilities for computer-aided, directed learning, so that tool users can expand their conceptual frameworks and abilities. By letting people manipulate audio on their own terms and enhancing their knowledge of such tools with directed learning, we expect to transform the interaction experience, making the computer a device that supports and enhances creativity, rather than an obstacle.This work will have a number of broader impacts. The tools developed will be directly usable by practicing musicians and will also facilitate learning and creativity for the general public. These techniques will also be applicable to personalization of hearing aids and new diagnostic systems for audiologists. Our approach to tool personalization is core work in human-computer interaction and should generalize to other creative activities (e.g. image manipulation). Resulting advances in active and transfer learning will be of great value to machine learning researchers. Finding the relationships between quantifiable parameters of audio and the language and metaphors used by practicing musicians to describe sound is central to this work. This is of great interest to cognitive scientists, linguists, artificial intelligence researchers, and engineers. Concept maps for audio terms should also prove useful for machine translation. Broad application of techniques to map human descriptive terms on to machine-manipulable parameters will change expectations for both artists and scientists. Artists will be able to explore new lines of creativity that currently require significant investments of time in vastly disparate fields (e.g. signal processing and painting). This has the potential to transform information science and lead to new cognitive models of creativity, forming the basis for new approaches to education and research in both technology and in art.