CAREER: An Integrated Framework for Multimodal Music Search and Discovery

音乐制作和发行方面的一场革命使数百万首歌曲可以在互联网上即时提供给几乎任何人。然而，在不知道相关艺术家或歌曲名称的情况下寻找“带大提琴的黑暗电子琴”或“像U2‘S那样的音乐”的听众，或者想要在大量未知的民族音乐中搜索的音乐专家，将面临严峻的挑战。需要新的音乐搜索和发现技术来帮助用户找到所需的内容。互联网上有关音乐的信息(音频片段、歌词、Web文档、图像、乐队网络等)不是基于文本的、多模式的特征。对依赖于单峰、基于文本的数据结构的现有数据库技术提出了新的困难挑战。该项目解决了作为解决这一挑战的核心的两个基本研究问题：(1)使用描述性关键字对(非基于文本的)音频内容进行自动注释；(2)自动集成多模式数据库的不同种类的内容，以改进在因特网或个人数据库上的音乐搜索和发现。由此产生的体系结构利用了机器学习的自动化和可扩展性以及人类计算的有效性，吸引了世界各地的音乐专业人员或爱好者。研究总体上解决了多媒体信息检索的核心问题，使得能够设计用于多模式数据库的新一代富有表现力和灵活性的检索系统，应用于音乐发现、视频检索、家用PC上的多媒体内容索引等。该项目的结果，包括软件库和注释音乐数据集，将纳入正在进行的教育和外联活动，并通过项目网站(http://cosmal.ucsd.edu/~gert/CAREER.html))传播，以加强音乐信息检索方面的研究和教育。

A revolution in music production and distribution has made millions of songs instantly available to virtually anyone, on the Internet. However, a listener looking for "dark electronica with cello" or "music like U2's", without knowing a relevant artist or song name, or a musicologist wanting to search through large amounts of unknown ethnic music, would face serious challenges. Novel music search and discovery technologies are required to help users find the desired content.The non-text-based, multimodal character of Internet-wide information about music (audio clips, lyrics, web documents, images, band networks, etc.) poses a new and difficult challenge to existing database technology that depends on unimodal, text-based data-structures. This project addresses two fundamental research questions at the core of addressing this challenge: (1) The automated annotation of (non-text-based) audio content with descriptive keywords; and (2) the automated integration of the heterogeneous content of multimodal databases, to improve music search and discovery on the Internet or in a personal database. The resulting architecture leverages the automation and scalability of machine learning with the effectiveness of human computation, engaging music professionals or enthusiasts around the world.The research addresses questions at the core of multimedia information retrieval in general, enabling the design of a new generation of expressive and flexible retrieval systems for multimodal databases, with applications to music discovery, video retrieval, indexing multimedia content on the home PC, etc.The results of this project, including a software library and annotated music data sets, will be incorporated in ongoing education and outreach activities and disseminated via the project website (http://cosmal.ucsd.edu/~gert/CAREER.html) to enhance research and education in music information retrieval.