Toward the Scientific Evaluation of Music Information Retrieval Systems

音乐信息检索（MIR）是一项新的、充满活力的国际多学科研究，致力于开发创新的基于内容的搜索方案、新颖的界面和不断发展的网络传递机制，以使世界上巨大的音乐存储可供所有人访问。图书馆员、音乐学家、心理学家、音频工程师、音乐教师和学生、计算机科学家、法律专家、音乐版权持有者和企业高管都参与其中。这个项目的目的是加强为在MIR研究界内建立科学评价范式所作的努力。该项目涉及的主要问题包括对音乐信息复杂性的理解；充分捕捉音乐查询的复杂性；以及对MIR背景下的“相关性”问题的深入研究。为此目的，本项目负责创建一套全面的数据丰富的查询记录，这些记录既基于实际需求，又与所审查的检索技术无关；采用并验证“理性人”方法进行“相关性”评估；建立科学有效的实验方案；并且，正在进行的音乐信息（音频，符号和元数据）的获取，以加强安全，但可访问的研究环境的开发，使研究人员能够远程参与使用大规模测试平台集合。因为这个项目显著增强了MIR研究的基础设施，它在改善获取世界上巨大的音乐资源方面起着关键作用。因此，这个项目影响了音乐爱好者、学者、教育工作者、学生、工业和商业开发商以及所有其他相关方未来与音乐互动的方式。项目网站（http://music-ir.org）提供查阅研究材料、出版物和最新资料的途径。

Music Information Retrieval (MIR) is a new and vibrant international, multidisciplinary research endeavor that strives to develop innovative content-based searching schemes, novel interfaces, and evolving networked delivery mechanisms in an effort to make the world's vast store of music accessible to all. Librarians, musicologists, psychologists, audio engineers, music teachers and students, computer scientists, legal experts, music rights-holders, and business executives are all involved. This project is designed enhance the efforts being made to establish a scientific evaluation paradigm within the MIR research community. Major issues addressed in this project include the comprehension of the complex nature of music information; adequately capturing the complex nature of music queries; and an in-depth examination of the "relevance" problem in the MIR context. To these ends, this project undertakes the creation of comprehensive set of data-rich query records that are both grounded in real-world requirements and neutral with respect to retrieval technique(s) being examined; the adoption, and subsequent validation, of a "reasonable person" approach to "relevance" assessment; the establishment of scientifically valid experimental protocols; and, the ongoing acquisition of music information (audio, symbolic and metadata) to enhance the development of a secure, yet accessible, research environment that allows researchers to remotely participate in the use of the large-scale testbed collection. Because this project significantly enhances the infrastructure of MIR research, it plays a pivotal role in improving access to the world's vast music resources. Thus, this project impacts the way music lovers, scholars, educators, students, industrial and commercial developers, and all other interested parties, will be interacting with music in the future. The project Web site (http://music-ir.org) provides access to the research materials, publications and updates.