Toward the Scientific Evaluation of Music Information Retrieval Systems

音乐信息检索（MIR）是一个新的和充满活力的国际，多学科的研究奋进，努力开发创新的基于内容的搜索方案，新颖的界面，并不断发展的网络传输机制，努力使世界上的音乐的巨大存储访问所有。图书馆员、音乐学家、心理学家、音频工程师、音乐教师和学生、计算机科学家、法律的专家、音乐版权持有人和企业高管都参与其中。该项目的目的是加强正在作出的努力，建立一个科学的评价范式内的和平号研究界。在这个项目中解决的主要问题包括音乐信息的复杂性的理解，充分捕捉音乐查询的复杂性，并在MIR上下文中的“相关性”问题的深入研究。为此目的，该项目着手建立一套全面的数据丰富的查询记录，这些记录既以现实世界的要求为基础，又对所审查的检索技术保持中立;采用并随后验证“合理的人”的方法来评估“相关性”;制定科学上有效的实验协议;不断获取音乐信息（音频、符号和元数据），以加强开发一个安全、可访问的研究环境，使研究人员能够远程参与使用大规模试验平台收集的数据。由于该项目大大加强了MIR研究的基础设施，因此它在改善对世界庞大音乐资源的访问方面发挥了关键作用。因此，这个项目影响了音乐爱好者、学者、教育工作者、学生、工业和商业开发者以及所有其他感兴趣的各方未来与音乐互动的方式。该项目的网址（http：music-ir.org）提供研究材料、出版物和最新资料。

Music Information Retrieval (MIR) is a new and vibrant international, multidisciplinary research endeavor that strives to develop innovative content-based searching schemes, novel interfaces, and evolving networked delivery mechanisms in an effort to make the world's vast store of music accessible to all. Librarians, musicologists, psychologists, audio engineers, music teachers and students, computer scientists, legal experts, music rights-holders, and business executives are all involved. This project is designed enhance the efforts being made to establish a scientific evaluation paradigm within the MIR research community. Major issues addressed in this project include the comprehension of the complex nature of music information; adequately capturing the complex nature of music queries; and an in-depth examination of the "relevance" problem in the MIR context. To these ends, this project undertakes the creation of comprehensive set of data-rich query records that are both grounded in real-world requirements and neutral with respect to retrieval technique(s) being examined; the adoption, and subsequent validation, of a "reasonable person" approach to "relevance" assessment; the establishment of scientifically valid experimental protocols; and, the ongoing acquisition of music information (audio, symbolic and metadata) to enhance the development of a secure, yet accessible, research environment that allows researchers to remotely participate in the use of the large-scale testbed collection. Because this project significantly enhances the infrastructure of MIR research, it plays a pivotal role in improving access to the world's vast music resources. Thus, this project impacts the way music lovers, scholars, educators, students, industrial and commercial developers, and all other interested parties, will be interacting with music in the future. The project Web site (http://music-ir.org) provides access to the research materials, publications and updates.