Toward the Scientific Evaluation of Music Information Retrieval Systems

点击翻译按钮获取中文摘要

Music Information Retrieval (MIR) is a new and vibrant international, multidisciplinary research endeavor that strives to develop innovative content-based searching schemes, novel interfaces, and evolving networked delivery mechanisms in an effort to make the world's vast store of music accessible to all. Librarians, musicologists, psychologists, audio engineers, music teachers and students, computer scientists, legal experts, music rights-holders, and business executives are all involved. This project is designed enhance the efforts being made to establish a scientific evaluation paradigm within the MIR research community. Major issues addressed in this project include the comprehension of the complex nature of music information; adequately capturing the complex nature of music queries; and an in-depth examination of the "relevance" problem in the MIR context. To these ends, this project undertakes the creation of comprehensive set of data-rich query records that are both grounded in real-world requirements and neutral with respect to retrieval technique(s) being examined; the adoption, and subsequent validation, of a "reasonable person" approach to "relevance" assessment; the establishment of scientifically valid experimental protocols; and, the ongoing acquisition of music information (audio, symbolic and metadata) to enhance the development of a secure, yet accessible, research environment that allows researchers to remotely participate in the use of the large-scale testbed collection. Because this project significantly enhances the infrastructure of MIR research, it plays a pivotal role in improving access to the world's vast music resources. Thus, this project impacts the way music lovers, scholars, educators, students, industrial and commercial developers, and all other interested parties, will be interacting with music in the future. The project Web site (http://music-ir.org) provides access to the research materials, publications and updates.