CAREER: Making music documents accessible in musical terms

职业：以音乐术语制作音乐文档

• 批准号: 0643752
• 负责人: Bryan Pardo
• 金额: $ 49.07万
• 依托单位: Northwestern University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-01-01 至 2012-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0643752&HistoricalAwards=false
• 关键词: CAREER; Making; music; documents; accessible

Making Music Documents Accessible in Musical TermsOne of the key problems facing us in the 21st century is information retrieval and management. Finding ways to automatically index, label, and access multimedia content (such as music documents) in meaningful ways is an open research question that increases in importance as multimedia databases proliferate and grow. Music collections, such as the 3.5 million recordings in Apple Computer's iTunes repository, comprise one of the most popular categories of on-line multimedia content.For scholars, musicians and even casual listeners, the music document is only the beginning, a tool to initiate the task at hand. Musicians may be interested in remixing a musical recording even though all they have available is the final mix. Scholars may wish to analyze the harmonies in a piece. Others may want karaoke that follows the singer's expressive timing, or a way to remove the sound of an unwanted cell phone ring from a recording of their daughter's flute recital.The objective of this research is to develop two key facilitating technologies to enable these kinds of interactions: score alignment and source separation. Score alignment, involves aligning an audio performance and to the events in a machine-readable music score. When aligned to a score, a performance can be addressed by melodic and harmonic content. We propose to advance the state-of-the-art by enabling a machine to follow partially specified scores (such as Jazz lead sheets). This alignment require significant inference about likely surface structures (the note sequence in an improvised solo) from deeper structural descriptions in the score (the chords in a lead sheet). This will enable alignment of entire classes of music, such as much Jazz, Pop and Rock, that cannot currently be aligned to scores.The second technology, source separation, is the process of isolating individual source signals, given mixtures of the source signals. With source separation, individual instruments and sounds can be accessed, identified and manipulated in ways beyond the power of commercial audio search and editing software. We will advance the field through score-informed separation, as well as new iterative methods for approximating source models from acoustic mixtures.The idea is to develop a synergistic system for music-information-retrieval and interaction that uses multiple document modalities (written scores, audio files, MIDI) to infer more about the music structure than is possible using a single modality.This research will impact the signal-processing community (source separation), the music information retrieval community (music indexing and search) and the artificial intelligence community (tools for intelligent abstraction of real-world data). To broadly disseminate the work, demonstration tools will be made available over the internet and results will be published in relevant journals and conferences. The PI is committed to involving undergraduates and members of historically underrepresented groups in research, working with the SROP and UROP programs to make this happen. The PI also teaches the course "Machine Perception of Music" where research results will be disseminated to a wide variety of students.

使音乐文献在音乐术语中可访问21世纪我们面临的关键问题之一是信息检索和管理。寻找以有意义的方式自动索引、标记和访问多媒体内容(如音乐文档)的方法是一个开放的研究问题，随着多媒体数据库的激增和增长，该问题的重要性日益增加。音乐收藏，如Apple Computer的iTunes库中的350万份录音，构成了最受欢迎的在线多媒体内容类别之一。对于学者、音乐家甚至普通听众来说，音乐文档只是开始，是启动手头任务的工具。音乐家们可能会对重新混音感兴趣，即使他们所能得到的只是最终的混音。学者们可能希望分析一首曲子中的和声。其他人可能想要遵循歌手富有表现力的节奏的卡拉OK，或者是一种从他们女儿的长笛独奏会录音中去除不想要的手机铃声的方法。这项研究的目标是开发两种关键的促进技术来实现这种互动：配乐对齐和来源分离。乐谱对齐，包括将音频表演与机器可读乐谱中的事件对齐。当与乐谱对齐时，演奏可以通过旋律和和声内容来处理。我们建议通过使机器能够跟踪部分指定的乐谱(例如Jazz Lead Sheet)来推进最先进的技术。这种对齐需要从乐谱中更深层次的结构描述(铅板中的和弦)对可能的表面结构(即兴独奏中的音符序列)进行重要的推断。这将实现整个音乐类别的对齐，例如许多爵士乐、流行音乐和摇滚乐，这些音乐目前无法与乐谱对齐。第二种技术，源分离，是在给定源信号的混合情况下隔离单个源信号的过程。通过源分离，可以使用商业音频搜索和编辑软件无法访问、识别和处理单个乐器和声音的方式。我们将通过乐谱信息分离以及从声音混合中近似源模型的新迭代方法来推动该领域的发展。想法是开发一个音乐信息检索和交互的协同系统，它使用多个文档模式(书面乐谱、音频文件、MIDI)来推断关于音乐结构的更多信息，而不是使用单一模式。这项研究将影响信号处理社区(源分离)、音乐信息检索社区(音乐索引和搜索)和人工智能社区(用于智能提取真实世界数据的工具)。为了广泛传播这项工作，将在互联网上提供演示工具，并将在相关期刊和会议上公布结果。PI致力于让本科生和历史上代表性不足的群体的成员参与研究，与SROP和UROP计划合作实现这一目标。国际音乐学院还开设了“机器感知音乐”课程，研究成果将被广泛传播给不同的学生。