CAREER: Making music documents accessible in musical terms

音乐文献的检索与管理是21世纪世纪我们面临的关键问题之一。寻找方法来自动索引，标签和访问多媒体内容（如音乐文件）在有意义的方式是一个开放的研究问题，随着多媒体数据库的激增和增长的重要性。音乐收藏，例如苹果电脑iTunes存储库中的350万个录音，是最受欢迎的在线多媒体内容类别之一。对于学者、音乐家甚至普通听众来说，音乐文档只是开始，是启动手头任务的工具。音乐家可能会对重新混音音乐录音感兴趣，即使他们所拥有的只是最终的混音。学者们不妨分析一下乐曲中的和声。另一些人可能希望卡拉OK遵循歌手的表达时间，或一种方法来消除一个不必要的手机铃声从他们女儿的长笛recital.Objective录音本研究的目的是开发两个关键的促进技术，使这些类型的互动：分数对齐和源分离。 乐谱对齐，涉及将音频表演与机器可读乐谱中的事件对齐。当与乐谱对齐时，表演可以通过旋律和和声内容来解决。我们建议通过使机器能够遵循部分指定的分数（例如Jazz铅表）来推进最先进的技术。这种对齐需要从乐谱中更深层次的结构描述（铅页中的和弦）中对可能的表面结构（即兴独奏中的音符序列）进行重要的推断。这将使整个类的音乐，如爵士乐，流行音乐和摇滚乐，目前无法调整到乐谱的对齐。第二种技术，源分离，是隔离单独的源信号的过程，给定的混合源信号。通过源分离，可以以超出商业音频搜索和编辑软件能力的方式访问、识别和处理单个乐器和声音。我们将通过分数通知分离，以及新的迭代方法，从声学混合物近似源模型推进该领域。其想法是开发一个协同系统，用于音乐信息检索和交互，使用多种文档模态（书面分数，音频文件，这一研究将对信号处理界产生影响（源分离）、音乐信息检索社区（音乐索引和搜索）和人工智能社区（真实世界数据的智能抽象工具）。为了广泛传播这项工作，将在互联网上提供示范工具，并将在相关期刊和会议上发表成果。PI致力于让本科生和历史上代表性不足的群体成员参与研究，与SROP和UROP计划合作实现这一目标。PI还教授“音乐的机器感知”课程，研究成果将传播给各种各样的学生。

Making Music Documents Accessible in Musical TermsOne of the key problems facing us in the 21st century is information retrieval and management. Finding ways to automatically index, label, and access multimedia content (such as music documents) in meaningful ways is an open research question that increases in importance as multimedia databases proliferate and grow. Music collections, such as the 3.5 million recordings in Apple Computer's iTunes repository, comprise one of the most popular categories of on-line multimedia content.For scholars, musicians and even casual listeners, the music document is only the beginning, a tool to initiate the task at hand. Musicians may be interested in remixing a musical recording even though all they have available is the final mix. Scholars may wish to analyze the harmonies in a piece. Others may want karaoke that follows the singer's expressive timing, or a way to remove the sound of an unwanted cell phone ring from a recording of their daughter's flute recital.The objective of this research is to develop two key facilitating technologies to enable these kinds of interactions: score alignment and source separation. Score alignment, involves aligning an audio performance and to the events in a machine-readable music score. When aligned to a score, a performance can be addressed by melodic and harmonic content. We propose to advance the state-of-the-art by enabling a machine to follow partially specified scores (such as Jazz lead sheets). This alignment require significant inference about likely surface structures (the note sequence in an improvised solo) from deeper structural descriptions in the score (the chords in a lead sheet). This will enable alignment of entire classes of music, such as much Jazz, Pop and Rock, that cannot currently be aligned to scores.The second technology, source separation, is the process of isolating individual source signals, given mixtures of the source signals. With source separation, individual instruments and sounds can be accessed, identified and manipulated in ways beyond the power of commercial audio search and editing software. We will advance the field through score-informed separation, as well as new iterative methods for approximating source models from acoustic mixtures.The idea is to develop a synergistic system for music-information-retrieval and interaction that uses multiple document modalities (written scores, audio files, MIDI) to infer more about the music structure than is possible using a single modality.This research will impact the signal-processing community (source separation), the music information retrieval community (music indexing and search) and the artificial intelligence community (tools for intelligent abstraction of real-world data). To broadly disseminate the work, demonstration tools will be made available over the internet and results will be published in relevant journals and conferences. The PI is committed to involving undergraduates and members of historically underrepresented groups in research, working with the SROP and UROP programs to make this happen. The PI also teaches the course "Machine Perception of Music" where research results will be disseminated to a wide variety of students.