ANR Multimodal analysis and knowledge inference for musical orchestration (MAKIMOno)

点击翻译按钮获取中文摘要

This project addresses scientifically one of the most complex aspects of music: the use of timbre to shape music through various modes of orchestration. In close interaction with a Canadian company, OrchPlayMusic Inc., this first-of-its-kind project will lead to the creation of information technologies for human interaction with digital media that will radically change orchestration pedagogy, provide better tools for the computer-aided interactive creation of musical content, and lead to a better understanding of perceptual principles underlying orchestration practice. Timbre is the complex set of tone colours that distinguish sounds emanating from different instruments or their blended combinations. Orchestration is the art of writing music that combines different instruments to achieve various sonic goals. Orchestration pedagogy focuses solely on describing how composers score instruments rather than understanding why they made such choices. We will address the why scientifically with the methods of computer science, digital signal processing, and experimental psychology. Drawing from the results of the co-applicants' previous research, this project will create efficient novel analysis, learning, and interaction techniques that reveal the underlying theoretical bases for orchestration. These techniques include multivariate time series analysis of musical signals and knowledge inference through deep representational learning that will decipher automatically the structure of multimodal musical representations (music symbols, acoustic properties, perceptual results) in order to provide optimal descriptors for the understanding of orchestration principles. The project will rely on both solid perceptual principles and empirically characterized orchestration examples to build a scientifically grounded theory of musical orchestration from a large existing multimodal database, to be extended within the course of the research. Solving these complex issues in a strong international collaboration with the French ANR will also lead to generic multimodal learning and analysis techniques broadly applicable to the fields of machine learning and computational perception that will place Canada at the forefront of interdisciplinary innovation.**************