A dynamic emotion feedback system for music using audio, symbolic, and physiological features

该项目建立在多伦多的音乐软件开发公司WaveDNA和瑞尔森大学SMART实验室之间长期合作的成果之上。该项目将专注于开发一个音乐创作和制作流程中的情感反馈系统。音乐作曲家，专业和业余音乐家在音乐创作过程中都有特定的情感目标。此外，他们会热衷于了解潜在的情感评价的听众，而听他们的作品。尽管数字音频工作站在音乐制作社区中被广泛使用，但目前还不存在向作曲家提供情感反馈的商业软件。该项目中提出的动态情感反馈系统不仅有利于合成过程，而且还将使WaveDNA与消费产品通过计算分析实现性能反馈的趋势保持一致。 从研究的角度来看，虽然一些研究已经取得了进展，预测的“一般听众”的基础上从音乐和音频特征的信息的情感判断，情感预测仍然是难以捉摸的，因为它的主观性质。SMART实验室的初步工作表明，生理反应是听众情绪状态的有力指标，可用于模拟听众对音乐的情绪反应。鉴于该实验室在音乐、情感和生理学领域的丰富经验，通过此次合作，我们的目标是：（1）通过使用机器学习方法开发和验证计算模型，进一步了解当前对音乐的情感反应;（2）将验证过的模型转化为适合实际使用的系统;（3）将这些情感预测模型集成到WaveDNA的软件套件中。

This project builds on the accomplishments of a long-standing collaboration between WaveDNA, a music software development company in Toronto, and the SMART lab at Ryerson University. The project will focus on developing an emotion feedback system in a music composition and production workflow. Music composers, professional and amateur have specific emotional goals in mind during music creation. Additionally, they would be keen to know about the potential emotion appraisals made by listeners, while listening to their compositions. Despite the widespread use of digital audio workstations in the music production community, there currently exists no commercial software that provides emotion feedback to composers. The dynamic emotion feedback system proposed in this project, would not only benefit the composition process but would also strongly align WaveDNA with the trend in consumer products towards performance feedback through computational analytics. From a research standpoint, although several studies have made advances in predicting emotion judgments of a "general listener" based on information from the music and audio characteristics, emotion prediction remains elusive because of its subjective nature. Preliminary work in the SMART lab has showed that physiological responses are powerful indicators of a listener's emotional state and may be used to model the listener's emotional responses to music. Given the lab's extensive experience in the area of music, emotion, and physiology, through this collaboration, we aim to (1) further current understanding of emotional response to music by developing and validating computational models using machine learning methods; (2) translate validated models into systems appropriate for practical use; and (3) integrate these emotion prediction models into WaveDNA's software suite.