A dynamic emotion feedback system for music using audio, symbolic, and physiological features

这个项目建立在多伦多的音乐软件开发公司WaveDNA和瑞尔森大学的智能实验室之间长期合作的成果之上。该项目将专注于开发音乐创作和制作工作流程中的情感反馈系统。无论是作曲家、专业作曲家还是业余作曲家，在音乐创作过程中都有特定的情感目标。此外，他们很想知道听众在听他们的作品时可能做出的情绪评估。尽管数字音频工作站在音乐制作社区中广泛使用，但目前还不存在向作曲家提供情感反馈的商业软件。本项目中提出的动态情绪反馈系统，不仅有利于创作过程，还将使WaveDNA与消费产品通过计算分析进行性能反馈的趋势保持强烈的一致。 从研究的角度来看，虽然一些研究已经在基于音乐和音频特征的信息来预测普通听众的情绪判断方面取得了进展，但由于其主观性，情绪预测仍然是难以捉摸的。智能实验室的初步工作表明，生理反应是听众情绪状态的强大指示器，可以用来模拟听众对音乐的情绪反应。鉴于该实验室在音乐、情感和生理学领域的广泛经验，通过这次合作，我们的目标是(1)通过使用机器学习方法开发和验证计算模型，进一步了解当前对音乐的情感反应；(2)将经过验证的模型转化为适合实际使用的系统；以及(3)将这些情感预测模型集成到WaveDNA的软件套件中。

This project builds on the accomplishments of a long-standing collaboration between WaveDNA, a music software development company in Toronto, and the SMART lab at Ryerson University. The project will focus on developing an emotion feedback system in a music composition and production workflow. Music composers, professional and amateur have specific emotional goals in mind during music creation. Additionally, they would be keen to know about the potential emotion appraisals made by listeners, while listening to their compositions. Despite the widespread use of digital audio workstations in the music production community, there currently exists no commercial software that provides emotion feedback to composers. The dynamic emotion feedback system proposed in this project, would not only benefit the composition process but would also strongly align WaveDNA with the trend in consumer products towards performance feedback through computational analytics. From a research standpoint, although several studies have made advances in predicting emotion judgments of a "general listener" based on information from the music and audio characteristics, emotion prediction remains elusive because of its subjective nature. Preliminary work in the SMART lab has showed that physiological responses are powerful indicators of a listener's emotional state and may be used to model the listener's emotional responses to music. Given the lab's extensive experience in the area of music, emotion, and physiology, through this collaboration, we aim to (1) further current understanding of emotional response to music by developing and validating computational models using machine learning methods; (2) translate validated models into systems appropriate for practical use; and (3) integrate these emotion prediction models into WaveDNA's software suite.