Predicting musical choices using computational models of cognitive and neural processing

近年来，音乐消费已经急剧转向从庞大的音乐库中流媒体，让用户承受了大量可能的音乐选择。这种新形势使得开发智能工具来帮助听众选择要听的音乐势在必行。音乐技术的研究传统上遵循纯工程方法，这在一定程度上推动了该领域的发展。然而，由于缺乏一个强大的、科学的听众模型，这个模型可以用来告诉数字音乐播放器（例如iTunes、Spotify、Last FM）用户选择音乐的偏好，因此阻碍了进展。拟议的研究通过开发创建计算模型所需的科学知识来解决这一差距，该模型可以根据使用脑电图（EEG）记录的音乐特征和脑电反应来预测听众的音乐选择。其主要思想是对听众选择听音乐时所涉及的心理和神经过程进行科学的理解。该假说认为，通过结合心理学原理、音乐特征和从头皮上记录的脑电反应，可以准确预测听者的音乐选择。本研究有两个基础：一是通过听众研究来识别这些心理原理、音乐特征和大脑反应；其次，利用这些知识建立一个计算模型，预测听众对音乐的选择。建模方法包括三个组成部分，以捕捉对音乐选择有影响的音乐特征。第一分量使用使用信号处理方法从音频中提取的声学特征，如不谐和时间规律性。第二个部分采用更高层次的认知方法，使用基于音符级音乐表示的信息论模型提取复杂性的度量。第三部分从抒情内容的情感文本分析中提取音乐的情感意图。了解这些特征的确切性质和权重，以及它们如何影响音乐选择，需要详细检查听众在听音乐时所做的选择。因此，调查实际听众的行为是本研究的核心，将进行两项研究。第一个重点是在参与者听音乐节选时收集脑电图数据，并选择它们供将来听。机器学习方法将用于预测听众的决定，使用在做出选择之前记录的时变神经反应的特征。第二个用户研究的目的是收集数据，从音乐本身的特征和听众的属性中预测选择的建模。这将涉及更大范围的音乐节选和更广泛的听众，而不是脑电图研究的实际情况。目的是了解音乐选择中涉及的心理过程，并利用这些知识来完善、参数化和优化音乐选择的计算模型。研究的最后阶段将开发一个综合的音乐选择预测模型，将使用音乐信号的预测模型与从神经信号进行预测的预测模型相结合。这种一体化模式的发展具有很强的创新性。在过去两年中，面向消费者市场的价格实惠的多通道无线脑电图耳机的出现，使得使用这些设备来控制媒体播放器和其他交互系统成为可能。因此，将神经科学、音乐认知和机器学习的研究结合起来，体现PI独特的跨学科专业知识，了解神经信号、音乐结构和歌曲选择之间的映射关系的时机已经成熟。

Music consumption has shifted dramatically in recent years towards streaming from vast music libraries, overloading the user with the enormity of possible musical choices. This new landscape makes it imperative to develop intelligent tools to help listeners choose music to listen to. Research in music technology has traditionally followed a pure engineering approach, which has taken the field some way. However, progress is being hindered by the lack of a robust, scientifically grounded model of the listener, which can be used to inform digital music players (e.g., iTunes, Spotify, Last FM) about users' preferences for selecting music.The proposed research addresses this gap by developing the scientific knowledge needed to create computational models which can predict listeners' musical choices from features of the music and electrical brain responses recorded using Electroencephalography (EEG). The principal idea is to develop a scientific understanding of the psychological and neural processes involved when a listener chooses music to listen to. The hypothesis is that accurate predictions of a listener's musical choices can be made using a combination of psychological principles, musical features and electrical brain responses recorded from the scalp. This research has two foundations: first, to conduct listener studies to identify those psychological principles, musical features and brain responses; and second, to use that knowledge to build a computational model that predicts a listener's choice of music.The modelling approach includes three components to capture features of music that have an impact on musical choices. The first component uses acoustic features such as dissonance and temporal regularity extracted from the audio using signal processing methods. The second component takes a higher-level cognitive approach, extracting measures of complexity using information-theoretic models based on note-level representations of music. The third component extracts the emotional intentions of the music from affective textual analysis of the lyrical content. Understanding the exact nature and weighting of these features and how they impact on musical choice requires the detailed examination of the choices that listeners make when listening to music.Therefore, investigating the behaviour of actual listeners is central to this research and two studies will be performed. The first focuses on collecting EEG data while participants listen to musical excerpts and select them for future listening. Machine learning methods will be used to predict the listeners' decisions using features of the time-varying neural response recorded prior to the choice being made. The purpose of the second user study is to collect data for predict modelling of choices from features of the music itself and attributes of the listener. This will involve a larger range of musical excerpts and a wider range of listeners than is practical for the EEG study. The objective is to understand the psychological processes involved in musical choice and to use this knowledge to refine, parameterise and optimise the computational models of musical choice.The final stage of the research will develop an integrated predictive model of musical choice by combining predictive models using the musical signal with those making predictions from the neural signal. The development of such an integrated model is highly innovative. The advent within the last two years of affordable, multi-channel, wireless EEG headsets for the consumer market makes the possibility of using these devices to control media players and other interactive systems a real possibility. Therefore, the time is ripe to combine research in neuroscience, music cognition and machine learning, reflecting the unique interdisciplinary expertise of the PI, to understand the mapping between neural signals, musical structure and song selections.