Predicting musical choices using computational models of cognitive and neural processing

使用认知和神经处理的计算模型预测音乐选择

• 批准号: EP/M000702/1
• 负责人: Marcus Pearce
• 金额: $ 12.77万
• 依托单位: Queen Mary University of London
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2015
• 资助国家: 英国
• 起止时间: 2015 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FM000702%2F1
• 关键词: Predicting; musical; choices; using; computational

Music consumption has shifted dramatically in recent years towards streaming from vast music libraries, overloading the user with the enormity of possible musical choices. This new landscape makes it imperative to develop intelligent tools to help listeners choose music to listen to. Research in music technology has traditionally followed a pure engineering approach, which has taken the field some way. However, progress is being hindered by the lack of a robust, scientifically grounded model of the listener, which can be used to inform digital music players (e.g., iTunes, Spotify, Last FM) about users' preferences for selecting music.The proposed research addresses this gap by developing the scientific knowledge needed to create computational models which can predict listeners' musical choices from features of the music and electrical brain responses recorded using Electroencephalography (EEG). The principal idea is to develop a scientific understanding of the psychological and neural processes involved when a listener chooses music to listen to. The hypothesis is that accurate predictions of a listener's musical choices can be made using a combination of psychological principles, musical features and electrical brain responses recorded from the scalp. This research has two foundations: first, to conduct listener studies to identify those psychological principles, musical features and brain responses; and second, to use that knowledge to build a computational model that predicts a listener's choice of music.The modelling approach includes three components to capture features of music that have an impact on musical choices. The first component uses acoustic features such as dissonance and temporal regularity extracted from the audio using signal processing methods. The second component takes a higher-level cognitive approach, extracting measures of complexity using information-theoretic models based on note-level representations of music. The third component extracts the emotional intentions of the music from affective textual analysis of the lyrical content. Understanding the exact nature and weighting of these features and how they impact on musical choice requires the detailed examination of the choices that listeners make when listening to music.Therefore, investigating the behaviour of actual listeners is central to this research and two studies will be performed. The first focuses on collecting EEG data while participants listen to musical excerpts and select them for future listening. Machine learning methods will be used to predict the listeners' decisions using features of the time-varying neural response recorded prior to the choice being made. The purpose of the second user study is to collect data for predict modelling of choices from features of the music itself and attributes of the listener. This will involve a larger range of musical excerpts and a wider range of listeners than is practical for the EEG study. The objective is to understand the psychological processes involved in musical choice and to use this knowledge to refine, parameterise and optimise the computational models of musical choice.The final stage of the research will develop an integrated predictive model of musical choice by combining predictive models using the musical signal with those making predictions from the neural signal. The development of such an integrated model is highly innovative. The advent within the last two years of affordable, multi-channel, wireless EEG headsets for the consumer market makes the possibility of using these devices to control media players and other interactive systems a real possibility. Therefore, the time is ripe to combine research in neuroscience, music cognition and machine learning, reflecting the unique interdisciplinary expertise of the PI, to understand the mapping between neural signals, musical structure and song selections.

近些年来，音乐消费发生了戏剧性的转变，从庞大的音乐库向流媒体播放转变，使用户不堪重负，拥有大量可能的音乐选择。这种新的格局使得开发智能工具来帮助听众选择要听的音乐势在必行。音乐技术的研究传统上一直遵循纯粹的工程方法，这在一定程度上推动了该领域的发展。然而，由于缺乏强大的、有科学依据的听众模型，可以用来向数字音乐播放器(如iTunes、Spotify、Last FM)告知用户选择音乐的偏好，这一进展受到了阻碍。拟议的研究通过开发创建计算模型所需的科学知识来解决这一差距，该模型可以根据使用脑电(EEG)记录的音乐特征和脑电反应来预测听众的音乐选择。主要的想法是对听众选择要听的音乐时涉及的心理和神经过程进行科学的理解。这个假设是，结合心理学原理、音乐特征和从头皮记录下来的脑电反应，可以准确地预测听者的音乐选择。这项研究有两个基础：第一，对听众进行研究，以确定这些心理原理、音乐特征和大脑反应；第二，利用这些知识建立一个计算模型，预测听众对音乐的选择。建模方法包括三个部分，以捕捉对音乐选择产生影响的音乐特征。第一个分量使用使用信号处理方法从音频中提取的声学特征，例如不协调性和时间规律性。第二部分采用更高层次的认知方法，使用基于音符级别的音乐表征的信息论模型来提取复杂性度量。第三部分从对抒情内容的情感文本分析中提取音乐的情感意图。要了解这些特征的确切性质和权重，以及它们如何影响音乐选择，需要详细检查听者在听音乐时做出的选择。因此，调查实际听者的行为是这项研究的核心，将进行两项研究。第一个重点是在参与者收听音乐摘录时收集脑电数据，并选择它们供未来收听。机器学习方法将被用来使用在做出选择之前记录的时变神经反应的特征来预测听者的决定。第二个用户研究的目的是收集数据，以便从音乐本身的特征和收听者的属性中预测选择的建模。这将涉及更大范围的音乐摘录和更广泛的听众，而不是EEG研究的实际情况。研究的目的是了解音乐选择所涉及的心理过程，并利用这些知识来提炼、参数化和优化音乐选择的计算模型。研究的最后阶段将通过结合使用音乐信号的预测模型和根据神经信号进行预测的模型来开发一个综合的音乐选择预测模型。这种一体化模式的发展具有很强的创新性。近两年来，面向消费者市场的价格实惠的多通道无线EEG头戴式耳机的问世，使使用这些设备控制媒体播放器和其他交互系统成为可能。因此，将神经科学、音乐认知和机器学习的研究结合起来，反映PI独特的跨学科专业知识，以了解神经信号、音乐结构和歌曲选择之间的映射关系的时机已经成熟。

项目成果

Compression-based Modelling of Musical Similarity Perception

基于压缩的音乐相似性感知建模

• DOI: 10.1080/09298215.2017.1305419
• 发表时间: 2017
• 期刊: Journal of New Music Research
• 影响因子: 1.1
• 作者: Pearce M

Simulating melodic and harmonic expectations for tonal cadences using probabilistic models

使用概率模型模拟音调节奏的旋律和和声期望

• DOI: 10.1080/09298215.2017.1367010
• 发表时间: 2017
• 期刊: Journal of New Music Research
• 影响因子: 1.1
• 作者: Sears D

QJE-STD-18-028.R2-SupplementaryMaterial - Supplemental material for Expectations for tonal cadences: Sensory and cognitive priming effects

QJE-STD-18-028.R2-补充材料 - 音调节奏期望的补充材料：感觉和认知启动效应

• DOI: 10.25384/sage.7436246
• 发表时间: 2018
• 作者: Sears D

Expectations for tonal cadences: Sensory and cognitive priming effects.

对音调节奏的期望：感觉和认知启动效应。

• DOI: 10.1177/1747021818814472
• 发表时间: 2019
• 期刊: Quarterly journal of experimental psychology (2006)
• 作者: Sears DR

Statistical learning and probabilistic prediction in music cognition: mechanisms of stylistic enculturation.

• DOI: 10.1111/nyas.13654
• 发表时间: 2018-05-11
• 期刊: Annals of the New York Academy of Sciences
• 影响因子: 5.2
• 作者: Pearce MT