Automatic Prediction of Impressions in Time and across Varying Context: Personality, Attractiveness and Likeability
Automatic Prediction of Impressions in Time and across Varying Context: Personality, Attractiveness and Likeability
复制标题
DOI：
10.1109/taffc.2015.2513401
复制
发表时间：
2017
影响因子：
11.2
通讯作者：
Oya Celiktutan;H. Gunes
中科院分区：
文献类型：
--
作者：
Oya Celiktutan;H. Gunes
在本文中，我们提出了一种新的多模态框架，自动预测的印象，外向，宜人性，热情，神经质，开放性，吸引力和可爱性，不断在时间和不同的情景背景。从现有的作品中，我们第一次在文献中连续获得同一组主题的仅视觉和仅音频注释，并将其与视听注释进行比较。我们提出了一种时间连续的预测方法，学习的时间关系，而不是单独对待每个时刻。我们的实验表明，当回归模型从视听注释和视觉线索，以及从视听注释和视觉线索结合音频线索在决策水平上学习时，获得了最好的预测结果。连续生成的注释有可能提供更好地理解哪些印象可以更动态地形成和预测，随情境上下文而变化，以及哪些印象随着时间的推移看起来更静态和稳定。
In this paper, we propose a novel multimodal framework for automatically predicting the impressions of extroversion, agreeableness, conscientiousness, neuroticism , openness, attractiveness and likeability continuously in time and across varying situational contexts. Differently from the existing works, we obtain visual-only and audio-only annotations continuously in time for the same set of subjects, for the first time in the literature, and compare them to their audio-visual annotations. We propose a time-continuous prediction approach that learns the temporal relationships rather than treating each time instant separately. Our experiments show that the best prediction results are obtained when regression models are learned from audio-visual annotations and visual cues, and from audio-visual annotations and visual cues combined with audio cues at the decision level. Continuously generated annotations have the potential to provide insight into better understanding which impressions can be formed and predicted more dynamically, varying with situational context, and which ones appear to be more static and stable over time.