Affective Computing Models: from Facial Expression to Mind-Reading

人类表现出广泛的情感和认知状态并进行交流。读心阅读使人类能够预测、模仿和解释其他动物能力之外的其他行为，尽管最近的研究表明猿类在错误信念任务中取得了成功，但这种说法仍然可以证明。因此，阅读是人类社会互动和交流的基础。在阅读中，最重要的标志之一是面部表情，因为它传达了反映心理状态的关键信息，并且与人们感知他人的感受和态度时的55%的信息有关。杜兴在1862年研究了电刺激个体面部肌肉的方法，10年后，达尔文发表了《人类和动物的情感表达》一书，为面部表情的共同祖先提出了理由。面部表情的研究已经引起了心理学、神经科学和计算机科学等不同学科的广泛关注。近年来，计算技术和大量在线面部图像/视频的发展推动了基于深度学习的面部表情识别（FER）的发展。目前，自动面部表情识别技术已经取得了很大的进展，从静态图像到动态视频分析，从表演表情到自发表情，从宏观表情到微观表情，但仍面临着以下挑战：1）大量的心理学研究成果支持将情绪评价理论应用于面部行为的内部情绪检测。与此同时，计算机科学的研究主要集中在外观或几何面部建模，而忽略了潜在的生物驱动机制; 2）来自不同文化的可用数据有限，阻碍了机器学习方法的研究发展; 3）微表情，快速（1/25至1/3秒），微妙的，不自觉的面部表情，难以控制通过一个人的意志力，不研究文化不一致;

Humans exhibit and communicate with a wide range of affective and cognitive states. Mind reading allows humans to predict, model, and interpret each other's behaviour beyond the capabilities of other animals, a claim that arguably can be made despite recent research suggesting apes being successful with false-belief tasks. Therefore, mind reading is fundamental to human social interaction and communication. In mind reading, one of the most important signs is facial expression, as it conveys critical information that reflects mental states and relates to 55% of information when people perceive others' feelings and attitudes. Since Duchenne5 studied the electro-stimulation of individual facial muscles in 1862 and ten years later, Darwin published "The Expression of the Emotions in Man and Animals", making a case for shared ancestry of facial expressions. Research on facial expressions has attracted a lot of attention from different disciplines such as psychology, neuroscience and computer science. In recent years, the development of computing technologies and massive online facial images/videos enabled the boosting of deep learning-based facial expression recognition (FER). To date, automatic FER has achieved excellent progress, from static image to dynamic video analysis, from acted/posed to spontaneous expressions, from macro-expressions to micro-expressions.In summary, the rising challenges include, 1) Substantial psychological works support the use of appraisal theories of emotion for internal emotion detection through facial behaviours. At the same time, research in computer science mainly focuses on appearance or geometric facial modelling but ignores the underlying biologically-driven mechanism; 2) There is limited available data from different cultures, hindering the research on machine learning method development; 3) Micro-expressions, rapid (1/25 to 1/3 second), subtle, and involuntary facial expressions that are difficult to control through one's willpower, is not studied for culture inconsistency;