Research of Emotion and Intention Understanding Based on Human Action Analysis and Its Application to Man-machine Communication

为了实现机器与人类的共生，我们研究了一种基于数据挖掘、图像识别、语音识别和情感信息处理的人体动作习惯和内在情感的通信方法。1运动数据的规则发现和情感质量分析(Uehara)研究当大量的运动数据像数字档案一样存储时，如果不根据动作的内容来组织或结构化人类的动作数据，就很难检索到运动数据。为了解决这一问题，将运动数据表示为三维空间中的运动轨迹，并对运动速度和姿态进行分析。通过这一分析，我们对动作技术进行了比较，发现了高技术程度的特点。利用动作捕捉系统EVA采集的动作数据，我们提取了各个动作的内在特征以及与其他动作的区别特征。2意图理解通过整合人类动作、情感、语音和图像识别(Ariki，Kumano)通过研究动作、语音、图像和情感的整合，可以分析意图是如何通过动作、语音、图像和情感来表达的。从这一点出发，我们特别分析了基于动作和情感的意图。例如，当用户通过说话和指着房间里像教室一样的大屏幕上显示的他的脸来询问他是谁时，系统集成了语音识别、指向动作和人脸识别。我们还对情绪进行了分析，以估计意图。

In order to realize symbiotic machines with human beings, we studied a communication method focusing on body action habits and the inside emotion, based on data mining, image recognition, speech recognition and emotion information processing.1 Studies on rule discovery from motion data and quality analysis of emotion (Uehara)It is difficult to retrieve the motion data unless the human action is organized or structured in terms of their contents when the large amount of motion data is stored like digital archives. To solve this problem, motion data is presented as motion locus in the three-dimensional space, and the motion speed and the posture are analyzed. Through this analysis, we compared the motion skills and discovered the features of high degree of skills. Using the action data captured by the motion capture system Eva, we extracted the intrinsic features of the respective motion and the distinguished features compared with other motions.2 Intention understanding by integrating human action, emotion, speech and image recognition (Ariki, Kumano)It is possible to analyze intentions in terms of how it is expressed with action, speech, image and emotion through the studies of action, speech, image and emotion integration. From this point, we especially analyzed the intention based on the action and emotion. For example, when a user asks by speaking who is he and by pointing his face displayed on the large screen in the room like a classroom, the system integrates the speech recognition, pointing action and face recognition. We also analyzed the emotion to estimate the intention.