Research of Emotion and Intention Understanding Based on Human Action Analysis and Its Application to Man-machine Communication

基于人类行为分析的情感和意图理解研究及其在人机交流中的应用

• 批准号: 14208037
• 负责人: ARIKI Yasuo
• 金额: $ 32.36万
• 依托单位: Kobe University (2003-2004)Ryukoku University (2002)
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (A)
• 财政年份: 2002
• 资助国家: 日本
• 起止时间: 2002 至 2004
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-14208037/
• 关键词: Data mining; Image recognition; Speech recognition; Emotion; Communication; Action analysis; Intention understanding; Motion capture; 感情情報処理

In order to realize symbiotic machines with human beings, we studied a communication method focusing on body action habits and the inside emotion, based on data mining, image recognition, speech recognition and emotion information processing.1 Studies on rule discovery from motion data and quality analysis of emotion (Uehara)It is difficult to retrieve the motion data unless the human action is organized or structured in terms of their contents when the large amount of motion data is stored like digital archives. To solve this problem, motion data is presented as motion locus in the three-dimensional space, and the motion speed and the posture are analyzed. Through this analysis, we compared the motion skills and discovered the features of high degree of skills. Using the action data captured by the motion capture system Eva, we extracted the intrinsic features of the respective motion and the distinguished features compared with other motions.2 Intention understanding by integrating human action, emotion, speech and image recognition (Ariki, Kumano)It is possible to analyze intentions in terms of how it is expressed with action, speech, image and emotion through the studies of action, speech, image and emotion integration. From this point, we especially analyzed the intention based on the action and emotion. For example, when a user asks by speaking who is he and by pointing his face displayed on the large screen in the room like a classroom, the system integrates the speech recognition, pointing action and face recognition. We also analyzed the emotion to estimate the intention.

为了实现机器与人类的共生，我们研究了一种基于数据挖掘、图像识别、语音识别和情感信息处理的人体动作习惯和内在情感的通信方法。1运动数据的规则发现和情感质量分析(Uehara)研究当大量的运动数据像数字档案一样存储时，如果不根据动作的内容来组织或结构化人类的动作数据，就很难检索到运动数据。为了解决这一问题，将运动数据表示为三维空间中的运动轨迹，并对运动速度和姿态进行分析。通过这一分析，我们对动作技术进行了比较，发现了高技术程度的特点。利用动作捕捉系统EVA采集的动作数据，我们提取了各个动作的内在特征以及与其他动作的区别特征。2意图理解通过整合人类动作、情感、语音和图像识别(Ariki，Kumano)通过研究动作、语音、图像和情感的整合，可以分析意图是如何通过动作、语音、图像和情感来表达的。从这一点出发，我们特别分析了基于动作和情感的意图。例如，当用户通过说话和指着房间里像教室一样的大屏幕上显示的他的脸来询问他是谁时，系统集成了语音识别、指向动作和人脸识别。我们还对情绪进行了分析，以估计意图。

项目成果

Masayuki Nakamura: "Improvement of Boosting Algorithm by Modifying the Weighting Rule"Annals of Mathematics and Artificial Intelligence, Kluwer Academic Publishers. (未定). (2004)

Masayuki Nakamura：“通过修改加权规则改进增强算法”，《数学与人工智能年鉴》，Kluwer 学术出版社（待定）。

Improvement of Boosting Algorithm by Modifying the Weighting Rule

• DOI: 10.1023/b:amai.0000018577.32783.d2
• 发表时间: 2004-05
• 期刊: Annals of Mathematics and Artificial Intelligence
• 影响因子: 1.2
• 作者: Masayuki Nakamura;Hiroki Nomiya;K. Uehara

Combination of GMM Based Speech Estimation Method and Temporal Domain SVD Based Speech Enhancement for Noise Robust Speech Recognition

基于 GMM 的语音估计方法和基于时域 SVD 的语音增强相结合用于噪声鲁棒语音识别

• 发表时间: 2005
• 期刊: IEICE Transactions on Information and Systems Vol.J88-D-II, No.2
• 作者: M.Harada;T.Nagao ほか全9名;Masakiyo Fujimoto
• 通讯作者: Masakiyo Fujimoto

STRUCTURING BASEBALL LIVE GAMES BASED ON SPEECH RECOGNITION USINGTASK DEPENDENT KNOWLEDGE AND EMOTION STATE RECOGNITION

使用任务相关知识和情绪状态识别基于语音识别构建棒球现场比赛

• 发表时间: 2005
• 期刊: International Conference on Acoustics, Speech and Signal Processing (ICASSP2005)
• 作者: C.Matsuoka;K.Nishihara;Masahito Kumano;Atsushi Sako
• 通讯作者: Atsushi Sako

Automatic Extraction of PC Scenes Based on Feature Mining for a Real Time Delivery System of Baseball Highlight Scenes

基于特征挖掘的棒球精彩场景实时传送系统的PC场景自动提取

• 发表时间: 2004
• 期刊: Proc.of ICME2004(International Conference on Multimedia and Expo) (CD-ROM)
• 作者: M.Kumano