CONSTRUCTION OF QUANTITATIVE ESTIMATION SYSTEM OF FACE EXPRESSION BASED ON 3D HUMAN HEAD STRUCTURAL MODEL

基于3D人体头部结构模型的人脸表情定量估计系统构建

• 批准号: 06555111
• 负责人: OOKURA Motohiro
• 金额: $ 7.74万
• 依托单位: SEIKEI UNIVERSITY
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (A)
• 财政年份: 1994
• 资助国家: 日本
• 起止时间: 1994 至 1996
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-06555111/
• 关键词: EXPRESSION SYNTHESIS; EMOTION SPACE; EXPRESSION RECOGNITION; EXPRESSION ESTIMATION; ACTION UNIT; TEXTURE MAPPING; NEURAL NETWORK; LIP SYNCHRONIZATION; コンピュータグラフィックス; 表情分析; 3次元モデル; モーションキャプチャ; コンピュータビジョン; ヒューマンコンピュータインタラクション; 感性情報処理; FACS

In this research, firstly a system for measuring 3d struction of face is constructed. In this system, markers are located on surface of the face and geometry is captured by 2 cameras located on front and side by tracking each marker. By using this system, quantitative description of Facial Action Coding System and feature of lip shape when utterance are realized. And this system make it possible to synthesized facial expression as it is by modifying 3d wireframe and texture mapping of frontal face image.Next, six basic expression are realized by combination of action unit and 3d emotion space is constructed by neural network for identity mapping. Specific facial expression is described by the point in 3d emotion space and this emotion space can realize both analysis and synthesis of facial expression. By estimation of this emotion space, expression is defined continuously in this space and it is approved this emotion space is good in psychological meanings.Finaly, automatic face expression recognition is realized without any markers. In this system, spatial frequency in mouth and eye region is calculated and converted to one of the emotion categories by neural network. This system is running in real time and recognition performance is similar to human subjective recognition.

在本研究中，首先建构一个脸部三维结构量测系统。在该系统中，标记位于面部的表面上，并且通过跟踪每个标记，由位于正面和侧面的2个相机捕获几何形状。该系统实现了面部动作编码系统的定量描述和说话时的唇形特征。该系统通过对人脸正面图像进行三维线框修改和纹理映射，实现了真实的面部表情合成;其次，通过动作单元的组合实现了六种基本表情，并利用神经网络构建了三维情感空间进行身份映射。在三维情感空间中用点来描述特定的面部表情，该情感空间可以实现面部表情的分析和合成。通过对该情感空间的估计，在该情感空间中对表情进行连续定义，并验证了该情感空间具有良好的心理学意义，最终实现了无标记的人脸表情自动识别。在该系统中，嘴部和眼睛区域的空间频率被计算，并通过神经网络转换为情感类别之一。该系统运行在真实的时间内，识别性能接近人的主观识别。

项目成果

坂口竜己、山田寛、森島繁生: "顔画像を基にした3次元感情モデルの構築とその評価" 電子情報通信学会論文誌A. J80-A. 1279-1284 (1997)

Tatsumi Sakaguchi、Hiroshi Yamada、Shigeo Morishima：“基于面部图像的三维情感模型的构建和评估”IEICE Transactions A. J80-A（1997）。

Shigeo Morishima: "Modeling of Facial Expression and Emotion for Human Communication System" International Journal of Display. 17. 15-25 (1996)

Shigeo Morishima：“人类交流系统的面部表情和情感建模”国际显示杂志。

Nobuo Ueki, Shigeo Morishima, Hiroshi Yamada, Hiroshi Harashima: "Expression Analysis/Synthesis System Based on Emotion Space Constructed by Multilayered Neural Network" Systems and Computers in Japan, Scripta Technica, INC.Vol.25, No.13. 95-107 (1995)

Nobuo Ueki、Shigeo Morishima、Hiroshi Yamada、Hiroshi Harashima：“基于多层神经网络构建的情感空间的表达分析/合成系统”日本系统和计算机，Scripta Technica，INC.Vol.25，No.13。

Fumio Kawakami, Shigeo Morishima, Hiroshi Yamada and Hiroshi Harashima: "Construction of 3-D Emotion Space using Neural Network" Proceedings of the 3rd International Conference on Fuzzy Logic, Neural Nets and Soft Computing. 309-310 (1994)

Fumio Kawakami、Shigeo Morishima、Hiroshi Yamada 和 Hiroshi Harashima：“利用神经网络构建 3-D 情感空间”第三届模糊逻辑、神经网络和软计算国际会议论文集。

Hiroshi Yamada et. al: "A Modeling of Facial Expression and Emotion for Recognition and Synthesis" Future Computing and Design for HCI. 251-256 (1995)

山田浩等。