Human Interface that detects user's intentions and emotions from their gestures and facial expressions

人类和机器之间的传统交互主要通过键盘或特殊的指向设备来执行。在这个项目中，我们研究了分析用户手势和面部表情并识别他们的意图和情绪的人机界面。本课题的主要研究内容如下：1.三维面部运动测量系统我们开发了一种测量三维运动面部形状的方法。该系统使用两个光源和一个狭缝图案投影仪。自然的面部运动可以输入到系统的采样率高达2/15秒。2.人类检测我们已经研究了人类检测技术，系统知道用户的存在。提出了一种基于边缘向量和边缘可靠性测度的运动目标检测技术和一种基于颜色索引的人体检测技术.跟踪和识别面部表情我们已经研究了用于跟踪和识别用户面部表情的技术。该技术是基于面部肌肉的位置和方向的解剖学知识，使系统可以从自然面部运动输入图像特征.手势分析手和手指在指示物体时起着重要作用。我们已经开发了一个立体的方法来跟踪三维运动的手，和一个方法来识别哪些手指是伸展。

Conventional interactions between humans and machines are performed mainly by keyboards or special pointing devices. In this project, we have investigated human interface that analyzes user's gestures and facial expressions and identifies their intentions and emotions. The main subjects of this projects are as follows :1. 3D facial motion measuring systemWe have developed a method for measuring three- dimensional moving facial shapes. The system uses two light sources and a slit pattern projector. Natural facial motions can be entered to the system with the sampling rate up to 2/15 sec.2. Human detectionWe have investigated human detection techniques for the system to know the presence of the users. We have presented a moving objects detection technique using edge vectors and edge reliability measures, and a human detection technique using color indexing.3. Tracking and recognizing facial expressionsWe have investigated techniques for tracking and recognizing user's facial expressions. The technique is based on anatomical knowledge on locations and orientations of facial muscles, so that the system can enter the image features from natural facial motions.4. Gesture analysisHands and fingers play an important role in indicating an object. We have developed a stereo method for tracking three-dimensional motions of a hand, and a method for identifying which fingers are stretched.