Object Recognition System through Multimodal Interaction with Human Users and Environment

我们研究了服务机器人的多模态界面和目标识别系统，为人类用户提供所需的对象。在机器人与用户的界面上，采用了手势识别技术。提出了一种通过检测和分析人的手或手指的运动来获取有用和必要信息的方法。在物体识别方面，机器人用手臂接触或轻拍周围的物体。由于机器人和物体之间的物理相互作用，视觉信息发生了变化。利用视觉变化信息，提高对日常生活中存在的某一物体的识别精度。

We do research on multimodal interface and object recognition system for a service robot that brings a requested object to a human user. Concerning the interface between the robot and the user, gesture recognition is used. The method to obtain useful and necessary information by detecting and analyzing the motion of human hands or fingers is proposed. As for the object recognition, the robot with an arm contacts or taps objects in the surroundings. The visual information changes as a result of physical interaction between the robot and objects. By using visual change information, the accuracy of identifying a certain object existing in a daily life is improved.