Object Recognition System through Multimodal Interaction with Human Users and Environment

我们做多模态接口和对象识别系统的服务机器人，把一个请求的对象给人类用户的研究。关于机器人和用户之间的接口，使用手势识别。提出了通过检测和分析人手或手指的运动来获取有用和必要信息的方法。至于物体识别，具有手臂的机器人接触或轻敲周围的物体。由于机器人和物体之间的物理交互，视觉信息会发生变化。通过使用视觉变化信息，提高了识别日常生活中存在的特定对象的准确性。

We do research on multimodal interface and object recognition system for a service robot that brings a requested object to a human user. Concerning the interface between the robot and the user, gesture recognition is used. The method to obtain useful and necessary information by detecting and analyzing the motion of human hands or fingers is proposed. As for the object recognition, the robot with an arm contacts or taps objects in the surroundings. The visual information changes as a result of physical interaction between the robot and objects. By using visual change information, the accuracy of identifying a certain object existing in a daily life is improved.