Understanding human intention and activities for versatile real-time human-machine-interactions

了解人类意图和活动以实现多功能实时人机交互

• 批准号: 13224051
• 负责人: MATSUYAMA Takashi
• 金额: $ 89.66万
• 依托单位: Kyoto University
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research on Priority Areas
• 财政年份: 2001
• 资助国家: 日本
• 起止时间: 2001 至 2005
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-13224051/
• 关键词: Man-machine symbiotic system; Human interface; Hybrid dynamical system; Interaction model; Understanding of human intentions and activities; Analysis of utterance pauses and overlaps in dialogs; Environment-embedded camera network; Wearable active

In the 21st century, our personal and social activities are conducted in two different domains: physical real world and cyber network society. For realizing smooth and casual integration of these domains, we need a novel interaction model that goes a step further beyond the existing "command-and-response model."In this research, we proposed a concept of "man-machine symbiotic systems" as the next generation interaction model. Man-machine symbiotic systems are characterized by such a feature that the systems work for humans even if they are not explicitly ordered. For realizing such a symbiotic system, it is required to design real-time bi-lateral multimodal information processing mechanisms: observing human behaviors to understand their activities and intentions, and supporting them voluntarily by presenting appropriate information at an appropriate timing.To realize man-machine symbiotic systems, we conducted the following fundamental researches and obtained a number of important results including theoretical models and real world interaction systems:1. Observing Human Activities (recognition system): We developed a camera-network system and wearable active vision sensor systems for understanding human intentions and activities based on realtime 3D motion measurement.2. Attracting Human (generation system): We developed an intelligent desk system, which presents versatile information on a desk top plate according to two-handed manipulations of projected objects on the plate.3. Interacting with Human (interaction model): We proposed a dynamic event representation model which we refer to as a "hybrid dynamical system". We verified the effectiveness of the model by applying it to classify fine-grained facial expression, and to analyze the synchronization mechanism of mouth motion and speech utterance. We also explained a basic principle for realizing smooth interaction through an analysis of Japanese stand-up comedy (manzai).

在21世纪，我们的个人和社会活动是在两个不同的领域进行的：物理现实世界和网络社会。为了实现这些领域的顺畅和随意的整合，我们需要一种新的交互模型，它超越了现有的“命令-响应模型”。在本研究中，我们提出了“人机共生系统”的概念，作为下一代交互模型。人机共生系统的特点是，即使没有明确的顺序，系统也可以为人类工作。为了实现这种共生系统，需要设计实时的双边多通道信息处理机制：观察人类的行为，了解他们的活动和意图，并在适当的时间提供适当的信息，自愿支持他们。为了实现人机共生系统，我们进行了以下基础研究，并获得了包括理论模型和现实世界交互系统在内的一系列重要成果：1.观察人类活动(识别系统)：我们开发了基于实时三维运动测量的摄像头-网络系统和可穿戴式主动视觉传感器系统，用于理解人类的意图和活动。吸引人(生成系统)：我们开发了一个智能桌面系统，它根据桌面上投影物体的双手操作在桌面上呈现各种信息。与人交互(交互模型)：我们提出了一个动态事件表示模型，我们称之为“混合动力系统”。通过应用该模型对细粒度人脸表情进行分类，并分析了嘴动与语音的同步机制，验证了该模型的有效性。我们还通过对日本单口喜剧(漫画)的分析，解释了实现流畅互动的基本原则。

项目成果

Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video

• DOI: 10.1016/j.cviu.2004.03.012
• 发表时间: 2004-12
• 期刊: Comput. Vis. Image Underst.
• 作者: T. Matsuyama;Xiaojun Wu;T. Takai;S. Nobuhara

Real-time cooperative multi-target tracking by communicating active vision agents

• DOI: 10.1109/icpr.2002.1048225
• 发表时间: 2002-12
• 期刊: Object recognition supported by user interaction for service robots
• 作者: N. Ukita;T. Matsuyama

3D Shape From Multi-viewpoint Images Using Deformable Mesh Model

使用可变形网格模型从多视点图像生成 3D 形状

• 发表时间: 2002
• 期刊: IPSJ Trans. on Computer Vision and Image Media Vol.43, No. SIG11(CVIM5)
• 作者: 岡部孝弘;岡部孝弘;Yasuto Nakanishi;Kenji Oka;Takahiro Okabe;Takahiro Okabe;杉本晃宏;川嶋宏彰;Kenji Oka;佐藤洋一;徳田泰久;浮田宗伯;延原章平;Takashi Matsuyama;Akihiro Sugimoto;Hiroaki Kawashima;Kenji Oka;Yoichi Sato;Yasuhisa Tokuda;Souhaku Ukita;Shohei Nobuhara
• 通讯作者: Shohei Nobuhara

Reflectance Estimation of a Moving Object under Complex Illumination

复杂光照下运动物体的反射率估计

• 发表时间: 2005
• 期刊: IEICE Trans. on Information and Systems Vol. J88-D-II, No.8
• 作者: 川嶋宏彰;Hiroaki Kawashima;Norimichi Ukita;Akihiro Sugimoto;杜菲;鈴木達也;岡兼司;池田友彦;Hiroaki Kawashima;Norimichi Ukita;Akihiro Sugimoto;Hiroaki Kawashima;Fei Du
• 通讯作者: Fei Du

Ego-Motion Estimation Based on Fixation Control Using Active Cameras

基于使用主动相机的注视控制的自我运动估计

• 发表时间: 2005
• 期刊: IEICE Trans. on Information and Systems Vol. J88-D-II, No.8
• 作者: 川嶋宏彰;Hiroaki Kawashima;Norimichi Ukita;Akihiro Sugimoto;杜菲;鈴木達也;岡兼司;池田友彦;Hiroaki Kawashima;Norimichi Ukita;Akihiro Sugimoto;Hiroaki Kawashima;Fei Du;Tatsuya Suzuki;Kenji Oka;Tomohiko Ikeda
• 通讯作者: Tomohiko Ikeda