Understanding human intention and activities for versatile real-time human-machine-interactions

点击翻译按钮获取中文摘要

In the 21st century, our personal and social activities are conducted in two different domains: physical real world and cyber network society. For realizing smooth and casual integration of these domains, we need a novel interaction model that goes a step further beyond the existing "command-and-response model."In this research, we proposed a concept of "man-machine symbiotic systems" as the next generation interaction model. Man-machine symbiotic systems are characterized by such a feature that the systems work for humans even if they are not explicitly ordered. For realizing such a symbiotic system, it is required to design real-time bi-lateral multimodal information processing mechanisms: observing human behaviors to understand their activities and intentions, and supporting them voluntarily by presenting appropriate information at an appropriate timing.To realize man-machine symbiotic systems, we conducted the following fundamental researches and obtained a number of important results including theoretical models and real world interaction systems:1. Observing Human Activities (recognition system): We developed a camera-network system and wearable active vision sensor systems for understanding human intentions and activities based on realtime 3D motion measurement.2. Attracting Human (generation system): We developed an intelligent desk system, which presents versatile information on a desk top plate according to two-handed manipulations of projected objects on the plate.3. Interacting with Human (interaction model): We proposed a dynamic event representation model which we refer to as a "hybrid dynamical system". We verified the effectiveness of the model by applying it to classify fine-grained facial expression, and to analyze the synchronization mechanism of mouth motion and speech utterance. We also explained a basic principle for realizing smooth interaction through an analysis of Japanese stand-up comedy (manzai).