Places and Activities: Video and Robots

点击翻译按钮获取中文摘要

My research group has been working on two related topics. First, we developed visually guided mobile robots with the goal of creating a home assistant robot, e.g., an intelligent wheelchair, that can navigate, find objects, and understand its environment. Second, we addressed understanding videos, in particular sports video, to enable a system to interpret the games, report on the actions, and provide viewers access to game information. Searching for objects with a robot entails controlling the robot to acquire multiple views of the scene, where range sensors combine with cameras to provide information for object recognition and categorization. Recently range cameras (e.g., Kinect) enable us to reason about occlusion and clutter. Having found objects we can identify a place, a site for an activity. Likewise, knowing a place tells us which objects to expect. Knowing activities can guide robot action. There is a rich semantic interplay at various levels that can be exploited in probabilistic methods for scene and action understanding. We have pursued methods for understanding sports video, which offers an opportunity to address multiple levels that are semantically simple, rich in activity types, with few objects, with a limited cast of actors. We will extend our work in tracking, player identification, and image rectification to augment our conditional random fields models with richer semantics, based on position, roles, activities, and the structure of plays and strategies. We can improve analysis of video through new features that capture local spatio-temporal structure, providing strong cues for motion and actions. Recent work on tracking the basketball, e.g., can leverage information about player position, pose, and motion for understanding ball position and movement. Assistive technology, e.g., stereo-guided wheelchairs, is an application of sensing and decision making. Activity recognition provide situation awareness, by giving information about contexts and activities, beyond simple geometric occupancy maps. A home or assistive robot that understands movement types can operate safely and effectively.