Places and Activities: Video and Robots

我的研究小组一直在研究两个相关的课题。首先，我们开发了视觉引导移动机器人，目标是创造一种家庭助理机器人，例如智能轮椅，可以导航，寻找物体，并了解其环境。其次，我们解决了理解视频，特别是体育视频，使系统能够解释比赛，报告行动，并为观众提供访问比赛信息。用机器人搜索物体需要控制机器人获取场景的多个视图，其中距离传感器与相机相结合，为物体识别和分类提供信息。最近，距离相机（如Kinect）使我们能够推断遮挡和杂乱。找到目标后，我们就可以确定一个地方，一个活动地点。同样地，知道一个地方就能告诉我们会有哪些物体。了解活动可以指导机器人的行动。在不同层次上有丰富的语义相互作用，可以在场景和动作理解的概率方法中利用。我们一直在寻求理解体育视频的方法，这为解决语义简单、活动类型丰富、对象较少、演员阵容有限的多个层次提供了机会。我们将扩展我们在跟踪、球员识别和图像校正方面的工作，以更丰富的语义增强我们的条件随机场模型，基于位置、角色、活动以及比赛和策略的结构。我们可以通过捕捉局部时空结构的新特征来改进视频分析，为运动和动作提供强有力的线索。例如，最近在追踪篮球方面的工作可以利用球员的位置、姿势和运动信息来理解球的位置和运动。辅助技术，例如立体导向轮椅，是感知和决策的一种应用。活动识别通过提供有关背景和活动的信息，超越了简单的几何占用地图，从而提供态势感知。一个能够理解运动类型的家用或辅助机器人可以安全有效地操作。

My research group has been working on two related topics. First, we developed visually guided mobile robots with the goal of creating a home assistant robot, e.g., an intelligent wheelchair, that can navigate, find objects, and understand its environment. Second, we addressed understanding videos, in particular sports video, to enable a system to interpret the games, report on the actions, and provide viewers access to game information. Searching for objects with a robot entails controlling the robot to acquire multiple views of the scene, where range sensors combine with cameras to provide information for object recognition and categorization. Recently range cameras (e.g., Kinect) enable us to reason about occlusion and clutter. Having found objects we can identify a place, a site for an activity. Likewise, knowing a place tells us which objects to expect. Knowing activities can guide robot action. There is a rich semantic interplay at various levels that can be exploited in probabilistic methods for scene and action understanding. We have pursued methods for understanding sports video, which offers an opportunity to address multiple levels that are semantically simple, rich in activity types, with few objects, with a limited cast of actors. We will extend our work in tracking, player identification, and image rectification to augment our conditional random fields models with richer semantics, based on position, roles, activities, and the structure of plays and strategies. We can improve analysis of video through new features that capture local spatio-temporal structure, providing strong cues for motion and actions. Recent work on tracking the basketball, e.g., can leverage information about player position, pose, and motion for understanding ball position and movement. Assistive technology, e.g., stereo-guided wheelchairs, is an application of sensing and decision making. Activity recognition provide situation awareness, by giving information about contexts and activities, beyond simple geometric occupancy maps. A home or assistive robot that understands movement types can operate safely and effectively.