Places and Activities: Video and Robots

我的研究小组一直在研究两个相关的主题。首先，我们开发了视觉导引移动机器人，目标是创造一种家庭助理机器人，例如智能轮椅，可以导航、找到对象并了解其环境。其次，我们解决了理解视频，特别是体育视频，使系统能够解释比赛，报告行动，并为观众提供对比赛信息的访问。使用机器人搜索对象需要控制机器人获取场景的多个视图，其中距离传感器与摄像头相结合，为对象识别和分类提供信息。最近，远程摄像头(如Kinect)使我们能够对遮挡和杂乱进行推理。找到物体后，我们就可以确定一个地点，一个活动的地点。同样，了解一个地方会告诉我们应该期待哪些对象。了解活动可以指导机器人的行动。在不同的层次上有丰富的语义相互作用，可以在场景和动作理解的概率方法中加以利用。我们一直在寻求理解体育视频的方法，它提供了一个机会来解决语义简单、活动类型丰富、对象少、演员阵容有限的多个层次。我们将扩展我们在跟踪、玩家识别和图像纠正方面的工作，基于位置、角色、活动以及游戏和策略的结构，以更丰富的语义来增强我们的条件随机场模型。我们可以通过捕捉局部时空结构的新功能来改进视频分析，为运动和动作提供强大的线索。最近关于跟踪篮球的工作，例如，可以利用关于球员位置、姿势和运动的信息来理解球的位置和运动。辅助技术，如立体引导轮椅，是传感和决策的应用。活动识别通过提供有关上下文和活动的信息来提供态势感知，而不仅仅是简单的几何占用地图。了解运动类型的家用或辅助机器人可以安全有效地操作。

My research group has been working on two related topics. First, we developed visually guided mobile robots with the goal of creating a home assistant robot, e.g., an intelligent wheelchair, that can navigate, find objects, and understand its environment. Second, we addressed understanding videos, in particular sports video, to enable a system to interpret the games, report on the actions, and provide viewers access to game information. Searching for objects with a robot entails controlling the robot to acquire multiple views of the scene, where range sensors combine with cameras to provide information for object recognition and categorization. Recently range cameras (e.g., Kinect) enable us to reason about occlusion and clutter. Having found objects we can identify a place, a site for an activity. Likewise, knowing a place tells us which objects to expect. Knowing activities can guide robot action. There is a rich semantic interplay at various levels that can be exploited in probabilistic methods for scene and action understanding. We have pursued methods for understanding sports video, which offers an opportunity to address multiple levels that are semantically simple, rich in activity types, with few objects, with a limited cast of actors. We will extend our work in tracking, player identification, and image rectification to augment our conditional random fields models with richer semantics, based on position, roles, activities, and the structure of plays and strategies. We can improve analysis of video through new features that capture local spatio-temporal structure, providing strong cues for motion and actions. Recent work on tracking the basketball, e.g., can leverage information about player position, pose, and motion for understanding ball position and movement. Assistive technology, e.g., stereo-guided wheelchairs, is an application of sensing and decision making. Activity recognition provide situation awareness, by giving information about contexts and activities, beyond simple geometric occupancy maps. A home or assistive robot that understands movement types can operate safely and effectively.