Places and Activities: Video and Robots

我的研究小组一直在研究两个相关的主题。 首先，我们开发了视觉引导的移动的机器人，目标是创造一个家庭助理机器人，例如，一个智能轮椅，可以导航，找到物体，并了解它的环境。 其次，我们解决了理解视频，特别是体育视频，使系统能够解释游戏，报告动作，并为观众提供游戏信息。 用机器人搜索物体需要控制机器人以获取场景的多个视图，其中距离传感器联合收割机与相机组合以提供用于物体识别和分类的信息。最近的测距相机（例如，Kinect）使我们能够对遮挡和混乱进行推理。 找到对象后，我们可以确定一个地方，一个活动的地点。同样，知道一个地方告诉我们哪些对象是可以期待的。了解活动可以指导机器人的行动。在各个层次上有丰富的语义相互作用，可以在场景和动作理解的概率方法中加以利用。 我们一直在寻求理解体育视频的方法，这为解决语义简单、活动类型丰富、对象少、演员有限的多个层次提供了机会。 我们将扩展我们在跟踪，玩家识别和图像校正方面的工作，以增强我们的条件随机场模型，使其具有更丰富的语义，基于位置，角色，活动以及游戏和策略的结构。我们可以通过捕捉局部时空结构的新功能来改进视频分析，为运动和动作提供强有力的线索。最近的工作跟踪篮球，例如，可以利用关于球员位置、姿势和运动的信息来理解球的位置和运动。 辅助技术，例如，立体导向轮椅，是一种传感和决策的应用。活动识别通过提供关于背景和活动的信息，提供了超越简单几何占用图的情况感知。了解运动类型的家用或辅助机器人可以安全有效地运行。

My research group has been working on two related topics. First, we developed visually guided mobile robots with the goal of creating a home assistant robot, e.g., an intelligent wheelchair, that can navigate, find objects, and understand its environment. Second, we addressed understanding videos, in particular sports video, to enable a system to interpret the games, report on the actions, and provide viewers access to game information. Searching for objects with a robot entails controlling the robot to acquire multiple views of the scene, where range sensors combine with cameras to provide information for object recognition and categorization. Recently range cameras (e.g., Kinect) enable us to reason about occlusion and clutter. Having found objects we can identify a place, a site for an activity. Likewise, knowing a place tells us which objects to expect. Knowing activities can guide robot action. There is a rich semantic interplay at various levels that can be exploited in probabilistic methods for scene and action understanding. We have pursued methods for understanding sports video, which offers an opportunity to address multiple levels that are semantically simple, rich in activity types, with few objects, with a limited cast of actors. We will extend our work in tracking, player identification, and image rectification to augment our conditional random fields models with richer semantics, based on position, roles, activities, and the structure of plays and strategies. We can improve analysis of video through new features that capture local spatio-temporal structure, providing strong cues for motion and actions. Recent work on tracking the basketball, e.g., can leverage information about player position, pose, and motion for understanding ball position and movement. Assistive technology, e.g., stereo-guided wheelchairs, is an application of sensing and decision making. Activity recognition provide situation awareness, by giving information about contexts and activities, beyond simple geometric occupancy maps. A home or assistive robot that understands movement types can operate safely and effectively.