Learning about Activities from Video

想象一下，一个系统可以在网络上搜索包含与已经突出显示的活动相似的视频片段（例如，停车或两个人正在交谈）；它可以在观察到其他人做同样的事情后参与纸牌游戏；它可以探测到有人在停车场从事不熟悉的活动。此外，假设它可以在没有关于所描述的特定对象和活动的先验知识的情况下完成所有这些事情。所有这些功能都可以在视频剪辑中查找类似或类似的活动。在过去的四十年里，人们做了大量的工作来设计在图片和视频片段中寻找物体和活动的方法，通过手工制作计算机模型来预测它们的样子。现在已经找到了通过从大量图片和视频剪辑中学习，完全自动化创建对象（例如行人）和简单动作（例如跑步）模型的方法。因此，这应该可以在没有先验知识的情况下搜索类似的物体和运动。最近在扩展这种自动化水平以处理非常简单场景中的有限范围的更复杂的活动方面取得了一些进展。这是通过首先学习物体的外观，然后使用逻辑归纳法学习它们所涉及的活动来实现的。不幸的是，目前还没有一种简单的方法来确保生成的对象类别适合要学习的活动。我们的主要目标是通过将对象类别的搜索转向那些导致最连贯的活动集来解决这个问题。这可能会改变我们对计算机视觉和逻辑推理中老问题的思考方式。

Imagine a system that could search the web for video clips containing an activity similar to one already highlighted (e.g. a car parking or two people having a conversation); that could participate in a card game after observing others do the same; and that could detect someone involved in an unfamiliar activity in a car park. Furthermore, suppose that it could do all of these things with no prior knowledge about the specific objects and activities depicted. All of these capabilities can be couched in terms of looking for similar or analogous activities in video clips.A lot of work has been done over the past forty years on devising methods for finding objects and activities in pictures and video clips by hand-crafting computer-models of what they are expected to look like. Ways have now been found to fully automate the creation of such models for objects (e.g. pedestrians) and simple movements (e.g. running) by learning from large sets of pictures and video clips. This should therefore make it possible to search for similar objects and movements with no prior knowledge of those things.Some progress has been made recently on extending this level of automation to handle a limited range of more complex activities in very simple scenes. This has been achieved by firstly learning about the appearance of objects and then learning about the activities in which they are involved using logical induction. Unfortunately there isn't yet an easy way to ensure the object categories produced are appropriate for the activities to be learnt. Our main aim is to resolve this problem by steering the search for object categories towards those that lead to the most coherent set of activities. A consequence of this could be to change the way we think about age-old problems in computer vision and logical reasoning.