Understanding Sports Video

这一建议解决了在体育视频的限制下理解视频中的群体活动的挑战。视频可以来自一个或多个静态或移动摄像机，并且可以伴随着评论、注释或简单的音频，例如实况转播。我们系统的结果将通过帮助球员评估和确定球队在各种情况下的运作方式来帮助教练。更重要的是，球迷们需要更多的在线比赛信息和统计摘要。为了支持这些需求，我们将探索可定制的交互和可视化。我们的目标是从视频中生成信息来满足所有这些需求。视频中的活动理解具有广泛的兴趣，从解释无约束事件的视频到非常集中的视频。从视频中可以提取很多信息：物体或人的位置，人的姿势，以及物体或人的运动。协调的动作表明有目的的行动，并引起与物体和人的相互作用。构成活动的动作集合。我们研究体育运动，是因为每项运动都有一个有限的动作集，在游戏规则下有特定的角色，群体活动有明确的目的。我们使用来自广播摄像机的视频，这样业余爱好者就可以利用我们的进步。我们将改进我们追踪和识别玩家的能力，描述他们的姿势和动作。许多这些进步将依赖于深度学习和循环神经网络来捕捉广泛的行动和活动。挑战在于如何表现游戏的结构和必要的信息，如玩家的外表、风格和角色、物理约束和游戏语义。我们将从多个视角整合事件描述，并将继续我们的摄像机规划工作，以便多个摄像机可以从不同的视角和视角（特写）跟随球员的运动。我们需要更丰富的游戏语义表示——人类摄像机操作员正在讲述一个需要关注突出游戏事件的故事，因此特写镜头取决于游戏的演变。我们还打算探索文本输入，甚至是比赛解说，当然也会探索文本生成——为教练和球迷提供报告，因为计算机视觉正在迅速扩展其用叙事注释图像和视频的能力。我们设计的方法将从体育推广到无限制的视频。

This proposal addresses the challenge of understanding group activity in videos, within the constraints of sports videos. The video can come from one or more static or moving cameras, and can be accompanied by commentary, annotation, or simply audio such as the play-by-play. The results of our system will aid coaches by helping in player evaluation and by identifying how teams operate in a variety of situations. More importantly, fans demand more game information online and in statistical summaries. To support these needs we will explore customizable interaction and visualization. Our goal is generate information to fill all these needs from the video.Activity understanding in video is of broad interest, ranging from interpreting videos of unconstrained events to very focused videos. Much information can be extracted from the video: the position of objects or persons, the pose of persons, as well as motion of objects or people. Coordinated motions suggest purposive actions and cause interactions with objects and people. Collections of actions form activities.We study sports because each sport has a limited action set with specific roles under the rules of the game and group activity has explicit purposes. We use videos from broadcast cameras so that amateurs can take advantage of our advances. We will refine our abilities to track and identify players, describe their pose and their actions. Many of these advances will rely on deep learning and recurrent neural networks to capture the broad range of actions and activities. The challenge is how to represent the structure of the game and the necessary information such as player appearance, styles, and roles, physical constraints and game semantics.We will integrate event descriptions from several viewpoints and will continue our work in camera planning so that several cameras can follow the movements of the players from different viewpoints and ranges of view (closeups). We need richer representations of game semantics - the human camera operator is telling a story requiring attention to salient game events, hence closeup shots depend on game evolution. We intend also to explore textual input, even game commentary, and naturally will explore text generation - reports for coaches and fans, as computer vision is rapidly expanding its ability to annotate images and videos with narratives. The methods we devise will generalize beyond sports to unrestricted videos.