Understanding Sports Video

该提案解决了在体育视频的限制范围内理解视频中的群体活动的挑战。视频可以来自一个或多个静态或移动摄像机，并且可以伴随有评论、注释或简单的音频，例如逐场播放。 我们的系统的结果将通过帮助球员评估和确定球队在各种情况下如何运作来帮助教练。 更重要的是，球迷需要更多的游戏信息在线和统计摘要。为了支持这些需求，我们将探索可定制的交互和可视化。我们的目标是从视频中生成满足所有这些需求的信息。视频中的活动理解具有广泛的兴趣，范围从解释不受约束的事件的视频到非常集中的视频。可以从视频中提取许多信息：物体或人的位置，人的姿势，以及物体或人的运动。协调的运动表明有目的的行动，并引起与物体和人的互动。 行动的集合形成活动。**我们研究体育运动，因为每项运动都有一个有限的行动集，在游戏规则下有特定的角色，团体活动有明确的目的。我们使用广播摄像机的视频，以便业余爱好者可以利用我们的进步。 ** 我们将完善我们的能力，以跟踪和识别球员，描述他们的姿势和他们的行动。其中许多进步将依赖于深度学习和循环神经网络来捕捉广泛的动作和活动。 挑战在于如何表示游戏的结构和必要的信息，如玩家外观，风格和角色，物理约束和游戏语义。我们将从多个视角整合事件描述，并将继续我们在摄像机规划方面的工作，以便多个摄像机可以从不同的视角和视野范围（特写镜头）跟踪玩家的运动。 我们需要更丰富的游戏语义表示-人类摄像机操作员正在讲述一个需要关注突出游戏事件的故事，因此特写镜头取决于游戏的演变。 我们还打算探索文本输入，甚至是比赛评论，当然也会探索文本生成-为教练和球迷提供报告，因为计算机视觉正在迅速扩展其用叙述注释图像和视频的能力。 我们设计的方法将超越体育推广到不受限制的视频。

This proposal addresses the challenge of understanding group activity in videos, within the constraints of sports videos. The video can come from one or more static or moving cameras, and can be accompanied by commentary, annotation, or simply audio such as the play-by-play. The results of our system will aid coaches by helping in player evaluation and by identifying how teams operate in a variety of situations. More importantly, fans demand more game information online and in statistical summaries. To support these needs we will explore customizable interaction and visualization. Our goal is generate information to fill all these needs from the video.******Activity understanding in video is of broad interest, ranging from interpreting videos of unconstrained events to very focused videos. Much information can be extracted from the video: the position of objects or persons, the pose of persons, as well as motion of objects or people. Coordinated motions suggest purposive actions and cause interactions with objects and people. Collections of actions form activities.******We study sports because each sport has a limited action set with specific roles under the rules of the game and group activity has explicit purposes. We use videos from broadcast cameras so that amateurs can take advantage of our advances. ******We will refine our abilities to track and identify players, describe their pose and their actions. Many of these advances will rely on deep learning and recurrent neural networks to capture the broad range of actions and activities. The challenge is how to represent the structure of the game and the necessary information such as player appearance, styles, and roles, physical constraints and game semantics.******We will integrate event descriptions from several viewpoints and will continue our work in camera planning so that several cameras can follow the movements of the players from different viewpoints and ranges of view (closeups). We need richer representations of game semantics - the human camera operator is telling a story requiring attention to salient game events, hence closeup shots depend on game evolution. We intend also to explore textual input, even game commentary, and naturally will explore text generation - reports for coaches and fans, as computer vision is rapidly expanding its ability to annotate images and videos with narratives. The methods we devise will generalize beyond sports to unrestricted videos.