Understanding Sports Video

这一建议解决了在体育视频的限制下理解视频中的群体活动的挑战。视频可以来自一个或多个静态或移动摄像机，并且可以伴随着评论、注释或简单的音频，例如实况转播。我们系统的结果将通过帮助球员评估和确定球队在各种情况下的运作方式来帮助教练。更重要的是，球迷们需要更多的在线比赛信息和统计摘要。为了支持这些需求，我们将探索可定制的交互和可视化。我们的目标是从视频中生成信息来满足所有这些需求。

This proposal addresses the challenge of understanding group activity in videos, within the constraints of sports videos. The video can come from one or more static or moving cameras, and can be accompanied by commentary, annotation, or simply audio such as the play-by-play. The results of our system will aid coaches by helping in player evaluation and by identifying how teams operate in a variety of situations. More importantly, fans demand more game information online and in statistical summaries. To support these needs we will explore customizable interaction and visualization. Our goal is generate information to fill all these needs from the video. Activity understanding in video is of broad interest, ranging from interpreting videos of unconstrained events to very focused videos. Much information can be extracted from the video: the position of objects or persons, the pose of persons, as well as motion of objects or people. Coordinated motions suggest purposive actions and cause interactions with objects and people. Collections of actions form activities. We study sports because each sport has a limited action set with specific roles under the rules of the game and group activity has explicit purposes. We use videos from broadcast cameras so that amateurs can take advantage of our advances. We will refine our abilities to track and identify players, describe their pose and their actions. Many of these advances will rely on deep learning and recurrent neural networks to capture the broad range of actions and activities. The challenge is how to represent the structure of the game and the necessary information such as player appearance, styles, and roles, physical constraints and game semantics. We will integrate event descriptions from several viewpoints and will continue our work in camera planning so that several cameras can follow the movements of the players from different viewpoints and ranges of view (closeups). We need richer representations of game semantics - the human camera operator is telling a story requiring attention to salient game events, hence closeup shots depend on game evolution. We intend also to explore textual input, even game commentary, and naturally will explore text generation - reports for coaches and fans, as computer vision is rapidly expanding its ability to annotate images and videos with narratives. The methods we devise will generalize beyond sports to unrestricted videos.