Understanding Sports Video

了解体育视频

• 批准号: RGPIN-2018-04830
• 负责人: Little, James
• 金额: $ 3.5万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=657725
• 关键词: Understanding; Sports; Video

This proposal addresses the challenge of understanding group activity in videos, within the constraints of sports videos. The video can come from one or more static or moving cameras, and can be accompanied by commentary, annotation, or simply audio such as the play-by-play. The results of our system will aid coaches by helping in player evaluation and by identifying how teams operate in a variety of situations. More importantly, fans demand more game information online and in statistical summaries. To support these needs we will explore customizable interaction and visualization. Our goal is generate information to fill all these needs from the video.******Activity understanding in video is of broad interest, ranging from interpreting videos of unconstrained events to very focused videos. Much information can be extracted from the video: the position of objects or persons, the pose of persons, as well as motion of objects or people. Coordinated motions suggest purposive actions and cause interactions with objects and people. Collections of actions form activities.******We study sports because each sport has a limited action set with specific roles under the rules of the game and group activity has explicit purposes. We use videos from broadcast cameras so that amateurs can take advantage of our advances. ******We will refine our abilities to track and identify players, describe their pose and their actions. Many of these advances will rely on deep learning and recurrent neural networks to capture the broad range of actions and activities. The challenge is how to represent the structure of the game and the necessary information such as player appearance, styles, and roles, physical constraints and game semantics.******We will integrate event descriptions from several viewpoints and will continue our work in camera planning so that several cameras can follow the movements of the players from different viewpoints and ranges of view (closeups). We need richer representations of game semantics - the human camera operator is telling a story requiring attention to salient game events, hence closeup shots depend on game evolution. We intend also to explore textual input, even game commentary, and naturally will explore text generation - reports for coaches and fans, as computer vision is rapidly expanding its ability to annotate images and videos with narratives. The methods we devise will generalize beyond sports to unrestricted videos.

这项建议解决了在体育视频的限制下理解视频中的群体活动的挑战。视频可以来自一个或多个静态或移动的摄像机，并且可以伴随着解说、注释或简单的音频，例如逐个播放。我们系统的结果将通过帮助评估球员和确定球队在各种情况下的运作方式来帮助教练。更重要的是，球迷在网上和统计摘要中要求更多的比赛信息。为了支持这些需求，我们将探索可定制的交互和可视化。我们的目标是从视频中生成信息来满足所有这些需求。*视频中的活动理解具有广泛的兴趣，从解读不受约束的事件的视频到非常专注的视频。从视频中可以提取很多信息：物体或人的位置，人的姿势，以及物体或人的运动。协调动作指有目的的动作，并与物体和人产生互动。我们研究体育是因为每项运动都有一个有限的动作集，在游戏规则下有特定的角色，而集体活动有明确的目的。我们使用广播摄像机的视频，这样业余爱好者就可以利用我们的进步。*我们将完善跟踪和识别球员的能力，描述他们的姿势和动作。其中许多进展将依赖深度学习和递归神经网络来捕捉广泛的行动和活动。挑战是如何表现游戏的结构和必要的信息，如玩家的外表、风格和角色、物理约束和游戏语义。*我们将整合多个视角的事件描述，并将继续我们的相机规划工作，以便多个相机可以从不同的视角和范围(特写)跟踪玩家的移动。我们需要更丰富的游戏语义表示--人类摄像机操作员正在讲述一个需要关注重要游戏事件的故事，因此特写镜头取决于游戏的演变。我们还打算探索文本输入，甚至是比赛评论，并自然会探索为教练和球迷提供文本生成报告，因为计算机视觉正在迅速扩展其使用叙事注释图像和视频的能力。我们设计的方法将超越体育运动，推广到不受限制的视频。