Analyzing Hockey Broadcast Videos Using Machine Learning

世界各地的公共娱乐活动都以体育为主，尤其是在电视上。向公众提供的一个突出问题是，在比赛结束后，直到公众和球队可以获得分析和有用的评论时，才会有明显的延迟。此外，这些数据的汇编、总结和呈现在很大程度上是由观看电视上体育赛事视频的人完成的。因此，目前可用和有趣的数据量非常有限。潜在地，这些分析将成为体育产业的一个非常有利可图的收入来源，也是体育团队教练和经理的一个有价值的工具。显然，自动内容感知的体育视频分析在这方面将非常有用。最近（2018年5月）的一篇论文b[1]非常详细地调查了这一领域的活动。它发现“……超过80%的论文涉及棒球、篮球、足球和网球。它总共只列出了三篇关于冰球的论文。本应用的研究旨在改变这一现状。利用计算机视觉和人工智能，这项研究的动机是两个社会因素：1)体育作为娱乐的普及，2)体育数据的自动收集和分析严重缺乏。由于冰球对加拿大人的重要性，我们要讨论的是职业冰球。它解决并强调机器学习来“理解”和确定曲棍球分析。具体来说，重点是提供离线自动分析视频内容的能力，以检测和量化相关的时间和空间事件，在本例中，NHL播放的曲棍球比赛视频。我们将此称为内容感知曲棍球视频分析。本提案的一个独特的技术特征是，我们将使用现成的视频和口头数据（封闭字幕）来执行曲棍球比赛的内容分析。研究计划将集中于以下相关项目。项目#1：一个分析冰球广播视频的系统，主要使用图像、视频和语言描述视频中的物体、动作和事件（冰上发生了什么？）稍后，我们将讨论项目#2：基于#1，计算实用曲棍球分析的新方法，以支持教练和培训师。（从视频中可以收集到什么实际数据？）所提出的方法可称为智能多模态视频内容分析（IMVCA）。即离线自动分析和解释视频和语言内容的能力，目的是检测和量化特定的时空曲棍球事件。在我们的案例中，我们利用了冰球比赛的广播视频，但实际上，该方法可以应用于解释任何包含封闭字幕（CC）的视频。因此，它将成为推进视频分析方法的工具。

Public entertainment around the world is dominated by sports, particularly on television. A prominent issue with its delivery to the public is that there is a significant delay after a game until analytics and useful commentary are available to the public and the teams. Also, the compilation, summarization, and presentation of these data is largely done by humans watching videos of a sporting event on TV. Therefore, the amount of available and interesting data are currently very limited. Potentially, these analytics would be a very lucrative source of income for the sports industry, as well as a valuable tool for coaches and managers of sports teams. Clearly, automated content-aware sports video analysis would be extremely useful in this regard. A recent (May, 2018) paper [1] surveys this field of activity in great detail. It found that "...more than 80% of papers in the literature addressed baseball, basketball, soccer and tennis. It lists only three(!) papers in total on the subject of hockey. The research in this application proposes to change this situation. Using Computer Vision and Artificial Intelligence, this research is motivated by two societal factors: 1) The pervasiveness of sports as entertainment, and 2) a significant lack of automated collection and analysis of sports data. Because of the significance of hockey to Canadians, the sport we are addressing is professional hockey. It addresses and emphasizes Machine Learning to "understand" and determine hockey analytics. Specifically, the focus is on providing the capability of automatically analyzing video content offline to both detect and quantify relevant temporal and spatial events, in this case, NHL broadcast videos of hockey games.We refer to this as content-aware hockey video analysis. A unique technical feature of this proposal is that we will employ both readily available video and verbal data (Closed Captioning) to perform the content analysis of the hockey game. The research program will focus on the following related projects. Project #1: A system for analyzing hockey broadcast videos, primarily employing pictorial, video and linguistic descriptions of the objects, actions, and events in the video (What is happening on the ice?). Later, we will address Project #2: Based on #1, new methods for computing practical hockey analytics to support coaches and trainers. (What practical data can be gleaned from the videos?). The proposed approach can be referred to as Intelligent Multimodal Video Content Analysis (IMVCA). That is, the capability of automatically analyzing and interpreting both video and language content offline with the aim of detecting and quantifying specific spatial and temporal hockey events. In our case, we exploit broadcast videos of hockey games, but in fact, the methodology could be applied to interpret any video containing Closed Captioning (CC). As such it will be a vehicle for advancing methods for the analysis of videos.