Adaptive cognition for automated sports video annotation (ACASVA)

自动运动视频注释的自适应认知（ACASVA）

• 批准号: EP/F069421/1
• 负责人: Josef Kittler
• 金额: $ 180.36万
• 依托单位: University of Surrey
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2009
• 资助国家: 英国
• 起止时间: 2009 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FF069421%2F1
• 关键词: Adaptive; cognition; automated; sports; video

The development of a machine that can autonomously understand and interpret patterns of real-world events remains a challenging goal in AI. Humans are able to achieve this by developing sophisticated internal representational structures for object and events and the grammars that connect them. ACASVA aims to investigate the interaction between visual and linguistic grammars in learning by developing grammars in a scenario where the number of different events is constrained, by a set of rules, to be small: a sport. We will analyse video footage of a game (e.g. tennis) and use computer vision techniques to progressively understand it as a sequence of (possibly overlapping) events, and build a grammar of events. We will do a similar audio/linguistic analysis on the commentary on the game. Both of these grammars will be used to build a representational structure for understanding the game. Visual representations are additionally constrained by the inference of game rules so that object-classification mechanisms are preferentially tuned to game-relevant entities like 'player' rather than game-irrelevant entities like 'crowd-member'. We will also investigate how the two modes, sight and sound, can influence each other in the learning process; interpretation of the video is affected by the linguistic grammar and vice versa. Furthermore, this coupling of modes will lead to improved recognition of both audio and video events when the grammars from the video modes are used to influence the audio recognition, and vice versa. The psychological component of the ACASVA correspondingly attempts to learn how these capabilities are developed in humans; how visual grammars are organized and employed in the learning problem, how these grammars are modified by prior linguistic knowledge of the domain, how visual grammars map onto linguistic grammars, and how game rule-inferences influence lower-level visual learning (determined via gaze-behaviour). These results will feedback into the machine-learning problem and vice versa, as well as providing a performance benchmark for the system.Potential beneficiaries of ACASVA (in addition to the knowledge beneficiaries within the fields of science and engineering) include the broadcasting and on-line video search industries.

开发能够自主理解和解释现实世界事件模式的机器仍然是人工智能领域的一个具有挑战性的目标。人类能够通过开发对象和事件的复杂内部表征结构以及连接它们的语法来实现这一目标。 ACASVA 旨在研究视觉语法和语言语法在学习中的相互作用，方法是在不同事件的数量受到一组规则限制的场景中开发语法：一项运动。我们将分析比赛（例如网球）的视频片段，并使用计算机视觉技术逐步将其理解为一系列（可能重叠的）事件，并构建事件语法。我们将对游戏的解说进行类似的音频/语言分析。这两种语法都将用于构建理解游戏的表征结构。视觉表示还受到游戏规则推理的限制，因此对象分类机制优先调整为与游戏相关的实体（例如“玩家”），而不是与游戏无关的实体（例如“人群成员”）。我们还将研究视觉和声音这两种模式在学习过程中如何相互影响；视频的解释受到语言语法的影响，反之亦然。此外，当视频模式的语法用于影响音频识别时，这种模式的耦合将导致音频和视频事件的识别得到改善，反之亦然。 ACASVA 的心理部分相应地尝试了解人类如何发展这些能力；视觉语法如何在学习问题中组织和使用，这些语法如何通过该领域的先前语言知识进行修改，视觉语法如何映射到语言语法，以及游戏规则推理如何影响较低级别的视觉学习（通过凝视行为确定）。这些结果将反馈到机器学习问题中，反之亦然，并为系统提供性能基准。 ACASVA 的潜在受益者（除了科学和工程领域内的知识受益者）包括广播和在线视频搜索行业。

项目成果

Class-Specific Kernel Fusion of Multiple Descriptors for Face Verification Using Multiscale Binarised Statistical Image Features

• DOI: 10.1109/tifs.2014.2359587
• 发表时间: 2014-12-01
• 期刊: IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY
• 影响因子: 6.8
• 作者: Arashloo, Shervin Rahimzadeh;Kittler, Josef
• 通讯作者: Kittler, Josef

Biased but in doubt: conflict and decision confidence.

• DOI: 10.1371/journal.pone.0015954
• 发表时间: 2011-01-25
• 期刊: PloS one
• 影响因子: 3.7
• 作者: De Neys W;Cromheeke S;Osman M
• 通讯作者: Osman M

Report on the FG 2015 Video Person Recognition Evaluation

• DOI: 10.1109/fg.2015.7163156
• 发表时间: 2015-05
• 期刊: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)
• 作者: J. Beveridge;Hao Zhang;B. Draper;P. Flynn;Zhenhua Feng;P. Huber;J. Kittler;Zhiwu Huang

Face Spoofing Detection Based on Multiple Descriptor Fusion Using Multiscale Dynamic Binarized Statistical Image Features

• DOI: 10.1109/tifs.2015.2458700
• 发表时间: 2015-11-01
• 期刊: IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY
• 影响因子: 6.8
• 作者: Arashloo, Shervin Rahimzadeh;Kittler, Josef;Christmas, William
• 通讯作者: Christmas, William

On detection of novel categories and subcategories of images using incongruence

利用不一致检测图像的新类别和子类别

• DOI: 10.1145/2578726.2578769
• 发表时间: 2014
• 作者: Coppi D