权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Deep Learning from Crawled Spatio-Temporal Representations of Video (DECSTER)

从视频的爬行时空表示中进行深度学习 (DECSTER)

基本信息

批准号：
EP/R025290/1
负责人：
Yiannis Andreopoulos
金额：
$ 63.27万
依托单位：
University College London
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2018
资助国家：
英国
起止时间：
2018 至无数据
项目状态：
已结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FR025290%2F1
关键词：
Deep Learning Crawled Spatio Temporal

项目摘要

Video has been one of the most pervasive forms of online media for some time. Several statistics show that video traffic will dominate IP networks within the next five years. Yet, video remains one of the least-manageable elements of the big data ecosystem. This project argues that this difficulty stems primarily from the fact that all advanced computer vision and machine learning algorithms view video as a stream of frames of picture elements. This is despite the fact that pixel-domain representations are known to be notoriously difficult to manage in machine learning systems, mainly due to: their high volume, high redundancy between successive frames, and artifacts stemming from camera calibration under varying illumination. We propose to abandon pixel representations and consider spatio-temporal activity information that is directly extractable from compressed video bitstreams or neuromorphic vision sensing (NVS) hardware. The first key outcome of the project will be to design deep neural networks (DNNs) that ingest such activity information in order to derive state-of-the-art classification, action recognition and retrieval results within large video datasets. This will be achieved at record-breaking speed and comparable accuracy to the best DNN designs that utilize pixel-domain video representations and/or optical flow calculations. The second key outcome will be to design and prototype a crawler-based bitstream parsing and analysis service, where some of the parsing and processing will be carried out by a bitstream crawler running on a remote repository, while the back-end processing will be carried out by high-performance servers in the cloud. This will enable for the first time the continuous parsing of large compressed video content libraries and NVS repositories with new & improved versions of crawlers in order to derive continuously-improved semantics or track changes and new content elements, in a manner similar to how search engine bots continuously crawl web content. These outcomes will pave the way for exabyte-scale video datasets to be newly-discovered and analysed over commodity hardware.

一段时间以来，视频一直是最普遍的在线媒体形式之一。几项统计数据显示，视频流量将在未来五年内主导IP网络。然而，视频仍然是大数据生态系统中最难管理的元素之一。该项目认为，这种困难主要源于所有先进的计算机视觉和机器学习算法将视频视为图像元素帧流的事实。尽管像素域表示在机器学习系统中非常难以管理，这主要是由于：它们的高容量，连续帧之间的高冗余，以及在不同照明下由相机校准产生的伪影。我们建议放弃像素表示，并考虑直接从压缩视频比特流或神经形态视觉传感（NVS）硬件中提取的时空活动信息。该项目的第一个关键成果将是设计深度神经网络（dnn），该网络可以吸收这些活动信息，以便在大型视频数据集中获得最先进的分类、动作识别和检索结果。这将以破纪录的速度和与利用像素域视频表示和/或光流计算的最佳DNN设计相当的精度实现。第二个关键成果将是设计和原型一个基于爬虫的比特流解析和分析服务，其中一些解析和处理将由运行在远程存储库上的比特流爬虫执行，而后端处理将由云中的高性能服务器执行。这将首次实现对大型压缩视频内容库和NVS库的持续解析，并使用新的和改进版本的抓取工具，以获得不断改进的语义或跟踪变化和新的内容元素，其方式类似于搜索引擎机器人如何持续抓取web内容。这些成果将为艾字节级视频数据集的新发现和分析铺平道路。

项目成果

期刊论文数量（10）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Biased Mixtures of Experts: Enabling Computer Vision Inference Under Data Transfer Limitations

DOI：
10.1109/tip.2020.3005508
发表时间：
2020-07
期刊：
IEEE Transactions on Image Processing
影响因子：
10.6
作者：
Alhabib Abbas;Y. Andreopoulos
通讯作者：
Alhabib Abbas;Y. Andreopoulos

PAC-Bayesian Bounds on Rate-Efficient Classifiers

DOI：
发表时间：
2022
期刊：
影响因子：
0
作者：
Alhabib Abbas;Y. Andreopoulos
通讯作者：
Alhabib Abbas;Y. Andreopoulos

Neuromorphic Vision Sensing for CNN-based Action Recognition

DOI：
10.1109/icassp.2019.8683606
发表时间：
2019-05
期刊：
ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
影响因子：
0
作者：
Aaron Chadha;Yin Bi;Alhabib Abbas;Y. Andreopoulos
通讯作者：
Aaron Chadha;Yin Bi;Alhabib Abbas;Y. Andreopoulos

Rate-Accuracy Trade-Off in Video Classification with Deep Convolutional Neural Networks