Illuminating the Path of Video Visualization

照亮视频可视化之路

• 批准号: EP/G006555/2
• 负责人: Min Chen
• 金额: $ 47.33万
• 依托单位: University of Oxford
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2011
• 资助国家: 英国
• 起止时间: 2011 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FG006555%2F2
• 关键词: Illuminating; Path; Video; Visualization

The notion of video visualization was coined by the PI and his postgraduate student in their 2003 IEEE VIS paper. It is a technology drawing the concepts and methodologies from volume and flow visualization, image and video processing, and vision science. It extracts meaningful information from a video data set and conveys the extracted information to users in appropriate visual representations. It is not intended to provide fully automatic solutions to the traditional problems in video processing, but involves human in the loop of intelligent reasoning while reducing the burden of viewing videos. In the subsequent work in collaboration with Stuttgart, the PI and CI introduced the concept of visual signatures in video visualization, and reported a major user study conducted at Swansea involving some 92 subjects [IEEE TVCG 2006]. This work offered an important scientific insight as to how human observers may learn to recognize visual signatures of events depicted in an abstract visual representation of a video.[Tsotsos01] stated that a bounded visual search (e.g., looking for all moving pixel clusters with 20-60 pixels) can be achieved in linear time, whist an unbounded visual search (e.g., looking for something abnormal in a video) is NP-complete. For most practical problems in video processing and computer vision, we rarely have perfectly bounded visual search. We often search simultaneously for entities (e.g., objects, motions or events) in different classes. The models that are used to guide a search are usually incomplete and may lead to uncertainty or errors in detection, segmentation andclassification. The dynamic and unpredictable nature of the input videos instigates mechanisms for heuristic reasoning and iterative decision optimization, which further depart from linear or polynomial performance.In contrast, the human eye-brain system is undeniable more powerful than any current vision system in performing visual searches, especially unbounded visual searches. Even we suppose that the human eye-brain system is a Turing machine, its 100 billion neurons and 100-500 trillion synaptic connections between neurons will unlikely to be matched by computers in the near future. Hence this raises the possibility that using video visualization to aid unbound visual search may provide a more scalable means for dealing with large volumes of video datasets.Video visualization can be deployed in many application areas, such as scientific experimentation and computation, security industry and media and entertainment industry. However, in traditional visualization (e.g., medical visualization), the users are normally familiar with the 3D objects (e.g., bones or organs) depicted in a visual representation. In contrast, human observers are not familiar with the 3D objects depicted in a visual representation of a video because one spatial dimension of these objects shows the temporal dimension. The problem is further complicated by the fact that, in most videos, each 2D frame is the projective view of a 3D scene. Hence, a visual representation of a video on a computer display is, in effect, a 2D projective view of a 4D spatiotemporal domain. In order to for us to see 'time' without using 'time', we need to address a range of challenges in science, technology, visual perception and applications. This project is intended to continue the UK's leadership in tackling these challenges by building on the existing expertise and excellence in video visualization.

视频可视化的概念是由PI和他的研究生在他们2003年的IEEE VIS论文中创造的。它是一种从体积和流动可视化、图像和视频处理以及视觉科学中提取概念和方法的技术。它从视频数据集中提取有意义的信息，并以适当的视觉表示将提取的信息传达给用户。它的目的不是为视频处理中的传统问题提供全自动的解决方案，而是在减轻观看视频的负担的同时，将人类引入智能推理的循环中。在随后与斯图加特合作的工作中，PI和CI在视频可视化中引入了视觉签名的概念，并报告了在斯旺西进行的一项涉及约92名受试者的主要用户研究[IEEE TVCG 2006]。这项工作为人类观察者如何学习识别视频的抽象视觉表示中描述的事件的视觉特征提供了重要的科学见解。[Tsotsos01]指出，有界视觉搜索(例如，寻找所有具有20-60像素的运动像素簇)可以在线性时间内实现，其中无界视觉搜索(例如，寻找视频中的异常)是NP-完全的。对于视频处理和计算机视觉中的大多数实际问题，我们很少有完全有界的视觉搜索。我们经常同时搜索不同类别的实体(例如，对象、运动或事件)。用于指导搜索的模型通常是不完整的，可能会导致检测、分割和分类中的不确定性或错误。输入视频的动态性和不可预测性激发了启发式推理和迭代决策优化的机制，这进一步偏离了线性或多项式的性能。相比之下，不可否认，人眼-大脑系统在执行视觉搜索，特别是无限视觉搜索方面比任何现有的视觉系统都更强大。即使我们假设人类的眼睛-大脑系统是一台图灵机器，它的1000亿个神经元和100-500万亿个神经元之间的突触连接在不久的将来也不太可能被计算机匹配。因此，使用视频可视化来辅助非绑定视觉搜索可能会为处理海量视频数据提供一种更具可扩展性的手段，视频可视化可以部署在许多应用领域，如科学实验和计算、安全行业和媒体和娱乐行业。然而，在传统的可视化(例如，医学可视化)中，用户通常熟悉以视觉表示描述的3D对象(例如，骨骼或器官)。相反，人类观察者不熟悉视频的视觉表示中描述的3D对象，因为这些对象的一个空间维度显示时间维度。在大多数视频中，每个2D帧都是3D场景的投影视图，这一事实使问题变得更加复杂。因此，视频在计算机显示器上的可视表示实际上是4D时空域的2D投影视图。为了让我们在不使用“时间”的情况下看到“时间”，我们需要解决科学、技术、视觉感知和应用方面的一系列挑战。该项目旨在通过在视频可视化方面的现有专业知识和卓越表现的基础上，继续保持英国在应对这些挑战方面的领导地位。

项目成果

Glyph sorting: Interactive visualization for multi-dimensional data

• DOI: 10.1177/1473871613511959
• 发表时间: 2015-01-01
• 期刊: INFORMATION VISUALIZATION
• 影响因子: 2.3
• 作者: Chung, David H. S.;Legg, Philip A.;Chen, Min
• 通讯作者: Chen, Min

Transformation of an Uncertain Video Search Pipeline to a Sketch-Based Visual Analytics Loop

• DOI: 10.1109/tvcg.2013.207
• 发表时间: 2013-12
• 期刊: IEEE Transactions on Visualization and Computer Graphics
• 影响因子: 5.2
• 作者: P. Legg;David H. S. Chung;M. L. Parry;R. Bown;Mark W. Jones;I. Griffiths;Min Chen

Information Theory Tools for Visualization

用于可视化的信息论工具

• 发表时间: 2016
• 作者: Chen Min

Facial expression recognition in dynamic sequences: An integrated approach

• DOI: 10.1016/j.patcog.2013.09.023
• 发表时间: 2014-03
• 期刊: Pattern Recognit.
• 作者: H. Fang;Neil MacParthaláin;Andrew J. Aubrey;G. Tam;R. Borgo;Paul L. Rosin;P. W. Grant;A. D. Marshall;Min Chen