Multimedia Ergonomics in the World of Big Data

大数据正在重新定义我们的世界。在以不断增长的速度生成的各种类型的数据中，多媒体因其绝对的数据量和创建的容易和速度而被称为“最大的大数据”。从地面和卫星监控视频，到科学和医学成像、娱乐、游戏、广告，以及数十亿用户为私人和社会消费创建自己的内容，多媒体不断突破我们的技术极限。我们要理解和有效利用多媒体数据这个浩瀚宇宙中存在的信息，就需要新的科学和工程。 然而，大数据问题并不新鲜--它在自然界中已经存在了很长一段时间。我们自己的大脑，以及大多数其他动物的大脑，为了生存，不得不进化出各种策略来跟上大量的感觉数据。由于大脑不能有意识地实时处理所有的感觉数据，所以必须选择什么是重要的，什么是重点。这些选择需要相当快地做出，才能有效。这类似于我们今天面临的技术大数据问题，与生成的数据量相比，可用处理能力微不足道，而且差距还在不断扩大。关于什么是相关的、重要的和值得我们关注的问题，也是技术大数据问题的核心。因此，人们应该期待通过研究大脑自身处理这一挑战的方式来获得有用的见解和策略。 拟议的研究计划旨在通过利用多媒体人体工程学来应对一些紧迫的多媒体大数据挑战，其灵感来自人类为应对感知数据流而进化出的策略。在这方面，计算注意力起着至关重要的作用，因为它允许人们从大量可用数据中选择少量最相关的数据。近年来，我们将这些原则应用于视频压缩和通信的传统问题，取得了最先进的结果。 在拟议的计划中，我们将从两个方面应对多媒体大数据挑战。在理论方面，我们将从多个多媒体源中为计算注意方法寻求坚实的科学依据和易于处理的数学模型。这些将使在大数据规模上设计多媒体系统成为可能。在应用方面，我们将致力于应用多媒体人体工程学的原理，并利用压缩数据表示法来开发复杂而低成本的算法，用于跟踪、活动检测和发现、注释和检索--这些都是理解多媒体大数据的必要组成部分。

Big Data is redefining our world. Among the various types of data being generated at ever increasing rates, multimedia has been termed "the biggest Big Data" due to its sheer volume, and the ease and pace with which it is being created. From surveillance video, both terrestrial and satellite, through scientific and medical imaging, entertainment, gaming, advertising, and billions of users creating their own content for private and social consumption, multimedia keeps pushing the limits of our technology. For us to make sense of and effectively utilize the information present in this vast universe of multimedia data, new science and engineering will be required. The Big Data problem, however, is not new - it has existed in nature for a long time. Our own brains, as well as those of most other animals, had to evolve various strategies to keep up with the large amount of sensory data in order to survive. Since the brain cannot consciously process all the sensory data in real time, selection has to be made as to what is important and what to focus on. And these selections need to be made rather quickly in order to be effective. This is akin to the technological Big Data problem we have today, where the available processing power is miniscule compared to the amount of data generated, and the gap keeps increasing. Questions about what is relevant, important, and deserves our focus are also at the core of the technological Big Data problem. One should therefore expect that useful insights and strategies can be obtained by studying brain's own ways of dealing with this challenge. The proposed research program aims to address some of the pressing multimedia Big Data challenges by leveraging on multimedia ergonomics, inspired by the strategies humans have evolved to cope with the streams of their sensory data. Computational attention plays a crucial part in this regard, as it allows one to select a small amount of the most relevant data from a large pool of available data. In recent years, we have applied these principles to the traditional problems of video compression and communications, achieving state of the art results. In the proposed program, we will address the multimedia Big Data challenge on two fronts. On the theory side, we will seek a solid scientific underpinning and tractable mathematical models for the computational attention approaches from multiple multimedia sources. These will enable the design of multimedia systems on the Big Data scale. On the application side, we will work to apply the principles of multimedia ergonomics and utilize compressed data representation to develop sophisticated, yet low-cost algorithms for tracking, activity detection and discovery, annotation and retrieval - necessary ingredients for making sense of multimedia Big Data.