Multimedia Ergonomics in the World of Big Data

大数据正在重新定义我们的世界。在以不断增长的速度生成的各种类型的数据中，多媒体由于其庞大的数量以及创建它的容易性和速度而被称为“最大的大数据”。从地面和卫星监控视频，到科学和医学成像、娱乐、游戏、广告，以及数十亿用户为私人和社会消费创造自己的内容，多媒体不断推动我们技术的极限。为了让我们理解和有效地利用存在于这个庞大的多媒体数据宇宙中的信息，将需要新的科学和工程。 然而，大数据问题并不是什么新鲜事，它在自然界中已经存在了很长时间。我们自己的大脑，以及大多数其他动物的大脑，为了生存，必须进化出各种策略来跟上大量的感官数据。由于大脑不能在真实的时间内有意识地处理所有的感觉数据，因此必须选择哪些是重要的，哪些是重点，这些选择需要相当快地做出，以便有效。这类似于我们今天遇到的技术大数据问题，与生成的数据量相比，可用的处理能力微不足道，并且差距不断增加。关于什么是相关的，重要的，值得我们关注的问题也是技术大数据问题的核心。因此，我们应该期待通过研究大脑应对这一挑战的方式，可以获得有用的见解和策略。 拟议的研究计划旨在通过利用多媒体人体工程学来解决一些紧迫的多媒体大数据挑战，其灵感来自人类为科普其感官数据流而进化的策略。计算注意力在这方面起着至关重要的作用，因为它允许人们从大量可用数据中选择少量最相关的数据。近年来，我们将这些原理应用于视频压缩和通信的传统问题，取得了最先进的成果。 在拟议的计划中，我们将在两个方面解决多媒体大数据的挑战。在理论方面，我们将寻求一个坚实的科学基础和易于处理的数学模型的计算注意力的方法从多个多媒体源。这将使大数据规模的多媒体系统的设计成为可能。在应用方面，我们将致力于应用多媒体人体工程学的原理，并利用压缩数据表示来开发复杂但低成本的算法，用于跟踪，活动检测和发现，注释和检索-理解多媒体大数据的必要成分。

Big Data is redefining our world. Among the various types of data being generated at ever increasing rates, multimedia has been termed "the biggest Big Data" due to its sheer volume, and the ease and pace with which it is being created. From surveillance video, both terrestrial and satellite, through scientific and medical imaging, entertainment, gaming, advertising, and billions of users creating their own content for private and social consumption, multimedia keeps pushing the limits of our technology. For us to make sense of and effectively utilize the information present in this vast universe of multimedia data, new science and engineering will be required. The Big Data problem, however, is not new - it has existed in nature for a long time. Our own brains, as well as those of most other animals, had to evolve various strategies to keep up with the large amount of sensory data in order to survive. Since the brain cannot consciously process all the sensory data in real time, selection has to be made as to what is important and what to focus on. And these selections need to be made rather quickly in order to be effective. This is akin to the technological Big Data problem we have today, where the available processing power is miniscule compared to the amount of data generated, and the gap keeps increasing. Questions about what is relevant, important, and deserves our focus are also at the core of the technological Big Data problem. One should therefore expect that useful insights and strategies can be obtained by studying brain's own ways of dealing with this challenge. The proposed research program aims to address some of the pressing multimedia Big Data challenges by leveraging on multimedia ergonomics, inspired by the strategies humans have evolved to cope with the streams of their sensory data. Computational attention plays a crucial part in this regard, as it allows one to select a small amount of the most relevant data from a large pool of available data. In recent years, we have applied these principles to the traditional problems of video compression and communications, achieving state of the art results. In the proposed program, we will address the multimedia Big Data challenge on two fronts. On the theory side, we will seek a solid scientific underpinning and tractable mathematical models for the computational attention approaches from multiple multimedia sources. These will enable the design of multimedia systems on the Big Data scale. On the application side, we will work to apply the principles of multimedia ergonomics and utilize compressed data representation to develop sophisticated, yet low-cost algorithms for tracking, activity detection and discovery, annotation and retrieval - necessary ingredients for making sense of multimedia Big Data.