Time Information for Multimedia Processing and Analysis

数字多媒体数据量每天都在以惊人的速度增长。多媒体研究的主要挑战之一是开发算法，从这些海量的多媒体信号中提取信息，并提供更有效的工具来与它们交互。使用数字信号处理和机器学习的各种技术，现在可以分析数以百万计的图像、视频和歌曲。最近的另一个趋势是使用丰富的表现力方式与计算机交互，超越了传统的屏幕/键盘/鼠标范例。这些包括触摸屏上的手势，以及基于各种传感器的全身跟踪界面，如加速计、麦克风阵列和使用结构光的深度相机。结合由丰富的多模式输入控制的大规模多媒体档案分析的工作的例子很少。将传感器、执行器、计算机和人类结合在一起的互动音乐表演是少数几个产生这种协同效应的领域之一。

The amount of multimedia data available digitally is growing daily at an astounding rate. One of the main challenges in multimedia research is to develop algorithms to extract information from these vast collections of multimedia signals and provide more effective tools for interacting with them. Using a variety of techniques from digital signal processing and machine learning it is now possible to analyze millions of images, videos and songs. Another recent trend is the use of rich expressive ways of interacting with computers moving beyond the traditional screen/keyboard/mouse paradigm. These include gestures on touchscreens as well as full body tracking interfaces based on a variety of sensors such as accelerometers, microphone arrays and depth cameras using structured light. There are few examples of work that combine large scale analysis of multimedia archives controlled by rich multi-modal input. Interactive music performances that combine sensors, actuators, computers and humans are one of the few areas where such synergies are taking place.