Biologically motivated Multimedia

人类视觉系统的特点是中心凹的分辨率非常高，外围分辨率迅速下降。中心凹从眼睛中心延伸出一个大约2度的圆锥体，捕捉到细节。因此，在给定的时刻，我们只能在阅读距离上清楚地看到几个字母的文本。然而，在我们的头脑中，我们认为一切都是清晰可见的。这种知觉是因为我们的眼睛是动态的或“活跃的”，并且总是被大脑引导，在给定的时刻准确地看着最重要的东西。在我的基础研究中，我引入了用于图像、视频和3D压缩的凹陷的概念。此外，考虑到眼睛的运动模仿人类的视觉，我开发了第一个不使用任何已知模式或匹配特征点的相机主动校准。我还介绍了用于背景减去的分布式学习、用于识别感兴趣区域的显著检测以及用于降低带宽需求的运动捕获(MOCAP)压缩的新算法。这些基础研究主题影响了编码标准纳入“感兴趣区域”的方式，导致多年来由合作者和受训者创建的副产品和由Discovery资助的研究的商业化。我未来五年研究的总体目标是开发新的生物激励算法，遵循健壮的多媒体捕获、表示和通信的主题。在短期内，我打算在以下方面取得新的进展：(I)无惯性传感器的主动校准；(Ii)稳健的背景减去和实例分割；(Iii)3D显著驱动的中心凹建模和由此产生的图像捕获和压缩；(Iv)注意力引导的运动捕获(MOCAP)数据压缩；以及(V)用于解决这些主题的数学模型和学习架构。我的长期愿景是使生物因素成为所有多媒体系统的核心，将它们纳入标准，追求现实世界实施的挑战，并将研究成果商业化。我工作的意义将在于：(A)改进的对象、人脸和形状识别；(B)多个动态中心凹的建模技术，以及新的中心凹全光成像；(C)改进的MOCAP传输策略，纳入观众的注意力；(D)稳健的主动摄像机校准，导致这种方法的更广泛应用；(E)新的优化技术和学习模型，可以改进底层计算框架，以获得更好和更稳健的场景理解问题解决方案；以及(F)通过基础发现研究开发的算法和技术在工业和医疗保健中的应用。这项拟议的研究如果成功，将对下一代多媒体捕获、处理和传输技术以及医疗和外科创新产生重大影响。

The Human Visual System is characterized by a very high-resolution fovea with rapidly declining resolution in the periphery. The fovea captures details in an approximately 2 degree cone extending from the center of the eyes. Thus, at a given instant in time we can only see a few letters of text clearly at reading distances. In our mind, however, we think that everything is clearly visible. This perception is a result of our eyes being dynamic or "active" and always being guided by the brain to look at precisely what is most important at a given instant. In my basic research, I introduced the concept of foveation for image, video and 3D compression. Furthermore, considering eye movements mimicking human vision, I developed the first active calibration of cameras without using any known patterns or matching feature points. I also introduced new algorithms for distribution learning for background subtraction, saliency detection for identifying regions of interest, and Motion Capture (MoCap) compression to reduce bandwidth requirements. These fundamental research topics have influenced the way coding standards have incorporated "region of interest," resulting in creation of spin-offs and commercialization of Discovery funded research over the years by collaborators and trainees. The overall goal of my research over the next five years is to develop new biologically motivated algorithms following the theme of robust multimedia capture, representation, and communication. In the short-term I intend to make novel advancements in: (i) Active Calibration without inertial sensors; (ii) Robust background subtraction and instance segmentation; (iii) 3D Saliency driven fovea modeling and resulting image capture and compression; (iv) Attention guided Motion Capture (MoCap) data compression; and (v) Mathematical models and learning architectures for addressing these topics. My long-term vision is to make biological factors the core of all multimedia systems, by incorporating them into standards, pursuing the challenges of real world implementations, and commercializing research results. The significance of my work will lie in: (a)Improved object, face and shape recognition; (b)Modeling techniques for multiple and dynamic foveae, and new foveated plenoptic imaging; (c)Improved MoCap transmission strategies incorporating attention of viewers; (d)Robust active calibration of cameras, leading to wider deployment of this approach; (e)New optimization techniques and learning models that can improve the underlying computational framework to obtain better and more robust solutions to scene understanding problems; and, (f)Applications of algorithms and techniques developed through fundamental Discovery research in industry as well as healthcare. The proposed research, if successful, will have significant impact in next generation multimedia capture, processing, and transmission technologies, as well as in medical and surgical innovations.