Biologically and Probabilistically guided Multimedia

人类的视觉系统由一个分辨率非常高的中心凹支撑，而中心凹的外围分辨率迅速下降。中心凹从眼睛中心延伸出一个大约2度的圆锥体，捕捉到细节。因此，在阅读距离上，我们一次只能清楚地看到几个字母的文本。然而，在我们的头脑中，我们认为一切都是清晰可见的。这种感觉是因为我们的眼睛是动态的或“活跃的”，总是被大脑引导着在给定的时刻准确地看着最重要的东西。在我的基础研究中，我引入了用于图像、视频和3D压缩的中心凹的概念。此外，考虑到眼睛的运动，我开发了第一个不使用任何已知模式或通过匹配单个特征点的相机主动校准。这是第一个与人眼的摇摄、倾斜和扭转旋转一致的校准方法，回答了一些关于人类视觉的更深层次的问题。我还考虑到支持人眼的宽广视野，引入了使用一个摄像头的全景立体声。这些基础研究主题影响了编码标准纳入“感兴趣区域”的方式，并导致了合作者和学员多年来创建的衍生产品，如PVSI和VisionSplend。*遵循生物动机设计多媒体系统只是我解决几个问题的方法的第一部分，补充这一点的第二部分是算法及其分析。对于第二部分，我主要强调的是概率方法。例如，通过对误差分布的统计分析，我证明了为什么我的主动算法更健壮。通过一个平均的个例分析，我证明了我们的拉格朗日平流的有效性。通过概率随机游走改进了图像融合。我使用随机扰动进行稳健的匹配和配准。最后，我还介绍了使用随机过程可靠地检测早产儿脑损伤的方法。*我计划在未来五年引入以下新组件：(A)将凹陷引入全景立体；(B)通过研究观众注意力的作用，进一步改进运动捕捉数据压缩；(C)将显著(或图像中重要区域的检测)与凹陷相结合，以便对人类观察者进行更好的多媒体(图像、视频和3D)编码；(D)使主动相机校准更加稳健；以及(E)开发内科和外科图像分析的新方法。我的算法的开发和分析将基于彻底的概率技术和分析，就像我在过去的研究中一样。*拟议的研究如果成功，将对下一代全景、3D和全光多媒体捕获、处理和传输以及医疗和外科创新产生重大影响。

The Human Visual System is supported by a very high resolution fovea with rapidly declining resolution in the periphery. The fovea captures details in about a 2 degree cone extending from the center of the eyes. Thus, we can only see a few letters of text clearly at a time at reading distances. In our mind, however, we think that everything is clearly visible. This perception is a result of our eyes being dynamic or “active” and always being guided by the brain to look at precisely what is most important at a given instant.******In my basic research I introduced the concept of foveation for image, video and 3D compression. Furthermore, considering eye movements I developed the first active calibration of cameras without using any known patterns or by matching individual feature points. It is the first calibration approach that is consistent with the pan, tilt and torsional rotations of the human eye, answering some deeper questions on human vision. I also considered supporting the wide field of view of the human eyes and introduced “Panoramic Stereo” using one camera. These fundamental research topics have impacted the way coding standards have incorporated “region of interest,” and have resulted in the creation of spin-offs, like PVSI and VisionSplend, over the years by collaborators and trainees. ******Designing multimedia systems following biological motivation is only the first part of my approach to addressing several problems, the second part complementing this are the algorithms and their analyses. For the second part my major emphasis is on probabilistic approaches. For example, by statistical analysis of the distribution of errors I proved why my active algorithms are much more robust. Through an average case analysis I demonstrated the efficiency of our Lagrangian advection. Through probabilistic random walks I improved image fusion. I used stochastic perturbation for robust matching and registration. Finally, I also introduced the use of stochastic processes for reliable detection of brain injuries in prematurely born infants. ******Over the next five years I plan to introduce the following novel components: (a) incorporating foveation into panoramic stereo; (b) making further improvements to Motion Capture data compression by studying the role of attention of viewers; (c) combining saliency (or the detection of important regions in an image) with foveation for better multimedia (image, video and 3D) coding with respect to human observers; (d) making active camera calibration more robust; and (e) developing new approaches in medical and surgical image analysis. The development and analysis of my algorithms will be grounded in thorough probabilistic techniques and analysis, as in my past research.******The proposed research, if successful, will have significant impact in next generation panoramic, 3D and plenoptic multimedia capture, processing, and transmission, as well as in medical and surgical innovations.*****