Biologically and Probabilistically guided Multimedia

生物和概率引导的多媒体

• 批准号: RGPIN-2017-04786
• 负责人: Basu, Anup
• 金额: $ 1.89万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=644771
• 关键词: Biologically; Probabilistically; guided; Multimedia

The Human Visual System is supported by a very high resolution fovea with rapidly declining resolution in the periphery. The fovea captures details in about a 2 degree cone extending from the center of the eyes. Thus, we can only see a few letters of text clearly at a time at reading distances. In our mind, however, we think that everything is clearly visible. This perception is a result of our eyes being dynamic or “active” and always being guided by the brain to look at precisely what is most important at a given instant.******In my basic research I introduced the concept of foveation for image, video and 3D compression. Furthermore, considering eye movements I developed the first active calibration of cameras without using any known patterns or by matching individual feature points. It is the first calibration approach that is consistent with the pan, tilt and torsional rotations of the human eye, answering some deeper questions on human vision. I also considered supporting the wide field of view of the human eyes and introduced “Panoramic Stereo” using one camera. These fundamental research topics have impacted the way coding standards have incorporated “region of interest,” and have resulted in the creation of spin-offs, like PVSI and VisionSplend, over the years by collaborators and trainees. ******Designing multimedia systems following biological motivation is only the first part of my approach to addressing several problems, the second part complementing this are the algorithms and their analyses. For the second part my major emphasis is on probabilistic approaches. For example, by statistical analysis of the distribution of errors I proved why my active algorithms are much more robust. Through an average case analysis I demonstrated the efficiency of our Lagrangian advection. Through probabilistic random walks I improved image fusion. I used stochastic perturbation for robust matching and registration. Finally, I also introduced the use of stochastic processes for reliable detection of brain injuries in prematurely born infants. ******Over the next five years I plan to introduce the following novel components: (a) incorporating foveation into panoramic stereo; (b) making further improvements to Motion Capture data compression by studying the role of attention of viewers; (c) combining saliency (or the detection of important regions in an image) with foveation for better multimedia (image, video and 3D) coding with respect to human observers; (d) making active camera calibration more robust; and (e) developing new approaches in medical and surgical image analysis. The development and analysis of my algorithms will be grounded in thorough probabilistic techniques and analysis, as in my past research.******The proposed research, if successful, will have significant impact in next generation panoramic, 3D and plenoptic multimedia capture, processing, and transmission, as well as in medical and surgical innovations.*****

人类视觉系统是由一个非常高分辨率的中央凹支持的，而周围的分辨率则迅速下降。中央凹捕捉从眼睛中心延伸出的约2度圆锥体的细节。因此，在阅读距离上，我们一次只能清楚地看到几个字母。然而，在我们的头脑中，我们认为一切都是清晰可见的。这种感知是由于我们的眼睛是动态的或“活跃的”，并且总是由大脑引导，在给定的时刻准确地看到最重要的东西。******在我的基础研究中，我介绍了图像、视频和3D压缩的注视点概念。此外，考虑到眼球运动，我开发了第一个不使用任何已知模式或通过匹配单个特征点的相机主动校准。这是第一个与人眼的平移、倾斜和扭转旋转一致的校准方法，回答了人类视觉的一些更深层次的问题。我也考虑过支持人眼的广角视野，并引入了单摄像头的“全景立体”。这些基础研究主题影响了编码标准纳入“兴趣区域”的方式，并导致了合作者和学员多年来创建的衍生产品，如PVSI和VisionSplend。******根据生物动机设计多媒体系统只是我解决几个问题的方法的第一部分，第二部分是算法及其分析。对于第二部分，我的重点是概率方法。例如，通过对误差分布的统计分析，我证明了为什么我的主动算法更健壮。通过一个平均案例分析，我证明了拉格朗日平流的有效性。通过概率随机漫步改进了图像融合。我使用随机扰动进行鲁棒匹配和配准。最后，我还介绍了使用随机过程来可靠地检测早产儿的脑损伤。******在接下来的五年里，我计划引入以下新组件：(a)将注视点纳入全景立体；(b)通过研究观众注意力的作用，进一步改进运动捕捉数据压缩；(c)将显著性（或对图像中重要区域的检测）与注视点相结合，以便对人类观察者进行更好的多媒体（图像、视频和3D）编码；(d)使有源相机标定更稳健；(e)发展医学和外科图像分析的新方法。我的算法的开发和分析将基于彻底的概率技术和分析，就像我过去的研究一样。******所提出的研究如果成功，将对下一代全景、3D和全光多媒体捕获、处理和传输以及医疗和外科创新产生重大影响。*****