Biologically and Probabilistically guided Multimedia

生物和概率引导的多媒体

• 批准号: RGPIN-2017-04786
• 负责人: Basu, Anup
• 金额: $ 1.89万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=711219
• 关键词: Biologically; Probabilistically; guided; Multimedia

The Human Visual System is supported by a very high resolution fovea with rapidly declining resolution in the periphery. The fovea captures details in about a 2 degree cone extending from the center of the eyes. Thus, we can only see a few letters of text clearly at a time at reading distances. In our mind, however, we think that everything is clearly visible. This perception is a result of our eyes being dynamic or “active” and always being guided by the brain to look at precisely what is most important at a given instant. In my basic research I introduced the concept of foveation for image, video and 3D compression. Furthermore, considering eye movements I developed the first active calibration of cameras without using any known patterns or by matching individual feature points. It is the first calibration approach that is consistent with the pan, tilt and torsional rotations of the human eye, answering some deeper questions on human vision. I also considered supporting the wide field of view of the human eyes and introduced “Panoramic Stereo” using one camera. These fundamental research topics have impacted the way coding standards have incorporated “region of interest,” and have resulted in the creation of spin-offs, like PVSI and VisionSplend, over the years by collaborators and trainees. Designing multimedia systems following biological motivation is only the first part of my approach to addressing several problems, the second part complementing this are the algorithms and their analyses. For the second part my major emphasis is on probabilistic approaches. For example, by statistical analysis of the distribution of errors I proved why my active algorithms are much more robust. Through an average case analysis I demonstrated the efficiency of our Lagrangian advection. Through probabilistic random walks I improved image fusion. I used stochastic perturbation for robust matching and registration. Finally, I also introduced the use of stochastic processes for reliable detection of brain injuries in prematurely born infants. Over the next five years I plan to introduce the following novel components: (a) incorporating foveation into panoramic stereo; (b) making further improvements to Motion Capture data compression by studying the role of attention of viewers; (c) combining saliency (or the detection of important regions in an image) with foveation for better multimedia (image, video and 3D) coding with respect to human observers; (d) making active camera calibration more robust; and (e) developing new approaches in medical and surgical image analysis. The development and analysis of my algorithms will be grounded in thorough probabilistic techniques and analysis, as in my past research. The proposed research, if successful, will have significant impact in next generation panoramic, 3D and plenoptic multimedia capture, processing, and transmission, as well as in medical and surgical innovations.

人类视觉系统由非常高分辨率的中央凹支持，其周边的分辨率迅速下降。中央凹捕获从眼睛中心延伸的约2度圆锥体中的细节。因此，在阅读距离内，我们一次只能清楚地看到文本的几个字母。然而，在我们心中，我们认为一切都是清晰可见的。这种感知是由于我们的眼睛是动态的或“活跃的”，并且总是被大脑引导，在给定的时刻精确地看最重要的东西。 在我的基础研究中，我介绍了图像，视频和3D压缩的视觉聚焦概念。此外，考虑到眼睛的运动，我开发了第一个主动校准相机，而不使用任何已知的模式或匹配个别特征点。这是第一种与人眼的平移、倾斜和扭转旋转一致的校准方法，回答了人类视觉的一些更深层次的问题。我还考虑支持人眼的宽视野，并介绍了使用一个相机的“全景立体”。这些基础研究主题影响了编码标准纳入“感兴趣区域”的方式，并导致多年来合作者和学员创建了PVSI和VisionSplend等衍生产品。 根据生物学动机设计多媒体系统只是我解决几个问题的方法的第一部分，第二部分是算法及其分析。在第二部分，我主要强调概率方法。例如，通过对错误分布的统计分析，我证明了为什么我的主动算法更鲁棒。通过一个平均的案例分析，我证明了我们的拉格朗日平流的效率。通过概率随机游走，我改进了图像融合。我使用随机扰动进行鲁棒匹配和配准。最后，我还介绍了使用随机过程来可靠地检测早产儿的脑损伤。 在接下来的五年里，我计划引入以下新的组件：（a）将视觉聚焦纳入全景立体;（B）通过研究观众注意力的作用，进一步改进运动捕捉数据压缩;（c）结合显着性（或检测图像中的重要区域），以获得更好的多媒体效果（d）使主动摄像机校准更加稳健;以及（e）开发医疗和手术图像分析的新方法。我的算法的开发和分析将基于彻底的概率技术和分析，就像我过去的研究一样。 这项研究如果成功，将对下一代全景、3D和全光多媒体捕获、处理和传输以及医疗和手术创新产生重大影响。