Biologically motivated Multimedia

生物动力多媒体

• 批准号: RGPIN-2022-03279
• 负责人: Basu, Anup
• 金额: $ 2.55万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750488
• 关键词: Biologically; motivated; Multimedia

The Human Visual System is characterized by a very high-resolution fovea with rapidly declining resolution in the periphery. The fovea captures details in an approximately 2 degree cone extending from the center of the eyes. Thus, at a given instant in time we can only see a few letters of text clearly at reading distances. In our mind, however, we think that everything is clearly visible. This perception is a result of our eyes being dynamic or "active" and always being guided by the brain to look at precisely what is most important at a given instant. In my basic research, I introduced the concept of foveation for image, video and 3D compression. Furthermore, considering eye movements mimicking human vision, I developed the first active calibration of cameras without using any known patterns or matching feature points. I also introduced new algorithms for distribution learning for background subtraction, saliency detection for identifying regions of interest, and Motion Capture (MoCap) compression to reduce bandwidth requirements. These fundamental research topics have influenced the way coding standards have incorporated "region of interest," resulting in creation of spin-offs and commercialization of Discovery funded research over the years by collaborators and trainees. The overall goal of my research over the next five years is to develop new biologically motivated algorithms following the theme of robust multimedia capture, representation, and communication. In the short-term I intend to make novel advancements in: (i) Active Calibration without inertial sensors; (ii) Robust background subtraction and instance segmentation; (iii) 3D Saliency driven fovea modeling and resulting image capture and compression; (iv) Attention guided Motion Capture (MoCap) data compression; and (v) Mathematical models and learning architectures for addressing these topics. My long-term vision is to make biological factors the core of all multimedia systems, by incorporating them into standards, pursuing the challenges of real world implementations, and commercializing research results. The significance of my work will lie in: (a)Improved object, face and shape recognition; (b)Modeling techniques for multiple and dynamic foveae, and new foveated plenoptic imaging; (c)Improved MoCap transmission strategies incorporating attention of viewers; (d)Robust active calibration of cameras, leading to wider deployment of this approach; (e)New optimization techniques and learning models that can improve the underlying computational framework to obtain better and more robust solutions to scene understanding problems; and, (f)Applications of algorithms and techniques developed through fundamental Discovery research in industry as well as healthcare. The proposed research, if successful, will have significant impact in next generation multimedia capture, processing, and transmission technologies, as well as in medical and surgical innovations.

人类视觉系统的特点是一个非常高分辨率的中央凹，在外围分辨率迅速下降。中央凹从眼睛中心延伸出一个约2度的圆锥体，捕捉细节。因此，在给定的时刻，在阅读距离下，我们只能清楚地看到文本中的几个字母。然而，在我们的头脑中，我们认为一切都是清晰可见的。这种感知是由于我们的眼睛是动态的或“活跃的”，并且总是由大脑引导，在给定的时刻准确地看到最重要的东西。在我的基础研究中，我介绍了图像、视频和3D压缩的注视点概念。此外，考虑到模仿人类视觉的眼球运动，我开发了第一个不使用任何已知模式或匹配特征点的相机主动校准。我还介绍了用于背景减法的分布学习、用于识别感兴趣区域的显著性检测和用于减少带宽需求的运动捕捉（MoCap）压缩的新算法。这些基础研究主题影响了编码标准纳入“感兴趣的领域”的方式，导致合作者和受训者多年来创建了Discovery资助的研究的衍生产品和商业化。在接下来的五年里，我的研究的总体目标是开发新的生物驱动算法，遵循健壮的多媒体捕获、表示和通信的主题。在短期内，我打算在以下方面取得新的进展：(I)无惯性传感器的主动校准；（ii）稳健的背景减法和实例分割；（iii） 3D Saliency驱动的中央凹建模以及由此产生的图像捕获和压缩；（iv）注意力引导运动捕捉（MoCap）数据压缩；（五）解决这些问题的数学模型和学习架构。我的长期愿景是使生物因素成为所有多媒体系统的核心，将它们纳入标准，追求现实世界实现的挑战，并将研究成果商业化。我的工作的意义在于：(a)提高物体、人脸和形状的识别；(b)多中心窝和动态中心窝的建模技术和新的中心窝全光学成像；(c)将观众的注意力纳入改进的动作捕捉传输策略；(d)对照相机进行有力的主动校准，从而更广泛地部署这种方法；(e)新的优化技术和学习模式，可以改进基本的计算框架，以便对场景理解问题获得更好和更可靠的解决办法；(f)通过基础发现研究开发的算法和技术在工业和医疗保健领域的应用。这项研究如果成功，将对下一代多媒体捕获、处理和传输技术以及医疗和外科创新产生重大影响。