Biologically motivated Multimedia

点击翻译按钮获取中文摘要

The Human Visual System is characterized by a very high-resolution fovea with rapidly declining resolution in the periphery. The fovea captures details in an approximately 2 degree cone extending from the center of the eyes. Thus, at a given instant in time we can only see a few letters of text clearly at reading distances. In our mind, however, we think that everything is clearly visible. This perception is a result of our eyes being dynamic or "active" and always being guided by the brain to look at precisely what is most important at a given instant. In my basic research, I introduced the concept of foveation for image, video and 3D compression. Furthermore, considering eye movements mimicking human vision, I developed the first active calibration of cameras without using any known patterns or matching feature points. I also introduced new algorithms for distribution learning for background subtraction, saliency detection for identifying regions of interest, and Motion Capture (MoCap) compression to reduce bandwidth requirements. These fundamental research topics have influenced the way coding standards have incorporated "region of interest," resulting in creation of spin-offs and commercialization of Discovery funded research over the years by collaborators and trainees. The overall goal of my research over the next five years is to develop new biologically motivated algorithms following the theme of robust multimedia capture, representation, and communication. In the short-term I intend to make novel advancements in: (i) Active Calibration without inertial sensors; (ii) Robust background subtraction and instance segmentation; (iii) 3D Saliency driven fovea modeling and resulting image capture and compression; (iv) Attention guided Motion Capture (MoCap) data compression; and (v) Mathematical models and learning architectures for addressing these topics. My long-term vision is to make biological factors the core of all multimedia systems, by incorporating them into standards, pursuing the challenges of real world implementations, and commercializing research results. The significance of my work will lie in: (a)Improved object, face and shape recognition; (b)Modeling techniques for multiple and dynamic foveae, and new foveated plenoptic imaging; (c)Improved MoCap transmission strategies incorporating attention of viewers; (d)Robust active calibration of cameras, leading to wider deployment of this approach; (e)New optimization techniques and learning models that can improve the underlying computational framework to obtain better and more robust solutions to scene understanding problems; and, (f)Applications of algorithms and techniques developed through fundamental Discovery research in industry as well as healthcare. The proposed research, if successful, will have significant impact in next generation multimedia capture, processing, and transmission technologies, as well as in medical and surgical innovations.