Towards Accurate Detection, Segmentation, and Tracking of Objects and other Scene Attributes

点击翻译按钮获取中文摘要

There is benefit in being able to identify, segment, and track individual objects in video, including rigid and articulated objects (e.g. humans, animals, cars, etc.), background elements (sky, clouds, trees, etc.), markers (i.e. objects specifically design to show as high-contrast to a camera), and feature points (fixed, high-contrast pixels, usually corners, that can be easily tracked over multiple frames and used as reference points). Apart from being able to accurately assign each pixel to a specific object, it is useful to determine where they are placed (in true 3D space if possible), and keep track of them through multiple video frames even as their appearance changes or they are partially/fully occluded by other objects as they move through the scene. This technology is used in a range of applications, including: video compression, depth estimation from single and stereo cameras (e.g. robotics), multi-camera stitching (to compensate for parallax), match-moving for visual effects, automatic camera calibration systems, marker-based and marker-less motion capture, and autonomous vehicle vision systems to name a few. The limitation of current systems is that it is difficult to extract the exact outline that distinguishes between two objects (especially as one pixel can represent a blending of both). The reasons for this include: lack of contrast, colour variation, texture, or lighting; frequent shifts in camera perspective and object orientation; motion blur (due to a fast moving scene and/or camera, or low light - meaning slower shutter speeds); light, shadow, and reflection variations (both under static and changing conditions), and various other smaller artifacts that are inherent within the camera (including sensor noise and simply the fact most sensors are Bayer-tile type and thus frames are constructed through demosaicing). In addition, most applications require real-time processing so that other systems can react to the information (e.g. autonomous vehicles, perimeter security). Therefore, identifying individual objects, determining their shape and outlines to sub-pixel accuracy and accurately tracking them in a sequence of frames is crucial for many mission-critical applications, especially under varying adverse conditions (e.g. snow or rain), without the need to rely on additional costly technology (such as LIDAR/RADAR) In this research program, we propose a range of new and complementary research directions based on a combination of traditional video processing techniques and deep learning methods that can be used to improve the accuracy of the current state of the art and deal with the more varied and complex conditions that can be expected in the application areas especially where the conditions cannot be controlled, but where they can be relied upon and trusted to produce precise real-time results.