Computer vision algorithms for live video processing using programmable graphics hardware

点击翻译按钮获取中文摘要

Today's technology is increasingly powerful and affordable. For example, an 800 dollar graphics card today can process one Trillion FLOPs in double precision. Merely a decade ago, such processing power would come with a one million dollar price tag. Similarly, digital video cameras at the time were only available to movie producers, whereas today they are in the hands of billions of users, as well as integrated into phones, vehicles, and game consoles. The availability of low cost processing power and digital video capturing devices allow computer vision techniques to affect and benefit our day to day lives. They are making our phones smarter, our vehicles safer, and our game consoles much more fun to interact with. This research investigates how to perform challenging computer vision tasks on live video at real-time speed. The tasks include inferring depth from video sequences captured from different viewpoints (stereo vision), detecting moving foreground objects from dynamic backgrounds (foreground segmentation), and extracting objects with fuzzy boundaries for seamlessly blending with new backgrounds (video matting). Being able to perform these tasks for live video has a widely range of applications in our daily life. For example, real-time stereo vision can be employed for sensing the 3D environment and foreground segmentation for detecting pedestrians; both are key components of the future driverless cars. Foreground segmentation and video matting can be used to extract video conference participants from captured video and place them into the same virtual environment, providing better sense of presence and better protection of privacy. The goal of the research is to develop novel algorithms that are not only fast enough for handling live video, but also having better or comparable performance to the state-of-the-art offline algorithms. The algorithms to be developed therefore need to be both effective and efficient. In addition, to harvest the processing power of modern graphics hardware, these algorithms will be designed with parallel execution in mind. Some preliminary work along this research direction has yielded very promising results.