Real-Time Modeling and Registration of Dynamic Geometry

点击翻译按钮获取中文摘要

The augmented reality market is forecast to reach $120 billion by 2020. The advent of consumer-level augmented reality devices (e.g. Microsoft HoloLens) has been fuelled by technological leaps in real-time sensors capable of simultaneously capturing colour and depth (e.g. Microsoft Kinect). While these sensors provide a way of measuring the 3D geometry of the real world, processing geometry in real-time is particularly challenging. For example, many techniques only focus on specific sub-problems like face, body or hand tracking, and thus often assume the object being observed by the camera is of known geometry. Adapting these algorithms to cope with an unknown, dynamic and ever-changing environment is a fundamental challenge, especially when targeting real-time applications with their conflicting requirements of efficiency and power consumption.***To face these challenges, my students and I will investigate a way to represent geometry that is different from what is typically adopted by the computer graphics and vision communities. This representation, a particular variant of the so-called “convolution surfaces” is used today by 3D artists to efficiently design the coarse articulated structure of digital content. Their mathematical properties make them ideal for virtual reality applications, as geometric queries essential to real-time registration algorithms (i.e. tracking) can be answered with extreme efficiency. Furthermore, these models can also be easily adapted to changes in geometry, thus making them suitable to represent our ever-changing world.***My research team will approach this investigation from three different angles. We will (1) study how to reverse engineer convolution surfaces from digital and acquired data; (2) investigate their use and scalability in real-time consumer applications; and (3) develop a cost-effective omnidirectional 3D acquisition system. The latter will help us devise statistical models allowing consumer applications to cope with poor quality data from real-time 3D sensors.***The proposed research has broad impact potential, ranging from entertainment (e.g. video games) to medical applications (e.g. virtual surgery) to security (e.g. behaviour monitoring) among many others. Any applicative scenario requiring a computer program to efficiently represent 3D geometry could benefit from the results in our research. Simply stated, the horizons of its potential impact are aligned with those of the upcoming augmented reality revolution.**