Real-Time Modeling and Registration of Dynamic Geometry

增强现实市场预计到2020年将达到1200亿美元。消费级增强现实设备（例如微软HoloLens）的出现受到能够同时捕获颜色和深度的实时传感器（例如微软Kinect）的技术飞跃的推动。虽然这些传感器提供了一种测量真实的世界的3D几何形状的方法，但实时处理几何形状特别具有挑战性。例如，许多技术只关注特定的子问题，如面部、身体或手部跟踪，因此通常假设相机观察到的对象具有已知的几何形状。使这些算法适应未知、动态和不断变化的环境是一个根本性的挑战，特别是当针对实时应用时，它们对效率和功耗的要求相互冲突。为了面对这些挑战，我和我的学生将研究一种表示几何的方法，这种方法与计算机图形学和视觉界通常采用的方法不同。这种表示法是所谓的“卷积表面”的一种特殊变体，如今被3D艺术家用来有效地设计数字内容的粗糙关节结构。它们的数学特性使其成为虚拟现实应用的理想选择，因为实时配准算法（即跟踪）所必需的几何查询可以非常高效地回答。此外，这些模型也可以很容易地适应几何形状的变化，从而使它们适合于代表我们不断变化的世界。我的研究团队将从三个不同的角度来进行这项调查。我们将（1）研究如何从数字和采集的数据中逆向工程卷积曲面;（2）研究它们在实时消费应用中的使用和可扩展性;（3）开发一个具有成本效益的全向3D采集系统。后者将帮助我们设计统计模型，使消费者应用程序能够科普来自实时3D传感器的低质量数据。拟议的研究具有广泛的影响潜力，从娱乐（例如视频游戏）到医疗应用（例如虚拟手术）再到安全（例如行为监测）等等。任何需要计算机程序来有效地表示3D几何形状的应用场景都可以从我们的研究结果中受益。简单地说，其潜在影响的范围与即将到来的增强现实革命的范围是一致的。

The augmented reality market is forecast to reach $120 billion by 2020. The advent of consumer-level augmented reality devices (e.g. Microsoft HoloLens) has been fuelled by technological leaps in real-time sensors capable of simultaneously capturing colour and depth (e.g. Microsoft Kinect). While these sensors provide a way of measuring the 3D geometry of the real world, processing geometry in real-time is particularly challenging. For example, many techniques only focus on specific sub-problems like face, body or hand tracking, and thus often assume the object being observed by the camera is of known geometry. Adapting these algorithms to cope with an unknown, dynamic and ever-changing environment is a fundamental challenge, especially when targeting real-time applications with their conflicting requirements of efficiency and power consumption.***To face these challenges, my students and I will investigate a way to represent geometry that is different from what is typically adopted by the computer graphics and vision communities. This representation, a particular variant of the so-called “convolution surfaces” is used today by 3D artists to efficiently design the coarse articulated structure of digital content. Their mathematical properties make them ideal for virtual reality applications, as geometric queries essential to real-time registration algorithms (i.e. tracking) can be answered with extreme efficiency. Furthermore, these models can also be easily adapted to changes in geometry, thus making them suitable to represent our ever-changing world.***My research team will approach this investigation from three different angles. We will (1) study how to reverse engineer convolution surfaces from digital and acquired data; (2) investigate their use and scalability in real-time consumer applications; and (3) develop a cost-effective omnidirectional 3D acquisition system. The latter will help us devise statistical models allowing consumer applications to cope with poor quality data from real-time 3D sensors.***The proposed research has broad impact potential, ranging from entertainment (e.g. video games) to medical applications (e.g. virtual surgery) to security (e.g. behaviour monitoring) among many others. Any applicative scenario requiring a computer program to efficiently represent 3D geometry could benefit from the results in our research. Simply stated, the horizons of its potential impact are aligned with those of the upcoming augmented reality revolution.**