Real-Time Modeling and Registration of Dynamic Geometry

预计到2020年，增强现实市场规模将达到1200亿美元。消费级增强现实设备(例如Microsoft HoloLens)的出现受到了能够同时捕获颜色和深度的实时传感器(例如Microsoft Kinect)的技术飞跃的推动。虽然这些传感器提供了一种测量真实世界的3D几何图形的方法，但实时处理几何图形尤其具有挑战性。例如，许多技术只关注人脸、身体或手跟踪等特定子问题，因此经常假设相机观察到的对象具有已知的几何形状。调整这些算法以应对未知的、动态的和不断变化的环境是一个根本的挑战，特别是当目标是具有相互冲突的效率和功耗要求的实时应用时。 为了面对这些挑战，我和我的学生们将研究一种不同于计算机图形和视觉社区通常采用的表示几何的方法。这种表示是所谓的“卷积曲面”的一种特殊变体，如今3D艺术家使用它来高效地设计数字内容的粗略铰接结构。它们的数学特性使其成为虚拟现实应用的理想选择，因为对实时注册算法(即跟踪)至关重要的几何查询可以以极高的效率得到回答。此外，这些模型还可以很容易地适应几何的变化，从而使它们适合代表我们不断变化的世界。 我的研究团队将从三个不同的角度来处理这项调查。我们将研究(1)如何从数字和采集的数据中对卷积曲面进行逆向工程；(2)研究它们在实时消费应用中的用途和可扩展性；以及(3)开发一种具有成本效益的全方位3D采集系统。后者将帮助我们设计统计模型，使消费者应用程序能够处理来自实时3D传感器的低质量数据。 拟议的研究具有广泛的影响潜力，从娱乐(例如视频游戏)到医疗应用(例如虚拟手术)到安全(例如行为监控)等等。任何需要计算机程序有效地表示3D几何的应用场景都可以从我们的研究结果中受益。简单地说，它的潜在影响的地平线与即将到来的增强现实革命的地平线一致。

The augmented reality market is forecast to reach $120 billion by 2020. The advent of consumer-level augmented reality devices (e.g. Microsoft HoloLens) has been fuelled by technological leaps in real-time sensors capable of simultaneously capturing colour and depth (e.g. Microsoft Kinect). While these sensors provide a way of measuring the 3D geometry of the real world, processing geometry in real-time is particularly challenging. For example, many techniques only focus on specific sub-problems like face, body or hand tracking, and thus often assume the object being observed by the camera is of known geometry. Adapting these algorithms to cope with an unknown, dynamic and ever-changing environment is a fundamental challenge, especially when targeting real-time applications with their conflicting requirements of efficiency and power consumption. To face these challenges, my students and I will investigate a way to represent geometry that is different from what is typically adopted by the computer graphics and vision communities. This representation, a particular variant of the so-called “convolution surfaces” is used today by 3D artists to efficiently design the coarse articulated structure of digital content. Their mathematical properties make them ideal for virtual reality applications, as geometric queries essential to real-time registration algorithms (i.e. tracking) can be answered with extreme efficiency. Furthermore, these models can also be easily adapted to changes in geometry, thus making them suitable to represent our ever-changing world. My research team will approach this investigation from three different angles. We will (1) study how to reverse engineer convolution surfaces from digital and acquired data; (2) investigate their use and scalability in real-time consumer applications; and (3) develop a cost-effective omnidirectional 3D acquisition system. The latter will help us devise statistical models allowing consumer applications to cope with poor quality data from real-time 3D sensors. The proposed research has broad impact potential, ranging from entertainment (e.g. video games) to medical applications (e.g. virtual surgery) to security (e.g. behaviour monitoring) among many others. Any applicative scenario requiring a computer program to efficiently represent 3D geometry could benefit from the results in our research. Simply stated, the horizons of its potential impact are aligned with those of the upcoming augmented reality revolution.