EAGER: Viewpoint Tracking via Acceleration Stabilized with Computer Vision

这个项目解决了一个阻碍虚拟现实广泛使用的问题。大约15年前，pc中的高性能显卡将虚拟环境的计算成本从数十万美元降低到（今天）数百美元。低成本的头戴式显示器刚刚出现。在视点跟踪方面还没有出现类似的进展；准确、低延迟的广域视点跟踪仍然非常昂贵。虚拟现实需要每只眼睛每秒60帧的立体声和低于50毫秒的系统延迟。这项研究正在开发一种新颖的系统，以提供准确、低延迟的视点跟踪，以满足消费者成本组件的这些要求。这项研究是基于最近演示的概念验证系统。一个标准的rgb深度摄像头安装在用户的头上。姿态是通过将图像与环境模型相匹配来计算的。卡尔曼滤波器集成了来自廉价高速惯性测量单元（IMU）的旋转速度和线性加速度，在帧间多次更新姿态估计。这不仅可以提供低延迟的姿势读数，还可以改善下一个相机计算的初始值。同时使用深度图像和重建软件增量构建/更新环境的深度模型，用于相机匹配。目前的研究正在证明该系统的潜力。为了完全成功地工作，概念和算法的进步都在进行中。国际计量单位的校准正在得到改进。在校准中必须补偿温度和动态偏差，以改善估计和减少抖动。使用多个摄像头来降低整体噪音和处理困难情况（如空白墙）正在通过新的算法得到解决和评估。合并新数据和建模数据在计算上是昂贵的。可行性演示使用两个gpu，一个用于渲染；一个用于跟踪。人们正在发明一种方法来做到这一点。未来的其他研究包括跟踪动态物体和结合物体识别（例如，桌子或椅子）来提高估计。虚拟现实的广泛使用很可能会开启这项技术意想不到的创新应用。该研究正在发明概念验证系统，以实现这一令人兴奋的飞跃。

This project addresses a problem which has kept virtual reality from widespread use. Some 15 years ago high-capability graphics cards in PCs reduced the cost of computing for virtual environments from hundreds-of-thousands of dollars to (today) hundreds of dollars. Low-cost head-mounted displays have just appeared. The similar advance in viewpoint tracking has not occurred; accurate, low-latency wide-area viewpoint tracking remains very costly. Virtual reality demands stereo 60 frames per second per eye and system latencies below 50 ms. This research is developing a novel system to provide accurate, low-latency viewpoint tracking to meet these requirements with consumer-cost components. The research is based upon a recently demonstrated proof-of-concept system. A standard RGB-Depth camera sits on the user's head. Pose is calculated by matching images against an environment model. A Kalman filter integrates rotational velocity and linear acceleration from a cheap high-speed inertial measurement unit (IMU) to update the pose estimate many times between frames. This not only gives low-latency pose readings, it also improves initial values for the next camera calculation. The depth images and reconstruction software are concurrently used to incrementally build/update the depth model of the environment for the camera matching. The current research is demonstrating the system's potential. To work completely successfully, both conceptual and algorithmic advances are in process. IMU calibrations are being improved. Temperature and dynamic bias must be compensated in the calibration to improve stimation and reduce jitter. Using multiple cameras to reduce overall noise and handle difficult cases (such as blank walls) are being addressed with new algorithms and evaluated. The merging of new and modeled data is computationally expensive. The feasibility demo uses two GPUs, one for rendering; one for tracking. Ways are being invented to do it on one. Additional future research includes tracking dynamic objects and incorporating object recognition (e.g., such as a desk or chairs) to improve estimates. Widespread access to virtual reality may well open new, unexpected creative uses of the technology. The research is inventing the proof-of-concept system forward to one that can make this exciting leap.