Development of an Intelligent desk using computer vision and augmented reality

我们开发了增强桌面系统，它将纸质信息和数字信息顺利地集成在一张桌子上。该系统为用户提供了一种智能环境，该环境通过使用计算机视觉自动检索和显示与桌子上的真实对象(例如，书籍)相对应的数字信息。通过引入红外摄像机、归一化相关模式匹配、云台云台摄像机，提出了一种在增强型桌面上实时跟踪手指的新方法。我们通过使用多个摄像头将该方法从二维扩展到三维。为了识别增强型桌面上的物体，我们首先使用了二维条形码。然而，有时很难将条形码附着到某些对象上。为了解决这个问题，我们提出了一种利用物体的颜色信息进行交互式物体配准和配准的新方法。然后通过实时人脸跟踪系统讨论了头部姿势和位置在人与人交互中的应用，最后在增强型办公桌上开发了一些应用程序。第一个应用是互动教科书，旨在提供一个有效的学习环境。第二个应用是交互式维恩图，其目的是支持有效的信息检索。第三个应用是Snap Link，它可以实现交互式对象注册和识别。第四个应用是增强型办公桌上的双手绘图工具

We developed augmented desk system, named Enhanced Desk, which smoothly integrates paper and digital information on a desk. The system provides users an intelligent environment that automatically retrieves and displays digital information corresponding to the real objects (e.g. books) on the desk by using computer vision. The system also provides users direct manipulation of digital information by using their own hands and fingers for more natural and more intuitive interaction.We developed a novel method for real-time finger tracking on Enhanced Desk by introducing a infrared camera, pattern matching with normalized correlation, a pan-tilt camera. We extended the method from 2-D to 3-D by using multiple cameras. In order to recognize objects on Enhanced Desk, we first used 2-D barcode. However, it is sometimes difficult to attach the barcode to some objects. In order to, solve this problem, we developed a new method for interactive object registration and registration by using color information of the objects. Then we discussed the use of head pose and position for human-interaction by using real-time face tracking system.Finally, we developed some applications on Enhanced Desk. The first application is Interactive Textbook, which is aimed at providing an effective learning environment. The second application is the Interactive Venn Diagram, which is aimed at supporting effective information retrieval. The third application is Snap Link, which enable interactive object registration and recognition. The forth application is a two-handed drawing tool on Enhanced Desk