ITR: Collaborative Research: Development of Head-mounted Projective Display for Distance Collaborative Environments

这是三年持续奖励的第一年资助。虚拟或增强环境的一个要求很高的应用领域是多用户协作环境，其中本地站点或远程站点的多个用户相互通信并与合成或虚拟场景交互。在必要的传感器和设备中，有效的可视化设备和实时图像采集系统是两个主要的挑战。该项目的目标是开发一种新型的可视化设备，即头戴式投影显示器（HMPD），通过将开发的HMPD技术与一种称为全焦相机的独特实时图像采集系统集成，构建一个多用户交互式工作台，并评估和量化该系统作为远程协作的有效工具。提出的头戴式投影显示器（HMPD）与柔软，不扭曲和耐用的投影表面相结合，作为当前可视化设备的替代方案。其新颖的概念解决了当前可视化设备存在的一些问题，如大视场的大畸变、虚拟与真实物体的遮挡矛盾、与背景照明的亮度冲突等。所提出的HMPD的几个特性使其非常适合多用户协作应用程序。通过引入衍射光学元件（DOE）和塑料材料，研究设计和实现轻量化和紧凑型头戴式原型机，并研究优化显示照明和成像用反反射材料性能的方法。在一个站点，将使用HMPD概念并添加从独特的全焦相机系统开发的图像采集系统，构建具有远程呈现功能的多用户交互式工作台原型。在另一个场地，将建立一个壁画展示系统，在壁画展示附近使用传统的立体视频系统，一个或几个合作者也将聚集在那里。远程协作工作将在伊利诺伊大学厄巴纳香槟分校的贝克曼研究所和中佛罗里达大学光学学院之间通过连接我们实验室的Internet II连接进行测试。最后，pi将量化深度和尺寸表示以及感知精度，评估遮挡感知方面，并为HMPD和工作台和壁画原型建立全面的校准程序。预计其结果将影响广泛的应用，如协作/远程协作、远程呈现、远程操作和可视化教育/远程教育。

This is the first year funding of a three-year continuing award. A demanding application area of virtual or augmented environment is multi-user collaborative environment where multiple users at either a local site or remote sites communicate with each other and interact with a synthetic or virtual scene. Among the necessary sensors and devices, an effective visualization device and a real-time image acquisition system are two main challenges. The objective of this project is to develop a novel visualization device referred to as head-mounted projective display (HMPD), build a multi-user interactive workbench by integrating the developed HMPD technology with a unique real-time image acquisition system known as an omni-focus camera, and evaluate and quantify the system as an effective tool for remote collaboration. The head-mounted projective display (HMPD) proposed is coupled with a supple, non-distorting and durable projection surface as an alternative to current visualization devices. Its novel concept suggests solutions to part of the problems of state-of-art visualization devices, such as large distortion with wide field of view, occlusion contradiction between virtual and real objects, and brightness conflict with background illumination. Several properties of the proposed HMPD make it extremely suitable for multiple-user collaborative applications. Research efforts will be made to design and implement a lightweight and compact head-mounted prototype by introducing diffractive optical element (DOE) and plastic materials, and investigate approaches to optimize the illumination of the display and retro-reflective material properties for imaging purpose. At one site, a multi-user interactive bench prototype with tele-presence capability will be built by using the HMPD concept and adding an image acquisition system developed from a unique omni-focus camera system. At the other site, a mural display system will be built with conventional stereoscopic video system located near the mural display, where one or several collaborators will also gather. Tele-collaborative work will be tested between the Beckman Institute at the University of Illinois--Urbana Champaign and the School of Optics-CREOL at the University of Central Florida through the Internet II connection linking our laboratories. Finally, the PIs will quantify the depth and size representation and perception accuracy, evaluate occlusion perception aspects, and set up a comprehensive calibration procedure for the HMPD and the workbench and mural prototypes. The results are expected to impact a wide range of applications such as collaboration/tele-collaboration, tele-presence, tele-manipulation, and visualized education/tele-education.