Enhanced video for shared reality environments

点击翻译按钮获取中文摘要

In order to bring videoconferencing technology to a new dimension of quality, we are developing the Shared Reality Environment, which supports high-bandwidth, high-fidelity telepresence.  The environment consists of two rooms with video projection, cameras for video acquisition, multichannel sound system for spatilalized audio rendering, and a vibro-mechanical platform to reproduce vibrosensory stimulus at the floor level. Unlike conventional videoconference environments, Shared Reality provides users with a sense of video and audio immersion in a common, shared space.  Remote users appear as if part of the local environment, and can manipulate synthetic (computer-generated) objects in the view.   To improve the quality of Shared Reality experience, this research now seeks to utilize additional cues available from input video in order provide faster, higher-resolution displays that do not suffer from shadows cast by users who block the projector output, and can render views that appropriate to the current participant position(s).  Some of the techniques employed include image mosaicing, robust tracking and gesture acquisition, dynamic background segmentation, and depth estimation from stereo.   Applied together, this research has the potential to support the demands of distributed immersive visualization, such as required by groups of engineers working collaboratively on a complex problem that requires interaction both with each other and the synthetic model.