Head-free 3D depth for telecommunication and e-commerce

我们的目标是开发和评估一个免头戴式3D显示系统的原型，以期将我们的技术转移到电子商务和基于视频的通信。视觉深度具有许多不同的功能，并且通过许多不同的视觉质量来传达。智能手机、平板电脑和普通电脑的标准屏幕仅传达图像深度线索，例如遮挡和线性透视。一些更复杂的屏幕设置为立体深度，但通常需要特殊的头盔，以确保不同的图像传输到两个眼睛。然后，立体视觉传达了关于距离的信息，但仍然无法弥补观看屏幕时缺乏方向性的问题。 方向性的缺乏在基于屏幕的电信中变得特别明显。如果人A看向他的相机，人B感觉被看，无论他位于她的屏幕前的什么地方。如果人A没有看着摄像机，她的目光似乎偏离了B的视角。即使B试图通过移动到其线中来捕捉A的注视，偏移也保持不变。缺乏方向性意味着缺少一个重要的沟通渠道，这可能是“变焦疲劳”的主要原因。方向性需要一个非常特殊的视觉质量：运动视差，这意味着视觉场景的方向随着观察者的移动而改变。我们的技术通过使用安装在设备上的传感器跟踪用户在屏幕前的头部位置来实现运动视差，并相应地更新屏幕上呈现的视图。在三个阶段中，我们首先将运动视差应用于被动对象的视觉探索，例如在网上商店（MPdepth I）中展示的商品。在第二阶段，我们实现了一个视觉电信系统，其中两个参与者是由现场动画计算机图形化身（MPDepth II）。最终，在第三阶段，我们用3D视频替换计算机图形内容：将人的多摄像机记录作为视点插值的输入流

We are aiming to develop and evaluate prototypes of a head-free 3D display system in anticipation of transferring our technology to e-commerce and video-based communication. Visual depth serves a number of different functions and is communicated by a number of different visual qualities. Standard screens of smart phones, tablets and regular computers convey only pictorial depth cues, such as occlusion and linear perspective. Some more sophisticated screens are set up for stereoscopic depth, but generally require special headgear to secure delivery of different images to the two eyes. Stereopsis then conveys information about distance but is still not able to mend the lack of directionality experienced when looking at screens. The lack of directionality becomes particularly obvious in screen-based telecommunication. If person A looks into his camera, person B feels being looked at no matter where he is located in front of her screen. If person A is not looking into the camera, her gaze seems to be offset from B's perspective. The offset remains even if B tries to catch A's gaze by moving into its line. Lacking directionality means that an important communication channel is missing, which is likely a main contributor to "Zoom fatigue". Directionality requires a very specific visual quality: motion parallax, which implies that the direction from which a visual scene is seen changes as the observer moves. Our technology implements motion parallax by tracking the user's head location in front of a screen with device-mounted sensors and updates the view presented on the screen accordingly. In three phases, we first apply motion parallax to visual exploration of passive objects, such as merchandise presented in an online shop (MPdepth I). In the second phase, we implement a visual telecommunication system where the two participants are presented by live-animated computer graphic avatars (MPdepth II). In the third phase eventually, we replace computer graphics contents with 3D video: Multicamera recordings of the person are streamed as input for a viewpoint interpolation