Head-free 3D depth for telecommunication and e-commerce

适用于电信和电子商务的无头 3D 深度

• 批准号: 557864-2020
• 负责人: Troje, Nikolaus
• 金额: $ 9.11万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Idea to Innovation
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=706383
• 关键词: Head; free; 3D; depth; telecommunication

We are aiming to develop and evaluate prototypes of a head-free 3D display system in anticipation of transferring our technology to e-commerce and video-based communication. Visual depth serves a number of different functions and is communicated by a number of different visual qualities. Standard screens of smart phones, tablets and regular computers convey only pictorial depth cues, such as occlusion and linear perspective. Some more sophisticated screens are set up for stereoscopic depth, but generally require special headgear to secure delivery of different images to the two eyes. Stereopsis then conveys information about distance but is still not able to mend the lack of directionality experienced when looking at screens. The lack of directionality becomes particularly obvious in screen-based telecommunication. If person A looks into his camera, person B feels being looked at no matter where he is located in front of her screen. If person A is not looking into the camera, her gaze seems to be offset from B's perspective. The offset remains even if B tries to catch A's gaze by moving into its line. Lacking directionality means that an important communication channel is missing, which is likely a main contributor to "Zoom fatigue". Directionality requires a very specific visual quality: motion parallax, which implies that the direction from which a visual scene is seen changes as the observer moves. Our technology implements motion parallax by tracking the user's head location in front of a screen with device-mounted sensors and updates the view presented on the screen accordingly. In three phases, we first apply motion parallax to visual exploration of passive objects, such as merchandise presented in an online shop (MPdepth I). In the second phase, we implement a visual telecommunication system where the two participants are presented by live-animated computer graphic avatars (MPdepth II). In the third phase eventually, we replace computer graphics contents with 3D video: Multicamera recordings of the person are streamed as input for a viewpoint interpolation

我们的目标是开发和评估无头 3D 显示系统的原型，以期将我们的技术转移到电子商务和基于视频的通信领域。视觉深度具有多种不同的功能，并通过多种不同的视觉质量来传达。智能手机、平板电脑和普通电脑的标准屏幕仅传达图像深度线索，例如遮挡和线性透视。一些更复杂的屏幕是针对立体深度而设置的，但通常需要特殊的头饰来确保将不同的图像传输到两只眼睛。立体视觉可以传达有关距离的信息，但仍然无法弥补观看屏幕时缺乏方向性的问题。 在基于屏幕的电信中，缺乏方向性变得尤为明显。如果 A 看着他的相机，B 就会感觉到无论他位于屏幕前的哪个位置，他都会被注视。如果 A 没有看着镜头，她的视线似乎会偏离 B 的视角。即使 B 试图通过移动到 A 的线中来吸引 A 的目光，偏移量仍然存在。缺乏方向性意味着缺少重要的沟通渠道，这可能是“Zoom疲劳”的主要原因。方向性需要非常具体的视觉质量：运动视差，这意味着视觉场景的方向随着观察者的移动而变化。我们的技术通过使用设备安装的传感器跟踪屏幕前的用户头部位置来实现运动视差，并相应地更新屏幕上呈现的视图。分三个阶段，我们首先将运动视差应用于被动对象的视觉探索，例如在线商店中展示的商品（MP深度 I）。在第二阶段，我们实现了一个视觉电信系统，其中两个参与者由实时动画计算机图形化身（MPdepth II）呈现。最终，在第三阶段，我们用 3D 视频取代计算机图形内容：人物的多机位录像作为视点插值的输入进行流式传输