3D human tracking and scene reconstruction for audio-visual AR/VR

视听 AR/VR 的 3D 人体跟踪和场景重建

• 批准号: 2480933
• 金额: --
• 依托单位: University of Southampton
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2480933
• 关键词: 3D; human; tracking; scene; reconstruction

In this research, the student will investigate identification and tracking of people throughout complex multi-person scenes using audio-visual input from a camera for AR/VR applications. Real-time 3D scene rendering and spatial audio reproduction algorithms require listener location and head pose (sweet spot) related to the display/loudspeakers and room geometry. Real-time scene tracking and adaptive 3D scene rendering will be the goal of this project. Two types of display scenarios will be considered. First one is a personalised Head-Mounted Display (HMD) system such as a VR headset with headphones. The second type is to use a communal display (e.g., TV or car windscreen) with loudspeakers.

在这项研究中，学生将研究使用来自AR/VR应用的摄像头的视听输入在复杂的多人场景中识别和跟踪人。实时3D场景渲染和空间音频再现算法需要与显示器/扬声器和房间几何形状相关的收听者位置和头部姿势（最佳点）。实时场景跟踪和自适应三维场景渲染将是本项目的目标。将考虑两种类型的显示场景。第一种是个性化的头戴式显示器（HMD）系统，例如带有耳机的VR头戴式耳机。第二种类型是使用公共显示器（例如，电视或汽车挡风玻璃）与扬声器。