Towards Seamless Interaction and Navigation in Virtual Worlds using Multiple 3D Sensors

使用多个 3D 传感器实现虚拟世界中的无缝交互和导航

• 批准号: RGPIN-2019-05092
• 负责人: Poovvancheri, Jiju
• 金额: $ 1.68万
• 依托单位: Saint Mary's University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=714932
• 关键词: Towards; Seamless; Interaction; Navigation; Virtual

The key trends report from the Canada Media Fund (CMF) identifies Transmedia (Augmented and Virtual Reality (VR)) as “poised for wide consumer release”. While “VR is still 5 to 10 years from mainstream adoption, content businesses must prepare now if they hope to compete” [excerpts from the CMF report]. The proposed research comprises two relevant problems pertaining to transmedia applications, 3D content (virtual worlds) creation and human interaction with virtual objects for a fully immersive VR experience. Virtual worlds, i.e., polygonal meshes of cities at scale, are increasingly used in VR based applications, e.g., Google Earth VR. In 2018, an area of 50 sq. kms of Helsinki, Finland, was digitized by Umbra technologies targeting at AR, VR gaming int real city settings [www.forbes.com]. We position our research to support the ongoing efforts of the graphics and gaming industry to utilize digitized cities in transmedia applications. The current practices in large scale city digitization include acquiring raw measurements of the physical world using LiDAR sensors and abstracting coarse models from the LiDAR data. However, such coarse building models lack geometric details and hence realism, and do not represent a good choice for 3D navigation, virtual reality and video game creation. So, we investigate into a fully automatic computational framework that creates highly detailed and semantically rich urban models at large scale. The work is a follow up to my research in LiDAR based urban modeling which began in 2015. A core enabler of interaction in VR spaces is real time full body tracking, e.g., a fully-immersive VR game brings human interaction to virtual objects in real time and such interactions call for robust and highly accurate full body tracking systems which are seamlessly deployable. We target highly robust and low-latency full-body motion capture in the presence of interfering geometry (e.g., joystick) using a multi camera system. A multi-camera 3D data acquisition system will be designed and set up, which would open up further avenues of research in futuristic applications such as virtual cloth fitting in online stores, and bio-mechanical studies in sports and fitness. We plan to collect annotated human motion data and make it publicly available for the geometric deep learning community. The result of this research will be valuable to industry practitioners as well as the academic community in developing better practices and tools for virtual world creation and AR/VR applications. Helping the Canadian VR industry to become a key player in the global VR market will result in a positive impact on the local economy, with capital inflow and the associated requirement for high tech workers in the booming VR/AR industry. The proposed LiDAR based reconstruction framework is expected to play a key role in digitizing existing cities around the globe which may be used as 3D content for VR based games or tours and 3D maps for navigation.

加拿大媒体基金（CMF）的主要趋势报告将跨媒体（增强和虚拟现实（VR））确定为“广泛的消费者发布”。虽然“VR距离主流应用还有5到10年的时间，但内容企业如果想要竞争，现在就必须做好准备”（摘自CMF报告）。建议的研究包括两个相关问题，涉及跨媒体应用，3D内容（虚拟世界）创建和人类与虚拟对象的交互，以实现完全沉浸式的VR体验。