Human-centric Understanding of 3D Environments

室内环境是人们为了方便人类活动而设计的。碗架紧挨着厨房的水槽，因为我们洗完盘子后会把它们擦干。厨房区域紧挨着餐桌，椅子围绕着桌子，因为我们做饭是为了吃饭，我们作为一个团队一起吃饭。通过这种方式，室内空间的结构和意义取决于我们在其中所采取的行动，以及我们的身体和我们日常生活中与之互动的物体之间的几何图案。不幸的是，没有现有的计算系统能够提供这种以人类为中心的对室内空间3D结构的理解。事实上，当人们由于遮挡而与常见物体互动时，最先进的物体检测和人类动作识别方法难以实现。*这项提议描述了一项研究议程，旨在开发算法，利用人与室内空间之间的联系来理解室内空间的3D结构和可能发生的人类行为。捕捉这种联系的算法对于检测、识别和预测在现实世界中有物体的人的存在和动作很有用。同样的算法还可以更有效地设计和生成室内环境和人类行为的交互式虚拟3D表示。这些3D表示用于建筑可视化、游戏和电影的视觉效果制作、虚拟和增强现实、人体工程学分析和人群模拟。这些行业在蒙特利尔和温哥华等加拿大科技中心都有很强的代表性，目前依赖训练有素的专家的大量手工工作来设计3D内容。*通过创造既能理解并生成3D场景和人类行为的算法，我们将在上述应用领域产生巨大的经济和社会影响。我们还使仿真能够使用逼真的、动态的3D场景来研究3D场景理解和人工智能。计算机图形学、计算机视觉和机器人学中的许多深度学习算法都依赖于大量的3D数据和仿真进行训练。改进的以人为中心的3D环境理解和生成计算系统将为视觉计算、机器学习和AI社区之间的桥梁奠定基础，使我们能够研究和开发通用AI，帮助我们构建成功导航、理解和操纵3D环境的系统。**

Indoor environments are designed by people to facilitate human action. Dish racks are next to kitchen sinks because we dry dishes after washing them. The kitchen area is next to the dining table and chairs are around the table because we cook to eat, and we eat together as a group. In this way, the structure and meaning of interior spaces is determined by the actions we take within them and the geometric patterns between our bodies and the objects with which we interact in our daily lives. Unfortunately, no existing computational systems can provide this level of human-centric understanding of the 3D structure of interior spaces. In fact, state-of-the art methods for object detection and human action recognition struggle when people are observed while interacting with common objects due to occlusions.******This proposal describes a research agenda to develop algorithms that leverage the connection between people and interior spaces to understand the 3D structure of both the interiors and the human actions that can occur within them. Algorithms that capture this connection are useful for detecting, recognizing, and predicting the presence and actions of people with objects in the real world. The same algorithms can also enable more efficient design and generation of interactive virtual 3D representations of interior environments and human actions. These 3D representations are used in architectural visualization, visual effects production for games and films, virtual and augmented reality, ergonomics analysis, and crowd simulation. These industries, which are strongly represented in Canadian technology hubs such as Montreal and Vancouver, currently rely on tremendous manual effort by trained experts to design 3D content.******By creating algorithms that can both understand and generate 3D scenes and human actions we will have tremendous economic and social impact in the above application domains. We also enable simulation to use realistic, dynamic 3D scenes for research in 3D scene understanding and AI. Many deep learning algorithms for tasks in computer graphics, computer vision, and robotics rely on large amounts of 3D data and simulation for training. Improved computational systems for human-centric understanding and generation of 3D environments will lay foundations for bridges between the visual computing, machine learning, and AI communities, enabling us to study and develop general AI that can help us to build systems that successfully navigate, understand and manipulate 3D environments.**