Human-centric Understanding of 3D Environments

点击翻译按钮获取中文摘要

Indoor environments are designed by people to facilitate human action. Dish racks are next to kitchen sinks because we dry dishes after washing them. The kitchen area is next to the dining table and chairs are around the table because we cook to eat, and we eat together as a group. In this way, the structure and meaning of interior spaces is determined by the actions we take within them and the geometric patterns between our bodies and the objects with which we interact in our daily lives. Unfortunately, no existing computational systems can provide this level of human-centric understanding of the 3D structure of interior spaces. In fact, state-of-the art methods for object detection and human action recognition struggle when people are observed while interacting with common objects due to occlusions.******This proposal describes a research agenda to develop algorithms that leverage the connection between people and interior spaces to understand the 3D structure of both the interiors and the human actions that can occur within them. Algorithms that capture this connection are useful for detecting, recognizing, and predicting the presence and actions of people with objects in the real world. The same algorithms can also enable more efficient design and generation of interactive virtual 3D representations of interior environments and human actions. These 3D representations are used in architectural visualization, visual effects production for games and films, virtual and augmented reality, ergonomics analysis, and crowd simulation. These industries, which are strongly represented in Canadian technology hubs such as Montreal and Vancouver, currently rely on tremendous manual effort by trained experts to design 3D content.******By creating algorithms that can both understand and generate 3D scenes and human actions we will have tremendous economic and social impact in the above application domains. We also enable simulation to use realistic, dynamic 3D scenes for research in 3D scene understanding and AI. Many deep learning algorithms for tasks in computer graphics, computer vision, and robotics rely on large amounts of 3D data and simulation for training. Improved computational systems for human-centric understanding and generation of 3D environments will lay foundations for bridges between the visual computing, machine learning, and AI communities, enabling us to study and develop general AI that can help us to build systems that successfully navigate, understand and manipulate 3D environments.**