CHS: Small: Learning to Automatically Design Interior Spaces

人们生活中的大部分时间都在室内度过，卧室，客厅，办公室，厨房等，对这些空间的虚拟版本的需求从未如此之高;机器人，计算机视觉，建筑，室内设计，虚拟现实和增强现实-所有这些领域都需要创建真实世界室内场景的高保真数字实例。为了满足这一需求，该项目将开发新的室内场景生成模型，可以快速合成新的环境。为了实现这一目标，一个场景合成系统应该是数据驱动的，能够快速生成各种似是而非的和视觉上吸引人的结果，并且是用户可控的。虽然先前的工作已经解决了室内场景合成，没有现有的方法满足所有这些要求。该项目不仅将实现这一目标，还将努力使用新的软件系统来训练机器人导航。 通过与两家家具和室内设计公司合作进行工业技术转让，将加强项目成果的更广泛影响。 该研究将创建免费的在线演示，并将吸引和指导女学生作为研究助理。设想系统的第一个组成部分将是一个基于深度卷积神经网络的新场景生成模型，该模型将基于平面图的详细的基于图像的场景表示与基于对象关系图的离散的场景符号表示相统一，从而获得两者的益处以生成各种似真的场景。图形和图像上的卷积将用于根据场景中的相关空间背景做出合成决策;生成的模型将是快速、可控和完全数据驱动的。该系统的第二个组成部分将是一个模型的视觉兼容性的场景对象，这是必要的生成视觉吸引力的场景。该模型将利用卷积网络来分析场景的渲染视图，捕捉场景和其中物体的视觉外观;该网络将在专业设计的室内场景的新数据集上进行训练。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

People spend a large part of their lives indoors, in bedrooms, living rooms, offices, kitchens, etc. The demand for virtual versions of these spaces has never been higher; robotics, computer vision, architecture, interior design, virtual and augmented reality -- all of these fields need to create high-fidelity digital instances of real-world indoor scenes. To meet this need, this project will develop new generative models of indoor scenes that can rapidly synthesize novel environments. To achieve this goal, a scene synthesis system should be data driven, be able to quickly generate a variety of plausible and visually appealing results, and be user-controllable. While prior work has addressed indoor scene synthesis, no existing approach satisfies all of these requirements. Not only will this project achieve that goal, it also includes efforts to use the new software system for training robots to navigate. Broader impact of project outcomes will be enhanced through industrial technology transfer in collaboration with two furniture and interior design companies. The research will create freely available online demos, and will engage and mentor female students as research assistants.The first component of the envisaged system will be a new scene generative model based on deep convolutional neural networks that unifies a detailed, image-based representation of scenes based on floor plans with a discrete, symbolic representation of scenes based on object relationship graphs, thereby gaining the benefits of both to generate a variety of plausible scenes. Convolutions on both graphs and images will be employed to make synthesis decisions based on the relevant spatial context in the scene; the resulting model will be fast, controllable, and fully data driven. The system's second component will be a model of the visual compatibility of scene objects, which is necessary for generating visually appealing scenes. This model will exploit a convolutional network to analyze rendered views of the scene, capturing the visual appearance of the scene and objects in it; the network will be trained on a new dataset of professionally designed interior scenes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.