3D Diffusion Models for Generating and Understanding 3D Scenes

扩散模型，如DALL-E2和Imagen，在生成逼真的图像方面取得了显着的成功，并有望解决长期存在的计算机视觉问题。然而，3D场景生成仍然未被探索。本研究项目旨在通过开发能够生成完整3D场景的3D扩散模型来弥合差距。这将推进我们对复杂3D环境中扩散的理论理解，并为虚拟现实、建筑和城市规划等领域的应用开辟新的可能性。提出的3D扩散模型还将提高与3D场景理解相关的计算机视觉任务的准确性，例如对象检测，跟踪和语义分割。

Diffusion models, such as DALL-E2 and Imagen, have achieved remarkable success in generating photorealistic images and hold promise to solve long-standing computer vision problems. However, 3D scene generation remains unexplored. This research project aims to bridge the gap by developing 3D diffusion models capable of generating complete 3D scenes. This will advance our theoretical understanding of diffusion in complex 3D environments and open up new possibilities for applications in fields such as virtual reality, architecture, and city planning. The proposed 3D diffusion models will also enhance the accuracy of computer vision tasks related to 3D scene understanding, such as object detection, tracking, and semantic segmentation.